Petteri's Pontifications
My musings about photography, mostly.
RSS feed
How To Test A Lens

How To Test A Lens

Lens testing is fun and informative. While it won't turn you into a Cartier-Bresson, it will have a visible impact on your photography too: if you understand the way your lenses behave, you'll be able to make the most of them under all circumstances. For example, you might have one lens that you know to work well across the entire aperture range, and another that's otherwise OK but starts to break down wide-open at maximum tele. Knowing this, you could make the intelligent decision to use the first lens wide-open for a shorter shutter speed, to minimize camera shake -- but stop down the second lens, risking camera shake to get rid of the lens softness, or bumping up the ISO a notch, knowing that the added noise will hurt less than the lens softness.

There are two sets of characteristics that you need to consider when examining a lens: build and optical. The build characteristics are too often ignored, although in my opinion they have much more of an immediate impact on photography than the optical ones -- they can either get or lose you the shot, while the optical quality only determines how the picture looks once it's in the box. And personally, I'd rather have the shot with a slightly lower quality than miss it altogether.

Build characteristics

Build characteristics describe the physical properties of a lens: the design decisions used to construct it, the features put on it, and the way it's been put together. Build determines what the lens can be used for at least as much as optics: if it's very big and heavy, you might not be able to (or want to) carry it where the pictures are, but if it's very light and flimsy, it might not handle well or be able to take the abuse in the field. If it focuses slowly, you won't be able to get consistent quick shots of fast-moving subjects. If it's a pump zoom, it'll cause you annoyance when shooting off a tripod, as the zoom will creep. Some characteristics to look for:

Size and weight

In my opinion, the most important build characteristic, as it can make the difference of having the lens with you when it counts... or not. Lighter and smaller is usually better (for us amateurs anyway), although a heavier lens can handle and balance better than a light one -- just the same as a camera body.

Fit and finish

Best: Lens feels like a Mercedes. Metal barrel, no creaks or squeaks, focus and zoom mechanism moves smoothly and easily, heavy.

Good: Lens feels like a Volkswagen. Metal or plastic barrel, slight slop or roughness in focus ring, but overall tight and "together" feel. No obvious artifacts such as cast seams visible.

Bad: Lens feels like a Lada. All-plastic, moving parts jiggle, squeak, and slop around, visible cast seams. It's likely that it'll develop quirks over time, like serious "zoom creep," may not focus very accurately, and will certainly not take much abuse.

Ramifications: A well-fitted lens will most likely last longer... and it's much more fun to use. A sloppy one is OK for an occasional-use one, but if you use it a lot, it'll probably break down at some point, meaning that you'll need a new one -- so you might've as well bought the better built one in the first place, even used.

Focusing motor

Best: ring USM, HSM, or other very fast and silent focus motor, with real-time manual focus (focus can be tweaked even with AF engaged). Found only on the best and most expensive lenses, such as most current L's.

Good: Fast non-ring-USM focusing motor, usually without RT-M. This could be AFD, micro USM, or even DC. Found on mid-priced lenses, such most consumer lenses with USM. Small lenses with DC are louder but otherwise no worse than USM ones!

Bad: Slower and louder focusing motor without RT-M. This could be a DC motor on a bigger lens (such as the 100-300/5.6L) or an otherwise inadequately powerful motor (such as the one on my Tokina 17).

Ramifications: for most everyday subjects, any AF motor is good enough. It starts to make a real difference if you're shooting stuff like birds, wildlife, or sports -- you need to lock on fast and track accurately, or get a lot of missed shots. The longer the lens, the more important the AF motor: on an ultra-wide, you should probably manual-focus anyway, as there's a huge amount of depth of field and the issue becomes getting your whole scene into it rather than just your subject, whereas on a long tele depth of field is very narrow and apparent subject motion is fast -- a slow AF motor will translate to more missed shots.

To test the focusing motor, simply point the lens around and get a feel for how fast it responds. It helps to have a "reference" lens with which you can repeat the experiment: if it's not really bad, you'll usually adapt to the lens's speed without even noticing it.

Focusing mechanism

Best: internal focusing. This means the lens's physical dimensions won't change when it focuses, and means that it's more resistant to dust and moisture.

Good: Non-rotating front element. This means the front element moves back and forth but doesn't rotate. It's good because it means that polarizers won't turn as the camera focuses.

Not-so-good: Rotating front element with dedicated focusing ring. This will make using polarizers a headache, but won't otherwise much affect handling.

Bad: Rotating front element, no focus ring. This means that the lens is unpleasant to focus manually. Usually found on the cheapest consumer zooms.

Ramifications: A normal-range lens where you're likely to want to use a polarizer does suffer a significant usability hit if the front element rotates. Otherwise, I don't consider this a terribly important characteristic.

Zoom mechanism

Best: Internal ring zoom. You zoom by twisting a ring, and the lens's physical dimensions don't change. This means better environmental resistance and more consistent balance.

Good: Ring zoom with moving front elements. You zoom by twisting a ring, and the front of the lens pistons out or in.

Bad: Pump zoom. You zoom by pushing and pulling the lens, directly stretching and compressing it. This is less precise and pleasant than either of the above.

Ramifications: Doesn't make that much difference in usability, although an internal ring zoom lens feels nicer. The main issue with the pump zoom is that the zoom setting will "creep" if you're shooting off a tripod and your camera isn't perfectly horizontal.

Lens mount

Best: Metal. This means that the lens is less likely to break if you swing your camera around from it (a bad idea in general, unless the lens is heavier than the camera).

Bad: Plastic.

Ramifications: In my opinion, the actual importance of this characteristic is more symbolic than real -- a lens with a plastic mount is likely to be pretty shoddily built in the first place.

Other details

Depth of field scale, barrel markings, IS, other special characteristics or eccentricities (like weather seals on "L" lenses or the focus clutch system on the Tokina AT-X Pros)... Lenses have cute things all over the place, and there's no way to list them all. But if something strikes you as unusual or interesting, it may be worth noting.

How do lenses behave?

Almost all modern lenses are capable of getting good images under some conditions. The difference between a mediocre lens and a superb one is that the superb one will get good quality under all conditions in which it can be used to take pictures in the first place. However, most lenses tend to behave in a similar way under different conditions. This makes testing somewhat easier.

"The sweet spot"

All lenses have a "sweet spot" where they work at their best: within it, there are little or no changes in optical characteristics. Outside it, some of the characteristics start to break down. Generally, the better the lens, the less things break down. One important purpose of lens testing is to find out how big the sweet spot is, where it is, and what happens outside it.

You can think of the sweet spot as a blob suspended in space. On one axis, we have aperture. On another, focusing distance. On a third, focal length. (The latter doesn't apply to primes, of course.) That is, there is a certain area of aperture, focal length, and focus distance combinations within which the lens is at its best.

Effects of aperture

One (as far as I know) unbreakable rule of optics is that lenses do not perform as well wide-open as stopped-down. Bugaboos like halation, light fall-off, and corner softening show up to some degree -- very little on the best lenses, but some on all of them. On the other hand, lenses peak at some aperture, maintaining their excellent optical characteristics until diffraction kicks in and starts softening the image again -- at f/16 or so with digital sensors, f/22 or so with film. By varying aperture, it's possible to find out where the lens peaks. It's usually about two stops down from wide-open -- f/2.8 on an f/1.4 lens, f/11 on an f/5.6 one. Better lenses peak earlier. A very few can be said to peak wide-open.

To get an idea of how very good lenses behave at different apertures, take a look at the Fifty versus Fifty shootout.

Focal length (zoom range)

Usually zooms are best near the middle of their range, but this depends on lens design. They are almost always at their worst at their maximal focal length: enlarging the image also enlarges any problems the lens may have. Therefore, it's important to pay special attention to the way the lens behaves near its maximum tele setting. Ultra-wide-angle zooms are sometimes an exception: there the problem becomes holding the very wide scene together, so they tend to break down near the short end.

Focus distance

Lenses usually perform better close-up or at medium distance than near-infinity. This is one reason why "newspaper tests" don't give a realistic idea of how a lens performs in real-life situations.

Distance from center of frame

Another (as far as I know) unbreakable rule of optics is that things are better near the center of the frame than at the edges. A hallmark of a really good lens is that image quality holds up all the way to the edges. Especially wide-angle lenses tend to go soft in the corners, and exhibit other problems there as well. Therefore, it's critically important to examine both the center and a corner of each frame, especially for sharpness (resolution and contrast) and chromatic aberration.

What is a good lens?

The answer to this question depends on your frame of reference and intended use. For example, if you're looking for a good 50 mm prime, the competition is so hard that you'd consider some legendary lenses at other focal lengths pretty mediocre. But if you're looking for a good normal-range zoom for under $500, you'd probably settle for good sharpness, acceptable contrast, more or less neutral color, and flare and halation kept more or less under control. If you want to use your lens for portraits, you'd care more about color and contrast than sharpness and flare. If you want it for architecture, you'd want maximal sharpness and minimal distortion. So when doing the tests and making your judgments, remember to keep in mind the purpose of the game -- and weight your assessments accordingly. However, let's first look at the various optical characteristics that lenses have.

Sharpness

Sharpness is a function of two characteristics: resolution and contrast, aka accutance. Resolution determines how fine detail a lens can see. Contrast determines how small a lightness difference a lens can see. A lens with high resolution and low contrast may have a flat, muddy, and lackluster look, like a scene seen through a dirty windowpane. A lens with high contrast and low resolution will produce scenes with a lot of pop, depth, and clarity... but if you enlarge them a lot, the small detail will go fuzzy. In general, I'd tend to prefer a lens with high contrast and low resolution over one with high resolution and low contrast.

A crop with high contrast but lowered resolution. In this case, it's a field curvature issue, though. This level of softness actually has very little impact on final image quality, even at fairly large print sizes. People do notice the color and contrast. Only photographers mash their face to the print close enough to catch it.

A crop with high resolution but low contrast: shot with a consumer-grade lens stopped-down. Even small prints will appear somehow undefinably lackluster compared to prints from better lenses.

A crop with both high resolution and contrast. This level of quality both has pop for all print sizes and enlarges extremely well.

Assessing sharpness: To assess sharpness, look at how small detail and fine gradations of light and dark are rendered. Do things look soft or smeared? That means low resolution. Do they look flat and muddy? That indicates low contrast. Contrast is especially tricky to assess from real-life shots, as different scenes contain different amounts of contrast to start with: a scene shot on a slightly hazy day with a high-contrast lens will look much the same as a scene shot with a low-contrast lens on a clear day. Ideally, shoot a reference frame or two of the test scene with a lens whose characteristics you know. (Many professional reviewers use the 50/1.4 for this purpose.)

Sharpness is one characteristic that can vary a great deal across the frame. A lens that has even contrast and resolution over the entire frame is much more desirable than one that's sharp in the center but soft in the corners, as it responds to post-processing better. It's actually pretty tricky to assess this quality. The reason is that lens problems aren't the only possible reason for soft corners. Especially with ultra-wides, the depth of the scene and field curvature come into play as well. Very few lenses have perfectly flat fields: instead, the focus distance curves somewhat. This has a couple of implications. For one thing, a newspaper test will show field curvature as soft corners. With real-life scenes, the field may not be where you expect it to be, and objects that you think would be within it are not -- and again, you may see soft corners. This is especially true for macro and ultra wide-angle photography. To make sure the corner softness isn't due to field curvature, shoot a lot of frames, varying the focus distance somewhat: if you get some frames with soft corners but others with good sharpness across the frame, it's field curvature. If you can't get the corners sharp no matter what you do (short of stopping down), it's a lens problem.

Excellent: Small detail is "tack sharp" and small gradations are rendered clearly and faithfully; image has an almost magical transparent and three-dimensional look.

Good: There is no obvious softening of the image, even in the corners. The picture generally looks "pretty good" although it doesn't "wow".

Not-so-good: Corners may look somewhat soft, but not enough to be noticeable on 8x10 or smaller prints. Overall look is flat and muddy, sometimes with a color tint (usually magenta).

Bad: Corners are very soft and "smeared," sharpness even at the center is iffy.

Ramifications: A lens with poor sharpness is a poor lens. If a picture isn't sharp, post-processing can only help so much. Given a choice between high resolution and high contrast, I'd tend to prefer contrast: it's visible even on normal-sized prints, whereas exceptionally high resolution only makes a difference on exceptionally large ones... and resolution is often limited by camera or subject motion anyway.

Remedies: Poor sharpness can be corrected -- up to a point -- by considered use of unsharp mask. This actually only affects contrast, not resolution, but it can sharpen up slightly soft images. Beyond a certain point, though, USM is helpless: it can't put in detail that wasn't there in the first place. See my lesson on Sharpening for more on this subject.

Bokeh

Bokeh is Japanese for the out-of-focus areas on a lens. "Good bokeh" means that they have an even, soft quality: ideally, like a sharp picture severely Gaussian blurred in Photoshop. "Bad" or "harsh" bokeh means that highlights will be ring-like, fine detail will be "ghosted" or "doubled," giving the bokeh a generally uneven and rough look. Bokeh is produced by the entire lens design: it's far too often simplistically attributed to the number of blades on the iris (more being better). The iris shape only immediately affects the look of out-of-focus point-like highlights: they take on the shape of the iris. On a five-bladed iris, they'll look like rounded pentagons; on a 9-bladed one, they'll be nearly circular.

Good bokeh. The girl in the background has an evenly soft, velvety appearance. This was shot on film with the classic Canon FD 50/1.4 SSC Mk I.

Bad bokeh. The reeds are distractingly ghosted and doubled. This is from the Minolta D7i: bokeh is the otherwise excellent lens's Achilles heel.

Assessing bokeh: Shoot a few pictures at different apertures with significant areas out of focus. Look for ghosting, doubling, or ring-like highlights. If you like, shoot another scene with a dark background and point-like highlights at a few apertures. Look at the shape and appearance of the highlights.

Excellent: Bokeh is soft and even at all distances from the field of focus. No ghosting or doubling is apparent. Highlights are round or nearly round. No color shifts in the out-of-focus areas.

Good: Bokeh is soft and even overall, although some ghosting or doubling may be seen near the field of focus. Highlights may look polygonal. There may be some radial smearing ("comatic aberration") visible in out-of-focus areas near the edges of the picture. Very little color shifting in out-of-focus areas.

Not-so-good: Ghosting or doubling is visible at most distances. Highlights may look polygonal and have starburst-like rays coming from them. Contrasty areas in the out-of-focus parts of the picture may acquire weird color shifts, most often purplish.

Bad: Ghosting or doubling is bad enough to distract seriously from the look of the picture. Highlights as above. Very strange things happen to the color in out-of-focus areas.

Ramifications: Good bokeh is next to impossible to put in afterwards (not without a lot of work anyway). It's a signature characteristic of the lens, and highly desirable -- it can make the difference between a quiet, calm, and intimate look and a noisy, harsh, "accidental" look. There are two schools of thought about the shape of the highlights: some consider round highlights highly desirable, others consider their characteristic shape part of the lens signature. I tend to fall in the latter school.

Color

One subtle characteristic of a lens is color. A good lens should have neutral color -- it should not tint the image in any way. Lenses may give the image a cool (blue) or warm (yellow) cast... and cheap ones can impart a nastier magenta cast.

Assessing color: Color is another difficult characteristic to assess, as so much depends on the scene. The only reliable way to assess it is to shoot the same scene with a reference lens (the 50/1.4 is considered the best for this purpose) and compare. Looking at lots of pictures from the lens will help give an impression of color as well.

Excellent: Neutral. No color shift in any direction.

Good: Slightly warm or cold. A small blue or yellow tint that's easy to fix in post-processing (and may even be desirable if kept under control).

Not-so-good: Strongly warm or cold or slightly magenta. This may be tougher to clean up later.

Bad: Strongly magenta.

Ramifications: A lens with good color will make post-processing easier. A lens with really bad color may make it difficult to get good results even with post-processing. Color used to be an extremely important lens characteristic for film, but the possibilities of digital color correction have greatly reduced its importance. I would consider a magenta cast a significant downside, but a minor yellow or blue one pretty insignificant. This doesn't stop people from hating "Sigma yellow" reputedly common in Sigma lenses.

Remedies: Color can be easily corrected to a fairly large extent by levels or curves manipulation. See my Levels and Curves lessons for techniques.

Chromatic aberration

Chromatic aberration happens when light of different wavelengths (colors) doesn't end up focused at the same point: it's either shifted radially (the different colored images are sharp but "out of register" with each other) or in distance (color extremes are "out of focus"). The former is called "sagittal chromatic aberration," and the latter, "axial chromatic aberration."

Sagittal CA

Sagittal CA shows up as a red or green "ghost" image around high-contrast edges near the edge of the frame. It's important not to confuse this with sensor blooming (a violet glow around a blown-out highlight). Most ultra-wides suffer from this type of CA to some degree. Good normal-to-tele lenses should show little or none of it. This type of CA is usually unaffected by aperture.

Sagittal CA: the red shadow along the right edge of the shirt.

Assessing CA: Shoot a scene with, say, branches against the sky near the corner of the frame. Look for red, violet, or green fringes or ghost images of them.

Excellent: No CA to be seen, even at 200% magnification.

Good: No CA to be seen at 100% and normal distance from computer screen.

Not-so-good: CA easily visible at 100% under some conditions.

Bad: CA visible in nearly all frames even in images downsampled to screen size.

Ramifications: CA is highly undesirable, but it is very hard to completely avoid. Again, digital post-processing can help a lot: you can desaturate reds and greens in the problem areas. This results in a slightly softer image, but gets rid of the distracting coloration.

Remedies: CA can be effectively cleaned up with the following technique:

  1. Zoom into 100% on a problem area.
  2. New Adjustment Layer: Hue-Saturation-Lightness.
  3. Reds (or Magentas, depending on the lens): Saturation: -100%.
  4. Cyans (or Greens, depending on the lens): Saturation: -100%.
  5. Tweak the two ranges above until the CA disappears (from the problem area you're examining).
  6. Select Mask channel.
  7. Fill: Black. (This will completely hide the changes.)
  8. Use the eraser in soft-edged brush mode in the mask channel on the problem areas.

I've made some attempts to automate this, but haven't been able to do it entirely satisfactorily. This doesn't usually take more than a few minutes, though, so it's not too onerous.

Axial CA: Halation and purple glows

One thing that very commonly happens on lenses when they're shot wide-open is "purple haze," technically known as halation (production of haloes). The main culprit of this is axial CA. What this means is that parts of the spectrum, usually red or purple, are not focused at the right plane: these colors form a soft "glow" around high-contrast edges. It has an overall softening effect on the image. Most consumer-grade lenses suffer from this type of CA to a greater or lesser degree, as do almost all exceptionally bright lenses. Canon's "L" lenses are known for having very little of this type of CA: they use elements made of fluorite crystals, which are very effective at correcting CA. On the better non-L lenses, wide-open axial CA is minimized and engineered to have a more neutral color than the nasty magenta (mix of red and violet) that cheaper lenses have.

Most lenses exhibit this to some degree when wide-open. On some, it's bad enough to render the wide-open aperture nearly unusable.

A very bad case of axial CA-caused halation: "purple haze."

The Canon 50/1.4 produces a particularly nice kind of halation. It's present wide-open, but it's fairly neutral in color. This type of halation can actually be used creatively, for example to smooth out skin imperfections in portraiture, or give a dreamy feel to landscapes.

The 50/1.4 wide-open: there is a "glow," but its color is close to neutral.

Intentional use of halation in a photo. I wanted to give a dreamy look to the scene, and shot the 50/1.4 wide-open. I like this version a lot more than the stopped-down frames, where the branches are tack-sharp. (Come to think of it, maybe I shouldn't sell it after all: the 50/1.8 Mk I has no veiling glow at all, so I'd lose this creative option...)

Assessing halation: Use your wide-open CA test shot, and examine the branches. If they have a translucent look, it's probably due to halation. Look for glows around highlights and high-contrast edges. Be careful not to confuse it with sensor blooming (a bright purple glow around a blown-out highlight), or with out-of-focus softness. Haloes cast a veil over fine detail, giving the picture a "soft-focus" look. If there's no color cast, this can be effectively used for portrait photography, as it nicely smooths out skin imperfections: this is why it can actually be considered a positive lens characteristic, if kept to a reasonable degree.

Excellent: No halation visible at any aperture, even around extremely high-contrast edges.

Good: Some neutral-colored glows visible wide-open, disappears one stop closed. This can even be desirable for portrait photography.

Not-so-good: Some purple glows visible wide-open, disappears one stop closed.

Bad: Lots of purple haze all over the picture wide-open, may not disappear even stopped-down.

Ramifications: A lens with little or no halation is a dependable lens. You can shoot it at any aperture and be fairly confident of the "look" it'll give you. Some halation can actually be nice for portraits, as it can give a pleasing soft-focus look to them, and gives a characteristic "lens signature." However, I'd consider significant "purple haze" as a pretty major point against a lens. Halation is especially important on long lenses, as it can render the wide-open aperture useless -- and with long lenses, you pay a lot of money and haul a lot of weight to get the wide-open aperture in the first place. The 50/1.4 that exhibits a moderate amount of halation makes for a pleasing portrait lens wide-open, and a super-sharp lens from f/2 down -- the extra weight from the extra stop is pretty negligible. But I wouldn't buy an 80-200/2.8 that has a lot of veiling glow.

Remedies: Purple haze can be effectively combatted with a similar technique to dealing with chromatic aberration: this will remove the magenta tint from it and turn it into ordinary, neutral glow. Halation itself is more problematic. It responds well to aggressive unsharp mask in low-contrast areas, but this will cause serious artifacting (sharpening glows) in higher-contrast areas. Entirely satisfactory results are difficult to achieve, but the following techniques can help a lot:

To get rid of the purple haze:

  1. Select: Color Range - Highlights.
  2. Expand selection: 8 px.
  3. Select background layer, and new: Adjustment Layer - Hue-Saturation-Lightness, Magentas - Saturation: -100.
  4. Deselect, and Select: Color Range - Highlights.
  5. Contract selection: 2 px.
  6. Select layer mask from adjustment layer, and Fill: Black
  7. Deselect, and Gaussian Blur: 3 px.

To mitigate the problem overall:

  1. Duplicate background layer.
  2. Select: Color Range - Highlights.
  3. Expand selection: 8 px.
  4. Deselect, and Select: Color Range - Highlights.
  5. Contract selection: 2 px.
  6. Select layer mask from adjustment layer, and Fill: Black
  7. Deselect, and Gaussian Blur: 3 px.
  8. Invert.
  9. Select master channel.
  10. Apply ultra-aggressive unsharp mask (e.g. 200%, 4 px, 0 levels). (The mask will leave out the high-contrast edges, where sharpening haloes would be most apparent.)
  11. Select mask channel.
  12. Use the eraser and airbrush to mask in and out problem areas -- sharpening artifacts on the one hand, soft areas on the other.
  13. Lower layer opacity until the picture stops looking unnatural.
  14. See if it looks better with Luminosity blend mode.

Flare

Apart from halation, a specific type of flare, two other types are worth looking at: flare spots and veiling. Flare spots are caused by reflections of a point-like light source (usually the sun) inside the lens, and appear as circular or polygonal shapes, often with a color tint, lined up in the picture. Veiling is more subtle: it's caused by incident light bouncing around in the lens and reducing overall contrast: a picture with heavy veiling looks like it was shot through pantyhose.

Veiling has played hell with the contrast on this picture: it looks like it was shot through pantyhose. You can also see a few small but pretty bright flare spots in it.

Assessing flare: Shoot two test frames: one with the sun just outside the frame, and another with, say, a bright window and darker room, exposing for the room and overexposing the window. Take a comparison shot of the room with the window outside the frame. Look for flare spots in the first one, and overall contrast compared to the reference shot in the other.

Excellent: Faint or no flare spots. Little or no visible loss of contrast in the window test.

Good: Visible but not very bright flare spots. Visible loss of contrast, but still usable frame.

Not-so-good: Bright flare spots. Severe loss of contrast; picture difficult to rescue.

Bad: Bright flare spots and lines. Picture ruined by veiling; low-contrast detail gone, impossible to recover in post-processing.

Ramifications: This depends a lot on what you intend to do with the lens. A landscape lens with bad veiling flare problems is a bad proposition, as the bright sky will wipe out much of the contrast in many cases. The same behaviour may make little difference on a portrait lens used for low-contrast scenes and controlled lighting.

Remedies: Especially on longer lenses, both kinds of flare can be very effectively combatted with lens hoods: if no incident light hits the lens, you will see no flare spots and much less veiling. Flare is difficult to clean up in post: contrast that is gone can only be restored to a degree, by "printing through" it using levels, curves, or a large-radius, low-amount unsharp mask (see my Levels, Curves, and Sharpening lessons for techniques) -- but the end result won't have the pop it would originally have had. Flare spots can actually be used effectively as a part of the composition, or pretty effectively toned down using the Burn tool.

Light fall-off (vignetting)

Especially wide-open, most lenses are brighter in the center than the edges. This is especially pronounced with wide-angles and some long teles. This phenomenon is known as light fall-off or vignetting.

A fairly mild case of vignetting. If a lens does no worse than this wide-open, it's pretty good.

Assessing vignetting: Take a test shot of a scene with even lighting. If there's significant vignetting, the center will be visibly brighter than the corners.

Excellent: No obvious vignetting at any aperture.

Good: Some vignetting wide-open; gone one or two stops down.

Not-so-good: Wide-open vignetting is bad enough to need post-processing. Some visible even stopped-down.

Bad: Vignetting bad enough to need correction in-camera (with a circular-graded neutral-density filter), or use of B/W negative film. You might see this on an ultra-ultra-wide rectilinear SLR lens, such as a 14 or 15 mm. It does happen on the widest rangefinder lenses currently available (the 12 mm Voigtländer Heliar, for example).

Ramifications: In my opinion, vignetting is close to a non-issue. It's extremely unlikely to be bad enough that it can't be pretty easily corrected in post-processing (with a curves tweak and circular-graded mask).

Remedies: Vignetting is easy to fix in post, unless it's really bad (over a stop and a half or so):

  1. New Adjustmen Layer: Curves, blend mode Luminosity.
  2. Pull up the curve so that the corners brighten up approximately to the original levels of the center. Don't worry about getting it exactly right: overshooting is OK.
  3. Select mask channel. Put a circular, graded fill from black to white, centered in the frame.
  4. Adjust translucency until vignetting mostly fades out. If you get a circular banding effect, redo the mask channel fill.

Distortion

A lens with no distortion renders straight lines as straight, no matter where they are. Most lenses aren't so good, though, and instead buckle them outwards (barrel distortion) or inwards (pincushion distortion), at least close-up. Macro lenses are specifically designed for minimal distortion, and wide-range zoom lenses often suffer from it more.

Assessing distortion: Take a test shot of a scene with lots of straight lines (an office block would do well). Print it. Compare the lines with a ruler.

Excellent: Lines are straight or very close to it at all focus distances.

Good: Lines look straight, but the ruler comparison shows some distortion.

Not-so-good: Barrel or pincushion distortion is apparent without a ruler.

Bad: Barrel or pincushion distortion is bad enough to need correction.

Ramifications: Distortion can be corrected out-of-camera fairly easily. Since it's a lens characteristic, it can even be automated: in fact, lots of people are using rectangular fisheye lenses that have a tremendous amount of barrel distortion for normal wide-angle photography, correcting for it out-of-camera. For this reason, I don't consider it a major issue anymore -- although it certainly used to be, before the advent of digital image processing.

Remedies: Distortion is best corrected with Panorama Tools. See Helmut Dersch's essay for tips and techniques.

Test suite

It's usually pointless to test for every characteristic of the lens you're interested in. Pick the characteristics that you consider most significant, and test for them. Simulated (or real) real-life scenes are far and away the best lens tests, even with the inevitable variation of circumstances. Resolution charts, MTF curves, or newspapers do not give a realistic idea of the lens's real-life performance: they can either be over-optimistic (since they don't catch near-infinity issues), over-pessimistic (since they make field curvature look like spherical aberration-caused corner softness), or just plain incomplete (for example with regards to flare).

Here's a proposed test suite for gathering the data you need to make the assessments. It only takes a few minutes and can be shot hand-held outside the camera store. If you have the time and possibility, use a tripod for best results. If shooting hand-held, crank up the ISO if necessary to ensure sharp pictures -- ideally 1 / (5 x 35 mm equivalent focal length), shoot more frames if necessary to increase your chances of getting in a sharp one, if shooting at slower shutter speeds. Take several frames for each, varying the focus point: otherwise the lens may look soft due to camera shake (for the stopped-down shots) or missed focus (for the wide-open shots). Pay attention to the handling characteristics while shooting the test shots -- it'll give you an idea about how it'll feel in real life.

  1. Examine the lens's build characteristics:
    • Size and weight
    • Fit and finish
    • Focusing motor
    • Focusing mechanism
    • Zoom mechanism
    • Other characteristics
  2. Move outside the store.
  3. Test AF performance by taking quick shots of passers-by. Ideally, compare to a similar lens with known performance. "Pretty fast" is nowhere near as informative as "Almost as fast as a 70-200/4L."
  4. Take the following test shots wide-open, stopped down two stops, and at f/11:
    1. Bokeh test: in-focus subject, out-of-focus chaotic background.
    2. Sharpness test: detailed scene as near infinity as possible, detail extending to at least one corner of the frame at a near-constant distance. (Usable also for assessing vignetting.)
    3. CA and halation test: branches or fine building detail against the sky, in focus near corner of frame.
    4. Flare test 1: Sun just outside frame, into the light.
    5. Flare test 2: Indoor scene, with bright window in the frame, but exposed for the interior.
    6. Distortion test: Building with straight lines. Does not need to be perpendicular.

The other optical characteristics can be assessed by examining these pictures for them.

Then take the pictures home and examine them as described above. Ideally, write up your experiences -- even just for your own future reference... but if you'll post them on the Internet, you will get a grateful audience, including yours truly.

In summary

The most useful test of a lens is to see how it is at its worst. So, to do this, shoot a few frames at maximum tele, wide-open, and near-infinity... and then look at the corners. If they look mostly OK, the lens is likely to be a star performer. If not, stop down a notch or two. If they're still not OK, the lens is likely to be a dog.

And remember to have fun: if you'd rather spend your time taking pictures than testing lenses, more power to you.