There are a few methods for screening lenses that are floating around. As procedures for non-technical folks, I think they all have issues. In this page, I’ll run through some common approaches and tell you what I think are the problems.
Full-frame flat target
One common approach to lens testing is to use a flat target that fills the frame. There are a host of practical difficulties with doing this.
- Target alignment is critical
- Target flatness is equally so
- The target size gets to be impractically large if the lenses are short
- It’s hard to practically impossible to test short lenses at realistic distances.
- It’s hard to light the target evenly
- It’s hard to find a space to set up the test with a big target
- Big targets are hard to store
- Big targets are expensive
- Big targets are subject to damage.
It is hard for the non-expert to align the camera to the target. Worse, the effects of small alignment errors are between extremely difficult and flat-out impossible to distinguish from common lens manufacturing defects without tedious cross-checking. In correspondence with many testers, I have found that hardly any of the general population of amateur lens testers is willing to do the cross-checking. Some are even offended if I question their alignment.
Many testers seem to deal with the huge size of the targets that you need to do this kind of testing by eschewing targets designed for the purpose of testing lenses and camera, and using found objects, such as brick walls, as targets. Such targets are far from optimal, lacking precise indication of focus error, often being non-uniform across the frame, having a preponderance of uninteresting low-spatial-frequency detail, and not being repeatable from day to day and lens to lens.
This test uses one or more distant natural or man-made objects for the target. The lens is focused with the target in the center, then, without refocusing, the image is recomposed, placing the target in the corners and possibly at the center right/left/up/down locations.
The Gletscherbruch test has a lot to recommend it. It gets around the alignment issues of the full-frame flat-target tests. It has two main drawbacks.
- Atmospheric effects. Here on earth, performing lens testing at great distances puts a lot of air between you and your target. That air is subject to thermally-created changes in density that cause time-varying bending of light rays that result in uncontrolled and unpredictable blurring of images. When performing Gletscherbruch testing in unpopulated areas in still early-morning air, this can be dealt with for all but the longest lenses, but at other times of the day or in cities with unseen, and often unsuspected, heat sources, it can be a problem even at moderately short focal lengths. The time-varying nature makes it especially tricky when you’re looking for differences in the image sharpness at the four corners of the frame. Sometimes the blurring caused by the atmosphere can be greater than that of the lens and camera, making it impossible to see small lens defects.
- Inappropriate target. This is the same problem that you have with found flat targets. They have the wrong spatial-frequency makeup and they aren’t repeatable. You can’t spot focusing errors by inspection of the captures. You can’t read out sharpness in any way that’s describable to others. You don’t know what the images are supposed to look like with any specificity.
Just go shoot pictures. Many people recommend this. It has some appeal. Your left brain can take a vacation. You won’t find bad things that aren’t relevant to the kinds of pictures you make (but then again, if you change your photography you may find that your perfect lens isn’t perfect any more). In the hands – and, more to the point, with the eyes – of highly experienced photographers, this can work, just like big flat targets can work for another kind of expert. But for most folks, this doesn’t appear to be so great.
A few sticking points:
- Is this lens OK? You see it all the time. Someone buys a new lens, runs out and make a few snaps, then posts them on the web wanting to know if the lens is any good. Half the answers seem to be on the order of: “That lens is a hunk of junk. The manufacturer should be ashamed, and you’re blind if you can’t see that.” Half the answers say: “That lens is perfect. Or, if it’s not perfect, it’s plenty good enough. And photography isn’t about lenses in the first place; it’s about making great images. Forget your testing and go out and shoot wonderful pictures.” And half (multiple points are frequently made) say: “You tested the lens wrong. All those things that you are worried about are probably the result of your not setting things up right. Go learn about depth of field.” All those points are usually at least partially valid, but that doesn’t help the person with a new lens who wants to know whether to keep it or not.
- How does my copy of this lens compare to yours? With no clear regime and found objects and lighting, there’s no way that people can communicate in a meaningful way about their results.
- How sharp is sharp enough? With no way to visually measure lens sharpness is isolation from the nature of the subject and the lighting, you can’t describe how sharp a lens is.
The out-of-focus point spread function
This test was first brought to my attention by Prof Hank on DPR. I have made a few small changes and written up instructions for it here.
It is simple and quick to perform; you don’t even have to make an exposure if you’re in a hurry. It is a good test for decentering. It is not particularly sensitive, but for lens screening by non-experts, that’s not a bad thing as it avoids false positives. However, it doesn’t test for field tilt or any number of lens defects that can results from faulty assembly.
Out-of-focus Siemens Star shots
Roger Cicala recommends defocused images of a Siemens Star for testing for decentering. This is a good test for that defect, but shares with the OOF PSF test immediately above insensitivity to other common lens assembly errors.