This is the first post in a long series on the subject of lens screening tests. To see links to all the other posts in this series, go to the bottom of this page — below the comments — and you will be able of click on each one in turn.
For the past few days, I have been spending a lot of time thinking about how to do some kinds of off-axis lens testing with small targets that take up only a small portion of the camera’s visual field. There are a host of problems associated with trying to get at the entire field of a lens in one shot:
- Target alignment is critical
- Target Flatness is equally so
- The target size gets to be impractically large if the lenses are short
- It’s hard to practically impossible to test short lenses at realistic distances.
- It’s hard to light the target evenly
- It’s hard to find a space to set up the test with a big target
- Big targets are hard to store
- Big targets are expensive
- Big targets are subject to damage.
What if we were looking to measure astigmatism? We could use a Siemens Star as a target:
If we wanted to do everything in one shot, we could put a whole bunch of those up on a wall, try to align the camera, try to light it, make a shot, and probably find that we didn’t have everything right.
Or, we could just take one such target, put it at a distance from the camera, make an on-axis exposure, then angle the camera up and down and back and forth taking a bunch of shots with the star at various places in the frame.
Sounds simple, huh? Bet you’ve already seen the problem.
Let’s look at a crude diagram looking down with the target on-axis:
We’ll assume a perfect rectilinear lens, so the focal plane is the flat line across the top. Note that if we place a target at the location labeled Target 2, it’s still in focus, but it’s farther away from the lens than Target 1. If we leave the camera at the focus position which made Target 1 sharp above and swing it to the side, we end up with the situation below:
Now the target is out of focus. It’s closer to the camera than where the side of the camera is focused.
It’s pretty obvious that this isn’t a problem at extreme distances. Infinity is infinity no matter where in the frame it is. So depth of field (DOF) can cover up this focusing error. In order to put some numbers on it, I wrote a computer program to compute the circle of confusion (CoC) for this kind of misfocus at the very corner of the sensor for lenses between 18 and 100 mm on a full frame (FF, 24×36 mm)) sensor at f/4 with a target distance of 20 meters.
We’d like the CoC to be quite a bit smaller than the pixel pitch. FF cameras of high resolution have pixel pitches on the order of 4.5 micrometers (um), so this distance qualifies for all the focal lengths we’re talking about, if only barely for the longer ones.
What if we make the sensor bigger?
Now the longer focal lengths are getting uncomfortably close the the 5.3 um pitch of the Sony 33×44 mm sensor. We need to back up a bit further:
That’s probably OK.
Now let’s look at how big the target has to be. What if it’s printed on a piece of C-size (17×22 inches) paper? Call that 400 mm, and we’ll compute the target height in pixels. For the 24×36 mm, 20 meter, case with a pitch of 4.5 um:
A comfortable target size would be 200 to 300 pixels. We get there for the long focal lengths, but not for the shorter ones.
With a 33×44 sensor and 30 meters (I left the pitch at 4.5 um (oops):
About the same.
It’s clear that one size isn’t going to fit all in this enterprise. We’ll have to get closer to the targets when we’re using short lenses. We probably have the room to do that, since we had comfortable small COCs with the short lenses. I’ll do some calculations to compute the optimum distances and report soon.
Just a quick speculation from someone who acknowledges much of your article was “over my head”
… any way a swinging pendulum could be used? Camera pointed downward?
Like a metronome?
Or the metronome is in normal position? At the end of the swinging arm where the weight would be is a small target?
Or am I once again “unclear on the concept”?
Jim,
All of this is why for many years now I’ve been using the tops of telephone poles at distances of roughly 60 meters to do exactly as you’re imagining, less the star target. The top of a pole, especially when you’re at an angle of roughly 40 degrees from the plan view line of the power lines, results in strong, high-contrast diagonals at opposite angles against a bright sky, and other details such as the HIGH VOLTAGE sign, insulators, etc., all of which make for an excellent basis for evaluating lens performance.
I’ve long observed that the issues that I’m screening for are considerably larger (10X sort of thing, often much more than that) than the issue of the ideal focus plane of the lens swinging behind the target when placing the pole top in the four corners of the image.
Usually I rely only on five locations, center plus corners, but sometimes I double-check shorter-radius positions, as occasionally a lens will have a ring of poor performance well inside of the circle corresponding to the corner image areas. But that’s quite unusual with modern designs and was never very common, or so is my impression after having examined dozens of lenses of many types. Focus shift upon stopping down has also not become obvious in even one single test, which allows one to simplify the test and have an easier time of making the initial, on-axis, wide open focus adjustment.
Usually I’ll initially look, for example, at a wide-open set of 5, then at a medium aperture set of 5, and go from there, often looking at sets of 5 from every aperture clear up to f/16 or smaller. That can get pretty slow, but if f/5.6 or f/8 look great in all four corners, there’s no need for most of the rest of the work in most cases, assuming demands such as my work imposes.
Sometimes the lens issues are obvious right on the camera’s LCD, fully zoomed in, but as a rule, if a lens is a maybe, one still has to examine the files on a computer, absolutely. Quite a few which look OK on a well-adjusted camera LCD turn out to be mediocre after all. (Note the importance of turning a Sony’s default JPEG sharpening setting way down, as this is used on both of the camera’s screens too and halos in the image will really mess with the clarity needed for seeing lens issues.)
For the most part, in my work I’m very happy with a lens if its sharp in all four corners on a plane of focus that 20 feet or more away at f/8. Sometimes I’m quite OK with a lens which requires f/11 to meet that condition (for medium format work usually but for a Sony also). I’ve never yet seen a lens which was so soft in the middle that a sharpened final result could be considered a failure there. But it’s true that some lenses give images which require a lot weaker sharpening and that’s certainly nice, if often unattainable for one reason or another.
My sense of the telephone pole test is that it works about equally well with all focal lengths too. Very handy.
Nice to hear from you, Joe. Thanks for your insights. Always appreciated.
Jim
I’m thinking that the setup depends on whether one wants to also include field curvature in the test or if one is content with inspecting the maximum attainable resolution. In the latter case, wouldn’t refocusing after the target is composed in the corner of the frame work?
You are correct. I’m interested in field curvature, curvature anisotropy, field tilt, astigmatism and its interaction with the previous, and the like for these tests. Focusing on each corner also introduces the possibility of differential focusing errors, which could make a good lens look bad.
I see. All of those are obviously valuable information for both evaluating whether a sample is good or bad and for predicting how to best utilize a lens in the field. But on the top of my head (and past my bedtime) I’m thinking that some calculating needs to be done to determine the offset of the optimal focus of the corner from the ideal. Good tests are not trivial; I can see why you have been thinking about this.
Are you referring to differential focus errors as errors in focus between each corner? My assumption was to do enough repeats to minimize testing error.
I’m trying to come up with simple tests that are hard to screw up, even if performed by people who are not careful, patient, and quantitative. An example of such a test is, I think, my Simple Decentering Test. My feeling is that the vast majority of lenses that users test and return as faulty are actually quite good, and I’m trying to minimize the probability of false positives. Differential focus errors can, if uncaught, masquerade as any number of lens defects.
I’ve done lots more calculating, and I’ll post the results soon. Thanks for your attention and comments.
Jim
What about refocusing at the first corner only? More precisely: refocus once for each distance from center that you want to test. That should reduce the necessary room/garden/court and target size and still get you decentering and astigmatism.
Whether that’s a good thing to do or not depends on what you’re trying to test for. If you car about field curvature, then it’s not the thing to do. I will be working on the topic of what further tests should be performed if the lens doesn’t do well on the basic test.
I must admit it is easy to show one’s ignorance on this site. That said, you don’t say if you are considering auto focus errors, so you are manual focusing? Also, in my humble experience with Nikon lenses, especially the super wide zoom, it had huge field curvature, which would mess up the proposed test, no?
Yes, I’m using manual focusing. But if an acceptable focus is achieved at the center of the image, it doesn’t much matter how that happens. What is important is that the same focus point be used for the whole set of shots in a series, so that the relative performance of the lens at various off-axis points can be observed. And yes, this will detect focus curvature. I suppose if the focus curvature is sufficiently extreme its blurring would keep you from seeing things like astigmatism, but I need to work out the details of the test and start putting lenses through it to see if that’s a problem in practice.
I can think of a workaround if it is, and that’s to ignore the center, focus on one of the corners, and then shoot the other three corners without touching the focus.
Jim