This is a continuation of a test of the following lenses on the Sony a7RII:
- Zeiss 85mm f/1.8 Batis.
- Zeiss 85mm f/1.4 Otus.
- Leica 90mm f/2 Apo Summicron-M ASPH.
- AF-S Nikkor 85mm f/1.4 G.
- Sony 90mm f/2.8 FE Macro.
The test starts here.
When John Henry was a little baby sittin’ on his daddy’s knee
He picked up a hammer and a little piece of steel
Said “Hammer’s gonna be death of me, Lord, Lord
Hammer’s gonna be the death of me”
Jack Hogan asked today for a comparison of manual focusing of the Zeiss Batis 85/1.8 to autofocusing. I was a bit concerned about putting my focusing skills up against a machine, particularly one like the a7RII that has been turning in spectacular results with the Sony AF lenses I’ve been testing it with.
I swallowed hard, and set up a test.
Now the captain he said to John Henry
“I’m gonna bring that steam drill around
I’m gonna bring that steam drill out on the job
I’m gonna knock that steel on down, Lord, Lord
I’m gonna drive that steel on down”
I mounted a Sony a7RII with the Batis 85/1.8 attached to the Cognisys computer-driven focusing rail,just as before, but I didn’t turn the rail on. I set the assembly 8 feet from the on-axis target, mounted the lens, focused on the Siemens star using manual focusing (more about that momentarily).
John Henry told his captain “Lord a man ain’t nothin’ but a man
But before I let that steam drill beat me down
I’m gonna die with my hammer in my hand, Lord, Lord
I’ll die with my hammer in my hand”
I used two other targets for the AF part of the testing:
When I used AF, I focused using AF-S with flexible spot, AF priority, small and medium spot sizes. I exposed 16-shot series with the lens wide open. I used Jack Hogan’s Matlab program, MTF Mapper, and DCRAW to pick the horizontal edges and calculate the MTF50s for the raw color planes, imported the data into Excel, and plotted the statistics of the results in cycles per picture height.
John Henry was driving on the right side
That steam drill driving on the left
Says, “Before I let your steam drill beat me down
I’m gonna hammer myself to death, Lord, Lord,
I’ll hammer my fool self to death”
The average results for horizontal edges:
The top set of bars, one for each raw channel, is for the images made with manual focusing at the high magnification with peaking turned off. I looked for aliasing as an indication of best focus.
The next set down is also for manual focusing, but with peaking turned on at minimum setting. As is usual with high contrast subjects and sharp lenses, the lowest peaking setting, even at maximum magnification, is more sensitive than would be optimum. Even so, peaking produced better results than the no-peaking test.
Notice that the manual focusing cases correctly and expectedly optimized the green channel at the expense of the red.
John Henry he hammered on the mountain
His hammer was striking fireBut he hammered so hard it broke his heart
And he laid down his hammer and he died, Lord, LordHe laid down his hammer and he died, Lord, Lord
Here are the averages for the vertical edges:
Pretty much the same except for the four lower groups, which have higher blue and green values than for the horizontal edges.
Let me draw your attention to the third group of bars. They are for AF on the zone plate target. On this target, unlike the others I’ve used, the camera and the lens were able to cooperate enough to properly optimize the green channel, essentially as well as your faithful scribe.
The next four bars are for AF with both spot sizes using the Siemens star and Horshack grid focusing targets. As before, when faced with those targets the camera and lens collectively give more weight to the raw channels than they should, by a small amount for the vertical edges and by a larger amount for the horizontal ones.
John Henry had him a woman
Her name was Polly Ann
She walked out to those tracks
Picked up John Henry’s hammer
Polly Ann drove steel like a man, Lord, Lord
Polly Ann drove that steel like a man
Now I’m going to show you where the machine excels. Here are the standard deviations for both sets of edges:
In the no-peaking manual focus case, I wasn’t very consistent; the machine beat me there. With peaking, I was pretty solid, but not as solid as AF with the Horshack grid and a medium spot, or the Siemens star and a small spot, or the zone plate with the medium spot.
I can’t make much sense of the variations among the AF standard deviations.
Now every Monday morning
When a blue bird begins to sing
You can hear John Henry from a mile or more
You can hear John Henry’s hammer ring, Lord, Lord
You can hear John Henry’s hammer ring
Excellent work, Jim, so comparing it to the earlier focus-rail curves it looks like one can indeed hit the top of the green channel consistently manually – aided by focus peaking. What a neat feature that is.
Assuming that it also ‘looked’ sharpest to you, your results confirm that best focus for a given lens/camera combo is at the peak of the green channel. In order to compute an overall MTF50 ‘perceived sharpness’ score for the system I still wonder what weights should be given to the readings of the red and blue channels when green peaks.
Jack
Jack, I haven’t done the work yet, but the way to get the right mixture of the raw channels for focusing is to multiply the raw channels by the compromise matrix to XYZ, white balance by von Kries or Bradford, then look at how the raw channels contribute to Y.
Jim
Ah, I see, much more complicated than I was hoping.
Going a bit off-topic, isn’t Y proportional to luminance (L), neutral L represented by equal amounts of raw R,G,B?
If so, assuming ‘suitable’ illumination, couldn’t we just be using as weights for the MTF50 readings off the individual R, G and B channels the ratios of G/R, 1 and G/B found in the mean raw data in a neutral patch?
You’re right about the illumination. For most cameras, you can get close to what you’re looking for with a 5500K lighting source and a CC30M filter.
That might be a good idea for testing, but I’d have to buy a bunch of CC30M filters (or one huge one), and changing the light that way runs a big risk of changing the way the AF systems works, so the results woudn’t necessarily be applicable to normal lighting.
Then again, some ETTR aficionados use CC30M filters for real photography to get more photons in the red and blue raw channels.
Jim
> That might be a good idea for testing, but I’d have to buy a bunch of CC30M filters (or one huge one),
do not filter lens – try to filter your LCD panel with gel instead ….
That’s a better idea. Thanks. I still think that changing the light is not the way to go, since it sets up a situation that the camera’s AF mechanism is unlikely to encounter in the real world.
Jim
Not under normal lighting conditions. The green raw channel usually has the highest average signal.
Jim
> since it sets up a situation that the camera’s AF mechanism is unlikely to encounter in the real world.
certainly you can’t imagine this outdoors (w/o a magenta, or whatever, filter on a lens instead of a gelled light) – but if you are shooting in a studio where you control the illumination and gels are a staple then it is a curious question whether you can have an additional benefit of a better AF (in addition to better S/N in R/B channels)… that is when you have to use AF in a studio of course, in many case you can spend time doing utmost manual focusing
Got those backwards, I meant R/G, 1 and B/G.
I’m not sure I see where you’re going with that. The right mix depends on the state of adaptation of the viewer, and hence on the lighting/filtration. Maybe you’re doing a crude adaptation by dividing everything by G.
Jim
Stumbling around where color is concerned, Jim 🙂 I was thinking that the compromise matrix is designed so that Y in XYZ is proportional to luminance, and white = 1 with every channel the same in both raw RGB and XYZ space. So if you have a neutral target illuminated by D50, aren’t the raw RGB coefficients in the matrix for Y effectively the white balance factors off the neutral target (Hunt, p.45)?
If so, shouldn’t we be able to use them to determine weights for the individual channel MTF50 readings?
You’re forgetting the white balance step that has to come after conversion to XYZ and before we get luminance referred to the adapted observer.
> You’re forgetting the white balance step that has to come after conversion to XYZ
nope, WB (in general) can be applied before even demosaicking on raw channels as simple per channel multipliers and a number of raw converters in fact do this… way before any color transform from camera Raw RGB to any PCS
You’re right that WB can be done before conversion to XYZ, but it’s less accurate when done that way than, say, by using Bradford (which can be baked into the conversion using a conpromise matrix. My main point was that WB needed to be performed, not where it needed to be done, and I assumed the more accurate approach.
Jim
> but it’s less accurate
both ways you refer to matrix operations – one can not be better in principle than the other… both ways you use matrix/vector multiplications to get from Raw RGB vector to the final cieXYZ/D50 vector (post WB still in the same colorspace), so you can always find WB vector to be used first with Raw RGB vector matching in the end your final cieXYZ/D50 vector in your calculations, no ?
If the RGB white balance correction in the camera’s EXIF were a 3×3 matrix, I’d have to agree with you. But in all the cameras that I’ve seen, it’s a vector.
Jim
OK, the DNG spec is totally unclear but I think I got something: I used a color temperature of 3900K (as estimated by ACR) to interpolate the ForwardMatrix of the a7RII. I multiplied it by a diagonal matrix calculated with the ratio of the mean raw channel values from a central white patch in your frame 7515. The Y (2nd) row of the resulting CameraToXYZD50 matrix is
0.2669 1.0000 0.0448
Does this make sense? So those should be the weights? I tried the reverse ColorMatrix procedure but I must be doing something wrong because the results are quite different and I get a negative coefficient for blue.
Jack
Jack,
That sounds close enough for government work. I’ll run some plots with that curve in them, if you think that multiplying MTF50s by that ratio and normalizing isn’t playing too fast and loose. To do MTF50s on an image converted with those numbers would require demosaicing, which would make it an apple to the other curves oranges.
Jim
Never mind Jim, I am just trying to understand how it works. In fact I realized that I was inverting one time too many. The correct second row is
0.9300 1.0000 0.1142
closer to what I get with the reverse matrices. Interesting that red is right up there. As for utility I need to think about it a bit more. If you consider a neutral image such as the ones we are using as targets, those weights would probably be meaningful as-is.
Jack
I am really surprised at the red weighting. This doesn’t seem to be congruent with the distinctly green tint in the finder of a Sony a7x camera that’s adjusted for UniWB.
Jim, you are the man where color is concerned, so I may very well have made a mistake somewhere. It looks like red gets its boost from the CameraCalibration matrix and the CameraNeutral (WB) values. It appears that your illuminant is closer to 4000 than to 5000K.
Jack, let me think this through. UniWB previews are greenish. Equivalently, light that is balances so that it has the same white point as the sensor is magenta. Magenta is red plus blue. So the sensor’s white point is greenish compared to D55 light. So if I’ve got sensor data, and I want to make it into a space with a D55 (or D65 — they’re close enought to each other for this thought experiment — I need to boost the red and blue channels. And that’s what your weighting vector does.
So I guess I’m coming around to your point of view.
Now, to the larger question of the advisability of weighting the MTF50 curves by the weights you’ve come up with, I just don’t know. I think I’ll run an experiment. I’ll take a wet of ARWs and run them through DCRAW/MTF Mapper for raw color plane results. I’ll also convert them with DCRAW/AHD and look at them with Imatest.
Alternatively, you could supply me with a DCRAW and MTF Mapper command line incantations for AHD and luminance MTF50. Then I wouldn’t have to use Imatest at all. I think I’ll try it with Imatest for just one run, and see what happens.
Thanks,
Jim
These are all at f1.8 right?
Much harder to focus manually stopped down, and there’s quite a bit of focus shift. Interesting to see how the Eye of Kasson goes against the machine at, say, f5.6…
It also looks like the Siemens star with the small spot is very consistent. But consistently wrong in its choice of red as the channel to focus on. A hint for your Batis focusing quest?
> If the RGB white balance correction in the camera’s EXIF were a 3×3 matrix, I’d have to agree with you. But in all the cameras that I’ve seen, it’s a vector.
it does not matter = you start with a vector and you will end with a vector in your approach… so I can always find a per channel multipliers that will get me (with Raw RGB to cieXYZ/D50 color transform matrix in mind) to the exactly same final vector as yours….