Testing IBIS on the Kolari-modified a7II

So far, I’ve written 18 posts on the prototype Kolari-modified Sony alpha 7II with various rangefinder lenses attached, and also, as a control, with the Sony-Zeiss 35mm f/2.8 FE. The series starts here. Kolari has been shipping thin sensor stack modified a7’s for quite some time, and the a7II sensor and sensor stack are very similar. So what’s the fuss about an a7II version?

The difference, from: Kolari’s perspective, is that the a7II has in body image stabilization (IBIS), and the a7 doesn’t. Kolari’s modification changes the weight of the sensor stack, and that could conceivably affect the way that IBIS performs. The folks at Kolari asked me to look into that.

I first tried to perform the same sort of handheld MTF testing that I performed on the a7II on both a modified and an unmodified camera. There was too much noise in the results for me to draw any firm conclusions. The noise comes from the non-repeatability, from shot to shot and session to session, of the handholding steadiness. I might be able to solve this by doing many more than 16 shots per data point, but the testing is already laborious and time-consuming. In order to reduce the noise by a factor of two, assuming that my unsteadiness is Gaussian, I need to go to 64 shots per data point. To reduce the noise by a factor of four would take 256 shots per data point. I can’t bear to even think about that.

After looking at the noisy handheld results, I came to the conclusion that I needed a repeatable, mechanical way to vibrate the camera. I started out by purchasing a used laboratory orbital shaker, which originally served to agitate samples in the hospital setting. It was cheap. It was sturdy. However, when I finally received it, it was broken.

I swallowed hard and bought a new one. It’s amazing what you can buy on Amazon. This one had a digital speed setting device, rather than the potentiometer of the first one, which should serve to make the results more repeatable. The way these things work is that they move a tray horizontally in about 1 inch circle. The speed at which a complete rotation takes place is variable between some low value and 300 or 500 RPM. The size of the circle is not variable.

I put the camera on a tripod, focused on the target, put 1 leg on the tray, and turn the shaker on. Whoops! The circle was way too big – so big, in fact that the tripod started walking around the room. I got a piece of aluminum foil and put it under the bottom of the tripod leg that was on the tray, so that it could slip. Better, but no cigar. The motion was still too great.

Then I had an idea. The shaker came with some magnetic posts that would stick to the tray. What if I put a rubber band around one of those posts, and attach the other end to the tripod leg? I could control the excursion of the tripod leg to some extent by the tartness of the rubber band. I wouldn’t have repeatability from session to session, but I would definitely have it within a session. I tried it, and it seemed to work just fine.

The next question was: how fast to tell the tray to move? I started with 60 RPM, or one cycle per second. Too slow. The IBIS seemed to consider motion that slow to be the users deliberately aiming the camera at a new point, and it did very little to correct it. I turned the speed all the way up to 300 RPM, or 5 Hertz. The IBIS seemed to consider that speed the jitter of an unsteady hand, and did what appeared to be a great job of correcting it when I looked at the magnified life your display.

Here’s the test protocol:

• A single Fotodiox LED-200WA-56 daylight balanced flood.
• Sony-Zeiss 55mm f/1.8 FE lens.
• ISO set to 800, f-stop set to 8,
• Focusing manually on the Siemens star in the slanted-edge target using maximum magnification and focus peaking wide open,
• Drive set to single
• EFCS on
• Manual exposure mode.
• Camera on tripod that’s attached to the shaker with a rubber band.
• Exposure protocol: Light to maximum,, shutter to 1/500 second, make 16, Turn the light down a stop, make 16 exposure, until you get to 1/15 of a second.
• Make one series with IBIS on, and one with it off.
• Develop in Lightroom CC 2015 with standard settings.
• Crop, export as TIFFs, analyze for vertical edge MTF50 in Imatest version 4.
• Find the files for which Imatest improperly identified the ROI,  and replace them with the extra files. This is a manual, fiddly, boring, unpleasant operation.
• Export the results to Excel, crunch the stats, and graph.

I did a test run with the Sony 55, with both cameras with the IBIS off.

These are the results:

The vertical axis is the MTF50 for the image: the point where, as the spatial frequency of the subject matter increases, the response on the sensor is reduced to half its very-low-frequency value. The units of spatial frequency are cycles per picture height. Since the a7II has a picture height of 4000 pixels, the highest that the MTF50 should ever get is 2000 cycles per picture height. 1500 is exemplary performance, and not something we are likely to ever see with edges perpendicular to the strong axis of the a7II’s anti-aliasing filter .

The horizontal axis is the shutter duration. The points on the graph correspond to 1/500, 1/250, 1/125, 1/60, 1/30, and 1/15 second.

The heavy lines are the average for all 16 exposures. The red is the Kolari-modified camera, and the blue is with the standard one. The light lines are the average plus one standard deviation and the average minus one standard deviation. If the statistics for handheld MTF50 turn out to be Gaussian, about two thirds of the images will fall between the two narrow lines. Mu stands for mean. and sigma for standard deviation.

The fact that the modified camera does better than the unmodified one at moderate and fast shutter speeds is probably due to the fact that the modification removes the antialiasing filter.

Turning IBIS on, we see the following:

It’s clear that, in this case, the IBIS operation is in no way impaired by the Kolari modification.

A few caveats are in order. This is a test that does not subject the cameras to all the possible forcing functions that they would see in normal handheld operation, so it is not definitive. It is clear that the IBIS performs better with the shaker 5 Hz input than it does with normal handheld jitter, probably because the shaker excitation is always within the amount that can be corrected with the amount of travel available in the IBIS mechanism, but also possibly because higher frequencies are present in human hand holding. I could experiment with different forcing frequencies, but I have no idea what the proper frequency/amplitude envelope for human ones is.

To really do this right, we’d instrument a camera being held by a person and measure 6-axis g forces. Then we’d mount the unit under test on a 6-axis electronic shaker and feed it the waveforms measured at the camera.

I suspect that’s how Sony and Nikon do it, but that experiment is beyond my expertise, spare time, and pocketbook.