How to judge a camera’s imaging

[If you’re wondering where yesterday’s post on the base ISO of the a7RII went, I pulled it. Bill Claff, Jack Hogan, and Iliah Borg helped me see that my model for an Aptina DR-Pix camera was grossly oversimplified, resulting in erroneous conclusions. I hope to have a better version to post before too long.]

A few days ago, I presented my findings on the below-base ISO settings on the Sony a7RII. I also posted on DPR the test results and my recommendation against using the below-base ISOs, which I called “fake ISOs” because they don’t actually reduce sensor sensitivity as measured by the raw file values. The thread rapidly reached the maximum number of posts and was therefore locked.

A lot of the posts amounted to an argument between people who, as I did, saw no difference in a7RII images exposed at ISO 100 and those exposed at ISO 50, save a difference in the preview image, and those who saw a big difference. It seemed like the first group was, as I was, looking at the data in the raw files to reach their conclusions. It seemed like the second group was looking at the images as displayed in their favorite raw converter to make their judgements. Both Lightroom (Lr) and its under-the skins twin Adobe Camera Raw (ACR) automatically reduce the brightness of ISO 50 a7RII images by a stop from those of ISO 100 a7RII images, the people who used those programs to draw their conclusions saw differences between the results of the two ISO settings.

It seemed to me that the two groups kept talking past each other because they were using different standards. I have seen other threads on DPR where this seemed to happen.

So I took a poll on DPR, asking how many people ever looked at the raw values in their files. Turns out, about three quarters of the respondents never look at the raw files themselves. There are two populations out there, and the population that never looks at raw values or images outnumbers those that do. As one of the responders to the poll pointed out, the actual proportion may skew even more dramatically to the never-look-at-raw group, since the slightly techie title of the poll may have prevented members of that group from participating.

What’s this mean to Internet discussions of cameras? I think it might explain many frustrating and inconclusive conversations about camera imaging characteristics.

Consider this generalization of the fake ISO discussion.

• Someone makes a claim about something the camera does based on looking at the raw files.
• Someone else challenges that claim based upon looking at the converted results in one or more raw developers.
• Discord and discontent ensue.

Or it could go the other way:

• Someone makes a claim about something the camera does based on looking at the raw files as developed by, say, Lr.
• Someone else challenges that claim based upon looking at the raw files.
• Sparks fly.

One of the insidious things about these discussions is usually that the different means of reaching conclusions is not explicit and clear. Even if it were, the people who are looking at the raw files might say their way is a better way to evaluate a camera, because the raw files show what the camera is doing, while the developed files show what the combination of the camera and the raw developer is doing; you can’t tell what the camera is doing by looking only at what the developer produced. The people basing their conclusions of the developed files might say that their way is better because no one can use an undeveloped raw file for a normal photographic purpose, and the best way to judge a camera’s suitability for a particular task is to perform that task and look at the results.

One of the trends of the past decade or so, led by Adobe, but followed by many, is for raw developers to become more capable and more opaque. Lr does more and more to — and/or for – your images than it used to, and it’s harder and harder to figure out what exactly it’s doing. Because of that, I expect this problem to get worse in the future.

I don’t see any easy resolution of this schism, but I do think that recognizing that there are two camps here, both with some justification for their perspective, may be the first step toward bridging the gap.