Today I turn the read noise analysis tools developed over the past couple of weeks to a new camera, the Leica M240. The M240 is one of those cameras that subtracts out the black point before it writes the raw file, so we won’t get a clear picture of the read noise from a dark-field exposure. We know from previous experience that the M240 is over-enthusiastic in its subtraction below ISO 1000, resulting in the “green shadows” problem. As we will see, there’s a method to Leica’s madness; the subtraction reduces low-frequency read noise in the dark-field images, making the camera test better. However, that doesn’t help photographers trying to make real images, since a little light falling on the sensor is enough to reveal the underlying read noise in all its ugliness. So there really is no benefit to the in-camera processing that brings us the green shadows.
Here are curves with one-dimensional averaging with various kernel sizes in both the horizontal and vertical directions:
The vertical and horizontal curves are very similar. ISOs 200, 400, and 800 enjoy the benefit of the aggressive black point selection.
We can look at the curves referenced to the input side of the pre-ADC amplification:
Note that the read noise numbers are much smaller than you’ve seen elsewhere. That’s because, thanks to Leica’s raw processing, we’re not seeing all the real read noise; it’s being cut off on the left side of the histogram. You should double the read noise for ISOs of 1600 and above to get an estimate of the real number, and ISOs below that will need even greater compensation. ISOs 1600, 3200, and 6400 are right on top of each other, indicating that the camera is “ISOless” through this range.
If the M240 read noise were white Gaussian noise, the above two graphs would look like this:
If we normalize the two sensor-referred graphs to the ideal, white-noise case, we get these two plots:
I suggest ignoring the curves for ISOs 200, 400, and 800; they’d probably look like the other ones if Leica didn’t do its black point trick at those ISOs. The other curves are on the good side of what we’re now starting to recognize as typical low-frequency performance. It is commendable that there is littl difference between the horizontal and vertical numbers.
Next up: pictures.
Several people have told me that they thought that the read noise patterns from CCD sensors were less objectionable than the patterns from CMOS ones. I have an old Hasselblad, the H2D-39, and I performed an analysis.
There’s a problem with looking at the read noise of this camera. Unlike the first two cameras I looked at, the Nikon D810 and the Sony alpha 7S, the Hasselblad cuts off the left side of the read noise histogram for dark-field images. You’ll still see all the read noise with low photon counts that lift the average exposure level away from the black point, but consistently achieving flat field photographs at those light levels is next to impossible with the equipment available to me.
Undaunted, I proceeded ahead with dark-field image samples. Real read noise will be about twice as bad, and may have somewhat different frequency characteristics.
I took an image made of blackness with the ISO control set to 200 and the shutter speed to 1/800 second and analyzed the frequency characteristics by convolving it with one-dimensional and square averaging kernels of various sizes:
The flattening of the curves indicates greater low-frequency content than we’d see if the dark-field images were white noise. We can see that more clearly by comparing the square-kernel numbers with what we’d get for white noise:
With one-dimensional kernels, we see similar effects:
If we compute the ratio of the actual values to those that would obtain for white noise, we can get a numerical measure of departure from the ideal case:
These curves look a bit better than the Sony a7S ones at low ISOs, about the same as the a7S in the worse direction at high ISOs, and worse than the a7S in the better direction at high ISOs. It’s only a sample space of one, but there’s no evidence in this data that CCDs are any better then CMOS sensors for low frequency read noise patterning.
Let’s look at some dark field images with various filtering. As before, the images have been scaled into the range [0,1], have had a gamma curve of 2.2 applied , been res’d down to 640×480, and JPEG’d.
With no filtering:
The relative darkness of this image after normalization is a result of the Hasselblad having uncorrected hot pixels.
With a set of 36-pixel kernels:
With 306 pixel kernels:
The Sony a7S switches to high-conversion-gain mode as the ISO dial is moved from 1600 to 2000. Thus ISO 3200 is the first whole f-stop ISO to get the benefit of increased conversion gain. In this post, I’ll show ISO 3200 dark-field images filtered with the same set of kernels as the ISO 100 images in the preceding post.
First, the unfiltered image:
Now, with 11-pixel kernels:
Now with 107 pixel kernels:
107 pixel vertical averaging kernel
And finally with 871 pixel kernels:
The plots indicated that the ISO 3200 read noise was higher frequency, and therefore less objectionable, than the ISO 100 noise. The way that these images are scaled prohibits absolute comparisons — for that you’ll have to look at the plots — but seems to bear out the conclusions from the plots.
Objectionable qualities being in the mind of the beholder, you’ll have to judge for yourself.
In the previous post, I showed plots of data derived from a series of Sony a7S images of the back of a lens cap at various ISO settings. Now I’d like to show some of the filtered images themselves. Since there are 540 images represented in the plots (10 ISOs, 18 kernel sizes, and 3 kernel shapes), I’ll have to be selective. All the images you’ll see in this post have been scaled into the range [0,1], have had a gamma curve of 2.2 applied (imageGamma = imageLinear^0.45), been res’d down to 640×480, and JPEG’d.
First, the ISO 100 unfiltered image:
What looked pretty awful on the graph doesn’t look bad at all in the flesh.
Here is the ISO 100 image with an 11×1, 1×11, and 11×11 averaging kernel applied:
Both horizontal and vertical low-frequency variations are visible.
Let’s make the kernel bigger and see what we can see:
Now let’s look with a really big set of kernels, like those that produced the rise in standard deviations shown in the previous post.
It’s unusual in modern Sony sensor designs that the vertically-oriented read noise features are visibly more objectionable than the horizontal ones, but we see it here.
Today, I’ll apply the same read noise analysis approach of the last few posts to a new camera, the Sony alpha 7S. The a7S has the — unique in my testing — ability to change its conversion gain as a function if the ISO setting. The conversion gain jumps when the ISO dial is turned from 1600 to 2000, That turns out to dramatically affect the read noise.
First, let’s look at the raw red-channel (since there’s no light falling on the sensor, it doesn’t make any difference what channel you pick) dark-field noise at 1/8000 second as a function of a horizontally-oriented 1xn averaging filter. This filter retains vertical variations, while eliminating more and more horizontal ones as the kernel size increases. The read noise in this case is the same as the dark-field noise, since the a7S does not cut off the bottom of the dark-field histogram like some cameras.
All of the curves have a positive second derivative (their slope is increasing), except for high ISO at large kernels. If the read noise were white, that wouldn’t be the case. In addition, all the plots at ISO 1600 and below, with the exception of the ISO 200 curve, have a pronounced rise in level as the kernel size increases after about 100. It’s not clear what this means, but I’ll show you the images in a future post, and maybe we can figure it out..
If we filter the image with a vertical kernel, we get this:
Roughly the same situation, except the systematic change of slope for all the curves is more pronounced. This means that more of the low frequency read noise is in the vertical than in the horizontal direction.
If the read noise of the a7S were white, the curves, referred to the input of the pre-ADC gain stages, would look like this:
The read noise starts out as about 12 electrons at ISO 100, and comes down as ISO is increased to 1600. Then it makes a big jump at ISO 3200, and stays close to that low level as ISO is further increased. As in the immediately preceding posts, these read noise numbers are higher than others have reported, and, indeed, higher than I have previously reported, because I measured teh standard deviation of the noise across the entire sensor, not just a small square in the center.
Normalizing the filtered curves to the ideal (white) behavior, we get the following plots:
At high ISOs, the low-frequency component of the read noise raises the standard deviation by a factor of 4 for vertical averaging, and less than 2 for horizontal averaging. At lower ISOs, things get somewhat worse, with ISO 100 being an outlier.
I published some statistics yesterday on D810 dark-field images of various ISOs that had been filtered with low-pass filters of various sizes. I used 1xn kernels to filter out horizontal variations and leave the vertical ones, and nx1 kernels to to filter out vertical variations and leave the horizontal ones.
As an adjunct to the curves, I’m posting some of the images themselves. I used symmetric reflection to get pixels for the kernel to process when it was near the edges of the image, and cropped the resulting image to its original size. The, for each image, the black point was subtracted out, the images scaled into the range [0,1], a gamma of 2.2 applied, resized to 640×480, and saved as JPEG’s.
The unfiltered image:
Can’t see much, can you? This illustrates how good the D810’s read noise really is, but you can’t count on leaving the shadows alone when you’re editing an image, so let’s look at the filtered images.
With a horizontal kernel of 11 pixels extent:
With a vertical kernel of 11 pixels extent:
With a square kernel of 11×11 pixels:
It’s hard to see much in the two one-dimensional kernels, and, indeed, the curves in yesterday’s post showed that they’re not too far from ideal. However, the square kernel shows that whole rows seem to vary at once.
With a horizontal kernel 216 pixels long:
Not the row variation is readily apparent.
With a vertical kernel of the same size:
Some column variation is evident.
With a 216×216 kernel:
This last makes the hot spot across the bottom of the image obvious.
I performed the horizontal and vertical filtering operations of the previous post on D810 dark-field images from ISO 100 to ISO 6400 in whole-stop steps. Here’s the resul, for horizontal kernels, which preserve vertical features:
And for vertical kernels:
You’ll notice that the horizontal kernel results are more nearly straight lines in the log-log graphs, indicating that the read noise in the horizontal direction is more nearly white. However, in both directions, especially at low ISO’s the noise stops being white within the kernel size, and the curves flatten.
Now let’s see what the curves would look like with the vertical axis referred to the input to the pre-ADC gain stages, so that the units are now electrons in the sensor photosites.
Those of you who have been looking at Sensogen or other sites reporting on D810 read noise will have noticed that the minimum values of the read noise in electrons are about twice as large as those on those sites. That’s because I am using the entire raw image for analysis, and the more usual sample for read noise is a 100×100 or 200×200 pixel block..
Here’s what the curves would look like if the read noise were white.
With vertical kernels:
And with horizontal kernels:
If we ignore the ISO 100 case, the low-frequency part of the horizontally-averaged curves are between one and a half and twice what they’d be with ideal noise. The numbers are about twice as bad for the vertically-averaged case.
A reader commented on the previous post, and posted this link to a web page where he analyzed the spatial aspects of the read noise on a Sony NEX-6. He contends, with excellent justification, that the construction of the sensor on that camera, and sensors with similar column-parallel ADCs, creates more low-frequency read noise in the row direction than the column direction (when the camera is in landscape orientation).
I thought I would adapt his convolution-based analysis techniques to my Nikon D810 ISO 12800 dark-field image, and see if that proved more illuminating than the frequency domain approach of he last post.
I constructed three families of convolution kernels: nxn (square), 1xn (row-filtering) and nx1 (column-filtering). I passed them in various sizes over the green channel of the raw D810 test, then measured the standard deviation of the resultant images:
You can see that there’s different behavior in the images filtered by the two one-dimensional kernels: that’s called anisotropy. The results are similar to what Ilya discovered with the NEX-6, but with two exceptions. First, the line labeled “Horizontal” has a slight upward curve to it at high kernel sizes, indicating that the noise isn’t white in that direction. Second, the Vertical and Horizontal lines diverge more slowly than in the NEX-6 results (which, to be fair, used a somewhat different methodology), indicating lower DAC noise with the S810 than the NEX-7.
By the way, note that the horizontal kernel preserves vertical features, and vice-versa.
Since I was doing all this filtering anyway, I thought I’d have the program write out the square-kernel-filtered images. What I saw surprised me. Here are some samples, all normalized so each maximum is full scale, and all encoded with a gamma of 2.2.
At a kernel size of 2, random hot pixels prohibit being able to see anything:
With a kernel of 11×11, the columns show much more variation than the rows, making DAC noise look like it’s the culprit:
With a kernel size of 36, a mottling is added to the mix, and it’s not all in one direction:
With a kernel size of 152, there is the crosshatching we saw in the previous post, and some areas of high noise are apparent across the bottom and on the left and, to a lesser extent,, the right side:
With a 434×434 kernel, the low-frequency systemic noise features are quite visible:
More of same at 1233×1233. By the way, the motion of the features is due to using mirroring near the edges of the images when the kernel needs more data than the image can provide.
What’s this all mean to my quest for finding a metric for the visual effects of read noise? It means it’s a harder problem than I thought. It also means that there may be opportunity to filter out some of the low-frequency read noise components by subtracting a low=pass filtered image, or, equivalently, varying the black point across the sensor field.
Beauty is in the mind of the beholder, so ’tis said. I guess ugliness must be as well. Maybe I’m on a fool’s mission, but I’d like to figure out a way to mathematically calculate the visual effect of read noise.
If you look at the histogram of dark-field noise for a digital camera, providing the firmware that creates the raw image hasn’t cut off the bottom, you’ll see something that looks approximately Gaussian, with a bit of over-representation in the upper regions.
However, it is widely believed that read noise is more damaging to image quality than a similar amount of Gaussian noise, because read noise usually forms a pattern with some regularity. The viewer detects that pattern, and it is more distracting than a similar amount of Gaussian noise.
That’s the theory, anyway. I thought I’d explore some of the details, and maybe come up with some kind of numerical description of how visually distracting read noise patterning is. Foreshadowing: I havn’t gotten there yet. The journey is the destination, though; join me on the journey.
Here’s the histogram of the central 512×512 pixels in one of the green channels of a Nikon D810 dark-field image made at 1/8000 second, ISO 12500, and high ISO noise reduction set to high:
The black point on this camera is 600, and the histogram is truncated a couple of hundred counts to the left of that. There is at least one pixel that is about 3000.
Here’s what the central 512×512 section looks like, with the black point subtracted out and scaled up in amplitude be a factor of 1000:
It looks like there is some kind of pattern, doesn’t it? The eye is really good at picking out patterns. Maybe too good it can make them up as well. Here’s a similar image of Gaussian noise — I haven’t matched the statistics:
Can you see mountains and plains? I can.
Many patterns are periodic. One way to find periodicities in images is to look at the Fourier transform. Here’s the magnitude of the Fourier transform of the 512×512 crop from the Nikon D810 dark field image:
There’s a vertical line. Let’s see what the origin looks like:
Except for the vertical line, I can’t see much. Let’s see some of the numbers:
Not real illuminating. What if we average the FFT image using a 3×3 kernel, then take a close look at the origin?
If looks like there are peaks in the vertical frequency axis just away from the origin.
If we look at the frequency distribution of the image spectrum averaged in each direction with 8 pixel wide buckets, we get this:
Except for the dc (zero frequency) component of the horizontal spectrum (that’s our white line), it looks like the dark field is white noise.
Let’s go back to the image in the space domain, and get rid of most of the high frequency information by passing a 25×25 pixel averaging kernel over it, then applying gain of 1000:
Now we can see some lumpiness and crosshatch features. What happens if we look at the spectrum of that image?
Are we looking at a pattern, or just the lowpass filter response of the 25×25 averaging? Here’s the spectrum of similarly-filtered Gaussian noise:
Hmm… We can’t see much of a pattern in the unfiltered read noise though there are hints. We can see a pattern if we lowpass filter the dark field image. And so far, I don’t have a way to assign a number to the patterning.