Optimal CFA spectral response

There is a widespread belief about Bayer color filter array (CFA) response spectra. It goes something like this:

1. The old CCD cameras used to have great color.
2. The reason they had such good color is the dye layers in the CFA were thick.
3. Thick dye layers led to highly selective (narrow band) spectral responses.
4. Modern CMOS cameras have lousy color.
5. The reason is that they are trying to reduce photon noise in the images.
6. Making the CFA filters less selective reduced the photon noise, but made the colors bad.

I have seen no evidence that the color is better  — which, for the purpose of this post, I’ll define as more accurate — on old CCD cameras than it is on new CMOS ones.  I’ve owned and used several old medium format CCD cameras and backs, and I don’t think the color was more accurate.

I decided to apply some analysis to the issue. Jack Hogan has demonstrated that you can get pretty darned close to a Macbeth CC24 Sensory Metamerism Index (SMI) of 100 with three Gaussian spectral responses of well-chosen peak wavelengths and standard deviations. I fired up a Matlab simulator that started a long time ago with some code I got from Jack, made some changes, and found such an optimal CFA response set for the CIE 1931 2-degree Standard Observer:

There are four quadrants in the above chart. On the top right is the response of the optimal Gaussians. The standard deviations in nanometers are given in the subtitle. To the left of that is the error, in CIELab DeltaE 2000, for each of the Macbeth CC24 patches, with an optimal compromise matrix. The overall SMI, 99.3, is shown. A perfect SMI is 100. I’ve never seen a real consumer camera get above the low 90s. Most real cameras are in the 80s.  Below that, on the left, in red, is the response of the simulated camera and compromise matrix to a range of spectral inputs spanning the visible wavelengths. The blue curve is the correct response.

The bottom right graph needs some explication. I simulated a camera with a full well capacity of 10,000 electrons, or a camera with a FWC of 40,000 electrons at two stops above base ISO. I simulated an exposure sufficient to illuminate the raw values of the lightest gray patch to 95% of full scale. I simulated a 40,000-pixel patch, and, assuming a Poisson distribution for photon counting, measured the mean chroma noise in CIElab DeltaAB. The average for all 24 patches is a bit over 6 DeltaAB.

Now let’s tighten up the standard deviations, and make them all 20 nm. Then we’ll search for the optimal wavelengths for the peaks:

The color accuracy is much worse, but about the same as many consumer cameras, and the noise is slightly better. This is with the same exposure as the top chart. Since the filters are more selective and less light is hitting the sensor, the signal to noise ratio of the photon noise is worse, but the compromise matrix means that that degradation doesn’t result in greater chroma noise.

If we tighten the standard deviations down to 10 nm and reoptimize the peak locations, here’s what happens:

Now both the SMI and the chroma noise is worse, and the locus of spectral colors is very short.

What happens if we make the standard deviations large?

The SMI isn’t too bad, but the chroma noise is worse, in spite of more light hitting the sensor. The spectral coverage looks pretty good.

Here’s the reason for the increased chroma noise:

Note the size of the off-diagonal terms.

Here are my conclusions from this exercise:

1. Too much overlap is bad for color accuracy.
2. Too little overlap is bad for color accuracy.
3. For optimal SMI performance, the red and green center wavelengths should be fairly close.
4. For optimal SMI performance, the red and green overlap should be larger than we see in most cameras.
5. You can achieve remarkably high SMIs — much higher than we see with consumer cameras — with simple Gaussian spectra.
6. The CFA spectra for high SMI isn’t that far off the CFA spectra for low chroma noise.
7. My guess is that the reason we don’t have better SMIs in consumer cameras is the availability of chemical compounds, not noise considerations.

Caveats:

• I optimized and tested with the same patch set. This is not ideal.
• 24 patches is a small number.
• I didn’t simulate demosaicing.