Constructing a compromise matrix

This is the fourth in a series of posts on color reproduction. The series starts here.

This is going to get pretty technical, so I’d like to first give the “See Spot run” version in this post, and get into the details in the next one. If you’re not into the techie stuff, you can just read this one.

We saw in earlier posts in this series that consumer cameras don’t have the right set of Color Filter Array (CFA) spectral responses for the cameras to see color the same way that people do. One way of getting color out of those cameras is to construct a compromise matrix, which, when multiplied by the black corrected raw image values will yield approximations to the colors the camera saw in a linear form of any desired RGB color space. That color space can be simply converted to a gamma-corrected color space like sRGB or Adobe (1998) RGB by applying the proper nonlinearity to each color plane.

How do we construct such a compromise matrix? We construct a target consisting of squares of constant color. We light the target evenly with a lamp whose spectrum is known and similar to that which will be used in actual photography, and measure each square’s color with a spectrophotometer. We convert those measurements to a color space which has some pretentions towards perceptual uniformity, such as CIEL*a*b* or CIEL*u*v*. Then we take a picture of the target. We demosaic the raw file, but don’t try to correct the colors. Then we construct (by scientific guessing) a starting point for the compromise matrix. We multiply the values in the image by the test compromise matrix to get the results in our preferred color space, then convert to the same perceptually uniform color space we used when we measured the target with the spectrophotometer.

Now we have the real colors and the measured colors in the same color space. We compare them, measure the differences, come up with some weighting scheme, and produce a single number (which mathematicians, engineers, and color scientists call a scalar) that describes how different the two sets of colors are. We fiddle with the values in the compromise matrix (in a most scientific and serious way), pass the raw image value through the new matrix, recompute our scalar error, fiddle again, and so on until we’re satisfied that the error is as small as we can make it.

Is it really that sloppy and ad hoc? Indeed it is. We engineers and scientists have come up with a name for the class that methods like these belong to: heuristic. Doesn’t that sound better than sloppy and ad hoc?

If we change the lighting much, say from photoflood to electronic flash, take a new set of measurements and a new picture and compute a new compromise matrix, it will be different.

If we change the target colors, the compromise matrix will be different.

If we change the anything in the math that determines the error scalar, such as which are the colors we consider important, whether we want to minimize the average or the worst errors, and when – and if – we decide that a color is so far off that we shouldn’t try to save it, the compromise matrix will be different.

It’s amazing this stuff works at all, but it does.