Processing raw B&W infrared images

For the couple of weeks, I have been spending way too much time trying to develop a way to demosaic infrared raw files without interpolation. My reasoning is that, with a deep IR filter in front of the sensor, that the spectral response of each of the color filter array (CFA) planes is pretty close to that of the others, so if I can equalize the response of the planes over some largeish area, I won’t need to interpolate. The result should be a sharper image.

The first area I chose for equalization was the whole image. I calculated the average of each color plane, and then I equalized each pixel in each plane so that all four planes had the same average value. In another, similar, experiment, I left the two green planes alone and just equalized the red and the blue planes so that they had the same average value as one of the green planes. Both techniques produced substantially the same results.

I compared the results to ACR demosaicing with default settings and the Matlab Image Pack implementation of bi-linear demosaicing. My first conclusion is that there is a lot of secret sauce in ACR, which makes for quite a high bar in comparison tests. I had to add some home-brew deconvolution to my images to make for a fair comparison.

My third conclusion is that the anti-aliasing filter in my test camera, an IR-converted D3x, blurs the image hitting the sensor enough that there’s not much sharpness to be gained by interpolation-free demosaicing.

I grew frustrated with my inability to turn off all of ACR’s processing, and cast around for other alternatives. I found one with some real advantages for IR images: Iridient Developer (ID).  In version 2.1 there is a new IR-processing capability that you ought to know about if you make B&W images from raw files produced by IR-modified cameras.

I saw Brian Griffith at the April 27th CPA Raw Processing Workshop in Menlo Park, and he and I talked about the special problems of making B&W images from IR files. I proposed the no-interpolation algorithm, but Brian had another idea. No moss grows on Brian. Last week, a new version of ID, 2.1, came out, and it included advances in conversion of IR raw images.

In the B&W conversion tab (I’ll call it a tab, but it doesn’t really look like a tab) there’s a new option: the raw channel mixer. This option bypasses the transformation of the camera primaries to a standard color space, and does the mixing of the camera primary values directly. There is no contamination (and it might well be called contamination if your objective is a monochrome image) of the camera’s green channel from the camera’s red or blue channels. The upshot is, if you choose green-only, your converted image has only half the pixels interpolated, and therefore should be sharper then if it included red and blue components, which are three-quarters interpolated. Unfortunately, using only the green channel gives you a noisier image, by a factor of 1.414, but that shouldn’t be a problem in the mid to high values, and you could do a conversion weighting all three channels more-or-less-equally, and use that with a luminance mask to get better signal-to-noise ratio in the low values. Perhaps Brian will automate this in the future.

I’d determined before that my technique offered no advantages on images made with cameras that employed anti-aliasing (AA) filters. I figured that, if that were true, Brian’s new method wouldn’t either. I’m since done limited testing that pretty much verifies that.

Lloyd Chambers graciously sent me some raw files made with an IR-modified D800E. The images were exposed with lens that is very sharp in IR, the Coastal Optics 60mm f/4 Macro. Lloyd used a B+W 093 filter on the lens to make sure that near-IR wavelengths were excluded. By the way, I recommend Lloyd’s website to anyone interested in IR, or in photography for that matter. It’s mostly a pay site, but I think it’s well worth it.

I subjected Lloyd’s raw IR images to processing in ID 2.1 using the raw channel mixer with all channels represented in 40/30/30 ratio, and using the green-only option. I also processed them using my no-interpolation algorithm, which balances the red and green channels so that their average values across the entire image is the same as the upper right green channel in the mosaic. (The D800 has an RGGB pattern). The green channels were left alone, even though there were slight differences in the average values.

Comparison of the three sets of images has led me to the following conclusions:

1)      The interpolationless demosaicing works fine on the deep-IR images Lloyd sent me. The variation in spectral response of the CFA channels is low enough in the deep IR that there are no visible (to my eyes) artifacts even with considerable sharpening.

2)      There is some minor improvement in detail over using the raw green channels only in Iridient Developer 2.1. The improvement is not visible until both images are sharpened.

3)      The green-channel only image is sharper than one produced in Iridient Developer with the raw channel mixer set to the “neutral” preset, which mixes in the red and blue channels, which are presumably less sharp because more interpolation is necessary to produce them because of the relative population of the red-sensing pixels and blue-sensing pixels to the green-sensing ones. As with the previous comparison, it is necessary to sharpen the images to see much in the way of difference.

4)      All these differences are minor, and are not visible at all on cameras with a medium or strong AA filter.

If you are doing deep-IR work with the D800E or any other camera with no AA filter, I recommend that you take a good look at Iridient Developer. Heck, take a good look at it anyway; it has much to recommend it, including its (to me) delightful transparency.

I expected the differences to be greater. Lloyd suspects that having half the AA filter in place in the D800E may be having some effect.

I would show you some images, but they are not mine, they’re Lloyd’s.

Thanks to both Brian and Lloyd.