There’s an oft-told line about how the Apple II made its way into businesses in the late 1970s and early 1980s. Thousands of businessmen walked into computer stores and said. “Sell me a copy of Visicalc and something to run it on.”

I just bought a Mac. It’s been more than ten years since I bought the last one. The precipitating reason was so I could run Iridient Developer, which is Mac-only.

There are other reasons. Many of my friends use Macs exclusively. Some of them, when they find out I use PCs, treat me with a disgusting mixture of sympathy, contempt, and a kind of patronizing indulgence. I tell them that I’ve used Macs, but I know that my experience is so old that it’s not particularly relevant to today’s world.

A little history. In the late 1980s and early 1990s, when I first started doing digital image processing, the industry was in the process of moving from a model where the work was performed on proprietary, special purpose workstations from companies like Scitex, Dainippon, Crossfield, and Hell, to general purpose desktop computers. The computers of the time were barely up to the task, and there was a brief flurry of applications developed for Unix-based workstations from companies like Silicon Graphics. PCs and Macs were much less expensive, and Macs were a more attractive platform for two reasons: the graphic artists of the day already used Macs because it had become the de facto platform for desktop publishing, and PCs were still struggling with memory addressing issues left over from the DOS days.

So, when I started looking for a photo editing environment, the Mac was the place to be. Photoshop initially was Mac-only, and, although Picture Publisher, the best PC alternative, wasn’t a bad program, it was clear that Photoshop had the legs. So I got a Mac and Photoshop 1.0. I used Macs for about 12 years, while still continuing to use Unix workstations and PCs and PC clones.  During part of that time I was also in charge of IT for a medium-sized company, and we ran a mixed Mac/PC shop, so I got a good look at what it took to support the two platforms. Sometimes Mac support was painful, especially in the dark OS 7.x days, and during the transition from Motorola 68xxx processors to the IBM-based PowerPC architecture.

In the early part of the current millennium, I decided that all the apps I really needed ran on the PC, and I mothballed my lozenge-shaped, candy-colored tower Mac. I loaded OS X on it just ot see what the fuss was about, but never did serious work under that OS.

In my Macless ten years, I’ve always wanted to really try to use and OS X computer, but I it never got to the top of the priority list. Iridient Developer pushed me over the top.

I’ve been reporting on the process of getting the new Mac running in my geeky blog. I will say here that the ten years of development have made Windows and Mac OS X more similar than different, that I love having a real Unix machine again (OS X could be thought of as Unix with a pretty face), and that Apple networking in a Windows Domain is not  a trivial exercise.

And, I do like Iridient Developer.

For the couple of weeks, I have been spending way too much time trying to develop a way to demosaic infrared raw files without interpolation. My reasoning is that, with a deep IR filter in front of the sensor, that the spectral response of each of the color filter array (CFA) planes is pretty close to that of the others, so if I can equalize the response of the planes over some largeish area, I won’t need to interpolate. The result should be a sharper image.

The first area I chose for equalization was the whole image. I calculated the average of each color plane, and then I equalized each pixel in each plane so that all four planes had the same average value. In another, similar, experiment, I left the two green planes alone and just equalized the red and the blue planes so that they had the same average value as one of the green planes. Both techniques produced substantially the same results.

I compared the results to ACR demosaicing with default settings and the Matlab Image Pack implementation of bi-linear demosaicing. My first conclusion is that there is a lot of secret sauce in ACR, which makes for quite a high bar in comparison tests. I had to add some home-brew deconvolution to my images to make for a fair comparison.

My third conclusion is that the anti-aliasing filter in my test camera, an IR-converted D3x, blurs the image hitting the sensor enough that there’s not much sharpness to be gained by interpolation-free demosaicing.

I grew frustrated with my inability to turn off all of ACR’s processing, and cast around for other alternatives. I found one with some real advantages for IR images: Iridient Developer (ID).  In version 2.1 there is a new IR-processing capability that you ought to know about if you make B&W images from raw files produced by IR-modified cameras.

I saw Brian Griffith at the April 27th CPA Raw Processing Workshop in Menlo Park, and he and I talked about the special problems of making B&W images from IR files. I proposed the no-interpolation algorithm, but Brian had another idea. No moss grows on Brian. Last week, a new version of ID, 2.1, came out, and it included advances in conversion of IR raw images.

In the B&W conversion tab (I’ll call it a tab, but it doesn’t really look like a tab) there’s a new option: the raw channel mixer. This option bypasses the transformation of the camera primaries to a standard color space, and does the mixing of the camera primary values directly. There is no contamination (and it might well be called contamination if your objective is a monochrome image) of the camera’s green channel from the camera’s red or blue channels. The upshot is, if you choose green-only, your converted image has only half the pixels interpolated, and therefore should be sharper then if it included red and blue components, which are three-quarters interpolated. Unfortunately, using only the green channel gives you a noisier image, by a factor of 1.414, but that shouldn’t be a problem in the mid to high values, and you could do a conversion weighting all three channels more-or-less-equally, and use that with a luminance mask to get better signal-to-noise ratio in the low values. Perhaps Brian will automate this in the future.

I’d determined before that my technique offered no advantages on images made with cameras that employed anti-aliasing (AA) filters. I figured that, if that were true, Brian’s new method wouldn’t either. I’m since done limited testing that pretty much verifies that.

Lloyd Chambers graciously sent me some raw files made with an IR-modified D800E. The images were exposed with lens that is very sharp in IR, the Coastal Optics 60mm f/4 Macro. Lloyd used a B+W 093 filter on the lens to make sure that near-IR wavelengths were excluded. By the way, I recommend Lloyd’s website to anyone interested in IR, or in photography for that matter. It’s mostly a pay site, but I think it’s well worth it.

I subjected Lloyd’s raw IR images to processing in ID 2.1 using the raw channel mixer with all channels represented in 40/30/30 ratio, and using the green-only option. I also processed them using my no-interpolation algorithm, which balances the red and green channels so that their average values across the entire image is the same as the upper right green channel in the mosaic. (The D800 has an RGGB pattern). The green channels were left alone, even though there were slight differences in the average values.

Comparison of the three sets of images has led me to the following conclusions:

1)      The interpolationless demosaicing works fine on the deep-IR images Lloyd sent me. The variation in spectral response of the CFA channels is low enough in the deep IR that there are no visible (to my eyes) artifacts even with considerable sharpening.

2)      There is some minor improvement in detail over using the raw green channels only in Iridient Developer 2.1. The improvement is not visible until both images are sharpened.

3)      The green-channel only image is sharper than one produced in Iridient Developer with the raw channel mixer set to the “neutral” preset, which mixes in the red and blue channels, which are presumably less sharp because more interpolation is necessary to produce them because of the relative population of the red-sensing pixels and blue-sensing pixels to the green-sensing ones. As with the previous comparison, it is necessary to sharpen the images to see much in the way of difference.

4)      All these differences are minor, and are not visible at all on cameras with a medium or strong AA filter.

If you are doing deep-IR work with the D800E or any other camera with no AA filter, I recommend that you take a good look at Iridient Developer. Heck, take a good look at it anyway; it has much to recommend it, including its (to me) delightful transparency.

I expected the differences to be greater. Lloyd suspects that having half the AA filter in place in the D800E may be having some effect.

I would show you some images, but they are not mine, they’re Lloyd’s.

Thanks to both Brian and Lloyd.

Here are a few pictures from yesterday’s Raw Processing Panel, put on by the Center for Photographic Art.  I had a great time, and learned a lot. Thanks to Rex Naden for organizing and moderating the panel, and to my fellow panelists, Eric Chan (Adobe)
Charles Cramer, Brian Griffith, (Iridient Digital) and Lionel Kuhlmann (Capture One).

Here’s a link to the slides that I used in my presentation.

Brian Griffith explains the fine points of Irident Developer (formerly known as Raw Developer)

A question from the audience.

Eric Chan talks about Lightroom

Eric continues to explain Lightroom as Brian Griffith and Lionel Kulhmann look on

We had quite a few region photographic luminaries in the audience. Joe Holmes is at the far right, across the aisle from Brigitte Carnochan

I was unable to figure out why Lightroom is boosting the brightness and chroma of 32-bit floating point TIFFs imported into it, so I reluctantly decided to use the correctly-exposed synthetic image with the minus one-stop Lightroom Exposure adjustment as the reference and compute errors from it.

I computed the CIEL*a*b* Delta-E stats of the differences in the Lab values of the 121 patches in each of the “underexposed and compensated stop for stop less one stop for the Lightroom processing” patches. They’re similar to what I have measured in real camera testing, but with far less noise.

The overall stats:

For reference, a 3D look at the difference between the baseline exposure and itself expressed as displacements of the original target values:

A 3D look at the image that was exposed 4-stops under and corrected four stops more than the baseline in LR, processed the same way:

A 2D look at the same image:

The one, two, and three stop under plots are boring; they’re virtually the same as the four stop under graphs above.

These are fairly small errors. Still, it’s nice to look at how they break down.

Here’s a set of Delta-H curves, the basic CIEL*a*b* difference metric that measures hue error:

And the chroma error:

There’s more chroma error than hue error (that’s a good thing) and the chroma error is mostly in the direction of increasing the chroma (could be a good thing, could be a bad thing).

Looking at the hue angle errors, we see no bias in direction:

And finally, here are the results using the CIE Delta-E2000 Color-Difference Formula:

Thanks to Gaurav Sharma, Wencheng Wu, and Edul N. Dala for the Matlab code to implement this measurement.

This color difference formula is new to me, and fairly new to the color science community. It is supposed to be more accurate than the old Delta-E formula. Assuming that it is, it is telling us that the errors we are seeing are nothing to worry about, since half a Delta-E is said to be one just-noticeable difference in the new metric (down from the 1 Delta-E which was supposed to be one JND in the old CIELab Delta-E.

Eric Chan has informed me that there are two image-processing pipelines in Lightroom: output-referred, and scene-referred. Raw files get the scene-referred pipeline. Integer TIFFs get the output-referred pipeline. Therefore, all my TIFF test images were getting a different set of processing than LR applies to raw files.

I’ve done some testing with real raw files, and determined that Lightroom Exposure adjustments have mean errors of around 1 Delta-E when photographing the same test pattern off the monitor, and worst-case errors of about 4 Delta-E. However, there’s a lot of noise in the real camera testing, and the actual situation is probably better than that.

All of the work I’ve done with real raw images has taught me to appreciate working with synthetic images. No lens flare. No need to align the images (aligning has the really unpleasant side effect of making the pixels in the aligned image not in the same place as pixels in the camera, so dust spots on the sensor look like inter-exposure differences). No photon noise. Faster.

On the other hand, like looking for your keys under lamp-post, it doesn’t do much good to have a nice, tight testing regimen if if doesn’t say anything about how real raw processing takes place. Eric says floating-point TIFFs get the same scene-referred pipeline as raw images, so I thought I’d try to use 32-bit floating-point synthetic TIFFs for testing. It took me a while to learn enough about TIFF tags to have Matlab write floating point files that Photoshop and Lightroom like, but I’m there as of this morning.

Everything looks great in Photoshop. However, when the files are brought into Lightroom, they are much more chromatic and brighter than they are in Photoshop. When exported from LR as 16-bit integer TIFFs, they are still too bright and too chromatic. In addition, when analyzed in Matlab, the exported images have greatly distorted CIELab scatter plots, possibly because of gamut mapping, and possible because of something else.

I went back to LR, created a set of virtual copies, and cranked the Exposure adjustment back one stop on each. When I exported them and read them into Photoshop, the L* values were about right, but they were too chromatic.

In 3d:

And in just the chrominance plane:

It looks like LR is looking at the fact that the files I’m feeding it are floating point and is invoking some default processing that it considers appropriate for HDR images. If that’s the case, I need to find out where that processing is, and figure out how to turn it off.

I suppose I could call the above the new reference image and see what Lightroom does with exposure compensation of one stop for each stop of underexposure. I may yet do that. But I’m a little worried that I can’t import an Image into LR, do nothing to it, export it, and have it be pretty much the same.

[Added after the original post. Eric Chan has informed me that there are two image-processing pipelines in Lightroom: output-referred, and scene-referred. Raw files get the scene-referred pipeline. Integer TIFFs get the output-referred pipeline. Therefore, the TIFF test images are getting a different set of processing than LR applies to raw files.]

I can’t just leave you with a whole bunch of unexplained graphs, can I? In order to make sense out of all the data on color errors when Photoshop Exposure adjustment layers and Lightroom Exposure control tweaks are used to correct underexposed images, I’ve computed the CIEL*a*b* Delta E’s for each of the 121 color patches in the test image and averaged them, giving an average Delta E for each program for pushes between zero and four stops.

Here’s the graph:

I’ve plotted the results for both Process Version 2012 (the current Lightroom algorithm) and Process Version 2010 (the Lightroom 3 algorithm). There isn’t much to choose between them.

To get the curves, I corrected all the values in the sample images by dividing them by the ratio of the mean of the luminance values in the sample image and the mean of the luminance values in the test (exposed for a 0 EV push) image. That corrects for Exposure slider settings that aren’t quite right.

The Lightroom color errors aren’t huge, but they are significant. They make me think twice about blithely underexposing and pushing hard in Lightroom. The Photoshop Exposure adjustment layer seems to work virtually perfectly.

I would expect Adobe Camera Raw to perform like Lightroom.

[Added after the original post. Eric Chan has informed me that there are two image-processing pipelines in Lightroom: output-referred, and scene-referred. Raw files get the scene-referred pipeline. Integer TIFFs get the output-referred pipeline. Therefore, the TIFF test images are getting a different set of processing than LR applies to raw files.]

Bill Janes suggested that I run the Lightroom Exposure control tests with PV 2010 (the way the control worked in Lightroom 3.x). Here are the results in 2D first, and then 3D.

The baseline image with all processing stages, and an Exposure value of 0:

EV -1:

EV -2:

EV -3:

EV -4:

The baseline image with all processing stages, and an Exposure value of 0:

EV -1:

EV -2:

EV -3:

EV -4:

[Added after the original post. Eric Chan has informed me that there are two image-processing pipelines in Lightroom: output-referred, and scene-referred. Raw files get the scene-referred pipeline. Integer TIFFs get the output-referred pipeline. Therefore, the TIFF test images are getting a different set of processing than LR applies to raw files.]

Now for a three-dimensional look at the results from the previous post. First, the Lightroom images. The a* axis is on the right, and the b* on the left.

The baseline exposure:

Basically, there’s just a little bit of noise. All the values are between L* = 48.85 and a hair over 48.95.

One stop underexposed:

Two stops underexposed:

Three stops underexposed:

Four stops underexposed:

Pretty much the same story in all of the underexposures; the blue/cyans and magentas are darker than they should be.

I’ll spare you looking at all the Photoshop images. Here’s the worst, the four-stop-under one:

Just noise on the L* axis, and not much of that. A really fine performance.

[Added after the original post. Eric Chan has informed me that there are two image-processing pipelines in Lightroom: output-referred, and scene-referred. Raw files get the scene-referred pipeline. Integer TIFFs get the output-referred pipeline. Therefore, the TIFF test images are getting a different set of processing than LR applies to raw files.]

In the previous post, I reported that the Exposure controls in Lightroom and Photoshop work differently, and I showed some of the differences in tonality. Today, I’d like to consider color in all three dimensions. The question that I’m trying to resolve is, what are the characteristics of each app’s exposure control when used to develop images that are deliberately underexposed according to the precepts of either Unity Gain ISO, or the Signal-to-Noise (SNR) driven method that I’ve talked about in the last few months.

In order to get an appropriate test, I created a test chart consisting of a 11×11 array of squares, with lab values from -50 to +40 in both a* and b*, and a constant L* value of 50 (a middle gray tonality). Here’s the chart, in ProPhoto RGB:

Then I went to my camera simulator program, and created a new camera. It’s in all respects like a Nikon D4, except that it has no noise except for electron and ADC quantizing noise, and a Fovean-like sensor that directly encodes the light into the ProPhoto RGB color space. I know, I know; you could never build a camera like that, especially since two of the PP RGB primaries aren’t physically realizable, but I wanted to minimize color space conversions and errors due to demosaicing.

With my simulated camera, I made five exposures of the target, one at the exposure that replicated the target tonality, and one each at – 1EV, -2 EV, – 3EV, and -4 EV from the first exposure.

I opened each exposure in Lightroom, and used the exposure control to compensate the four underexposed images to about the same RGB values (in the middle of the image) as the normally exposed image. To my surprise, it didn’t take exactly one EV of correction for each EV of underexposure; it took somewhat less.

I exported all the images from Lightroom in ProPhoto RGB, brought them into Photoshop and converted them to CIELab, then brought those images into Matlab and analyzed them.

First, I looked at the chromaticity — only the a* and b* values. Here’s the normally exposed image:

The one-stop underexposed image:

The two-stop underexposed image:

The three-stop underexposed image:

and the four-stop underexposed image:

As you can see, there is a loss of chroma in the blues and magentas.

Then I performed the analogous (but not equivalent) operations in Photoshop, using the Exposure adjustment layer. Like the exposure control in Lightroom, I didn’t have to slide the slider as far to the right as I would have thought. It was much easier to match the levels of the five images, since the Info tool in Photoshop can read out in Lab, something that you can’t do in Lightroom — yet.

Here’s the normal image with no adjustment.

The one-stop underexposed image: