Yesterday I went back and looked at some of the simulation results, and grew suspicious that there were problems with the way the simulated camera was sampling the simulated target. As I dug deeper, I started to question some of my original assumptions, and started experimenting with changes. What I ended up with is close to, but not identical to, where I started. I will modify the posts of the last couple of days to reflect the small changes, but today I’d like to report on what I changed and why.
Warning, this will get detailed and geeky.
First off, let me explain how the simulator works. I start off with a RGB target image of about 12000×8000 pixels. I want to end up with a simulated sampled image that’s much smaller. How much smaller? Good question. Stay tuned.
Most of the simulation operates at the resolution of the target. I know the pixel pitch of the sampled image, and I can calculate the ratio of the size of the target to the size of the sampled image. That allows me to compute kernels to apply to the target in terms of the dimensions of the simulated sensor. The first thing that happens is that I construct a kernel to simulate diffraction, and apply it to the target. There are actually three kernels, one for each color plane, and they are computed assuming red light at 650 nm, green light at 550nm and blue light at 450 nm. Then I apply kernels to simulate lens defects. Then motion blur, if any. Then AA filters, if any, Then the sampling of the sensor – this is where the fill factor comes in. Then I do a point sample of the target image to the resolution of the sensor, assigning color planes to sensels with the RGGB Bayer CFA. Next, I add photon noise if desired. Photon noise is not a useful addition for slanted edge spatial frequency response analysis, since the algorithm averages out noise. Then I digitize at the desired precision; I’ve been using 14 bits for the MTF studies.
Now I have a simulated raw file. I “develop” it with bilinear interpolation demosaicing, and feed the resultant RGB image to Dr. Burns’ sfrmat3 function for the MTF analysis, and either save the results, or compute indirect measures from the MTF curves, like MTF50 and MTF 30 and save those measures.
What could go wrong?
The first thing that made me nervous was the amount of energy above the Nyquist frequency with diffraction-limited lenses at wide apertures. The second was instability in the MTF30 and MTF50 results when I ran series with close spacing between the f-stop or pixel-pitch values. With factors of 1.05 between the data points, I saw saw-tooth ripples in the results. Both of those things made me think that I wasn’t sampling the target right.
Was the target resolution sufficiently greater than the sensor resolution? I recoded the sim so that the ratio of the two resolutions was an explicit input, then I ran series with diffraction-limited lenses and perfect AA-less sensors – the toughest cases – at ratios of 8, 16, 32, and 64. 32 and 64 were substantially identical, but the other two were different. That told me that I needed target resolution of at least 32 times the simulated sensor resolution. That was a surprise.
The next thing to look at was sampling jitter. Before, the ratio between the target and the sensor could be anything. But I could only sample at integer target pixel indices. I thought about coming up with a way to interpolate between target pixels, but in the end, I took a simpler approach: crop the target so that it has dimensions that are integer multiples of the sensor resolution and the target to sensor ratio, which I constrained to an integer power of two. Thus the distance, measured in target pixels, between the sensor samples is fixed. Once I’d eliminated the sampling jitter, the MTF50 instability went away.
By the way, the target is already aliased. Here’s a close-up:
I still had a lot of aliasing. Was it real? I took a look at some of the simulated demosaiced images. Here’s one for an f/2.8 diffraction-limited lens on a sensor with 100% fill factor:
Aliasing? I’ll say! Want to see it at 1% fill factor?
Now that’s just silly, but I don’t think I can blame the SFR analysis for the aliasing.
So, at the end of all this technological navel gazing, I’m pretty much back where I started. I’ve tweaked the model to be more accurate, but my early results were darned close. At least my level of confidence has improved.
Why don’t other people see this much high-frequency stuff in their simulations? The ones I’m most familiar with are looking at raw channels, not demosaiced images, and that makes all the difference. Why don’t I look at raw channels, too? That’s not what I’m interested in; I want to know about aliasing in demosaiced images. I can’t print raw ones.