Noise performance of downsampling with eliptical weighted averaging

Over on the Luminous Landscape forum, Nicolas Robidoux, Bart van der Wolf, Alan Gibson, and others have been working on developing downsampling algorithms. They’re using the command-line image editor ImageMagick as the engine to do the work, so their methodology is to write scripts that call ImageMagick. Some of their algorithms are producing impressive results, some of which you’ve seen in the last few posts.

I took one of Bart’s scripts, version 1.2.2, and ran it on a Gaussian noise image. I’ll report the results in this post, but first I’d like to let Bart tell you something about the algorithm. If digital filter design isn’t your thing, fee free to skip ahead.

On my occasional perusals on the ImageMagick discourse server, I read about some of Nicolas Robidoux’s ideas about halo suppression, in which he suggested temporarily adjusting the tone curve before resampling and restoring the original tone curve after resampling. Having tested those ideas when Nicolas was still experimenting with sigmoidal contrast, I saw some benefits, but also lack of control over shadows and highlights with a single parameter. Now that he declared that round-trip to Sigmoidal contrast dead, and introduced separate gamma corrected resampling with blending, I was convinced about the usefulness of the principle.

When Nicolas suggested to investigate further, and with no takers, I volunteered to create a tool that allowed to test the concept a bit further and in a more structured manner.

All resampling methods that are used in the script, are based on Elliptical Weighted Averaging or EWA (therefore in two dimensions) instead of a two pass orthogonal tensor approach. This has the benefit of producing more ‘organic looking’ resizing because the resulting resolution is circularly symmetric, rather than having a higher diagonal resolution that is possible in a square pixel grid. BTW, EWA resampling also has benefits for other types of (asymmetrical or variable) distortion.

A drawback of EWA resampling, besides the higher computational cost of having to process more samples, is that a (nearly) No-Op (scale close to 1.0) resampling will be slightly blurry, instead of having no effect on sharpness. Therefore, and for adaptability to different subject matter and viewing conditions, a sharpening parameter is added which allows to modify the behavior of the resampling and windowing filter used.

There are different filter methods available for upsampling and for down-sampling, and they are chosen because they perform better (less artifacting) for the intended scaling. The script is set up to allow choosing either filter method regardless of the fact of upsampling or down-sampling, but that’s more for experimentation purposes.

The actual resampling results are a linear gamma space blend between two resampling operations. The resampling operations are performed in two different gamma spaces in order to cope with the different halo under-/overshoot amplitudes. The blending of those in linear gamma space is luminance driven which allows to address the different halo tendencies in darker and lighter tones, and it reduces the risk of clipping.

This is all based on a proposal by Nicolas Robidoux, after experimenting with other temporary contrast adjustment functions based on sigmoidal tone curves. The approach with resampling in separate gamma spaces allows to better target the suppression of halo under/overshoots in different luminance ranges.

The upsampling operation (labeled as ‘generic’, because it also does pretty decent down-sampling) is implemented by using the EWA version of ‘Lanczos’ filtering, also known as ‘Jinc’ filtering, to make it circularly symmetric when simply scaling both dimensions by the same factor. Some ‘deblur’ is used, the amount is user adjustable. The deblur is controlled by modification of the filter support and window size.

The down-sampling operation is implemented by using the EWA version of ‘Cubic’ filtering, with an adjusted parameter choice to use a relatively soft filter version (from the family of Keys filter parameters), which is achieved with the following ‘-define filter:c=0.1601886205085204 -filter Cubic’. That defines a somewhat blurry filter, but also relatively halo and ringing free, which helps because down-sampling needs to actively avoid aliasing and ringing artifacts. It also blends the results of two resampling operations that were done in different gamma spaces, to allow and control halo generation.

To compensate for the softness/blur of the Cubic filter version, deconvolution sharpening is added. The deconvolution uses a Gaussian blur based weighting, and is implemented as a Unity kernel subtracted by a Gaussian blur, with the ImageMagick DoG (difference of Gaussians) function, which produces a 2-D convolution filter kernel.

There is a different down-sampling operation that is used when the choice is made to eliminate all sharpening, e.g. because one wants to use a separate utility for that. That operation only uses the EWA version of the ‘Quadratic’ filter in linear gamma space, without blending with other gamma space results. That filter produces very clean, essentially halo free, results (thus the possibility to skip blending with non-linear gamma space resampling) with very little aliasing but it’s a bit blurry, and will therefore allow significant (e.g. local) sharpening without risk of enhancing existing halos. As a bonus, an experimental option is added to the script to add a simple separate deconvolution sharpening operation, when zero or ‘negative sharpening’ (i.e. blur) was used.

All operations are executed in the spatial domain, to circumvent potential issues with limited 16-bit precision calculations that will create artifacts in the (Fourier converted) frequency domain. The future versions of ImageMagick will apparently also be available as already complied binaries that allow floating point Fourier transforms, which may help with faster and more precise calculations. It may also generate some other resampling options.