This post is part of a series about some experiments I’m doing combining space and time in slit scan photographs. The series starts here.
The experiments that I’m doing can use a lot of computer resources, if you’re not paying attention, and even if you are.
I decided to scale the 8000 pixel-wide images down to 1000 pixels. While images of that size wouldn’t produce decent exhibition prints, I thought that I could use them to develop my computer programs, and also to find out what strategies worked best with what images.
I brought about 8000 images into Lightroom, selected them all, and exported them as 1000-pixel-wide JPEGs. That was at three yesterday afternoon. About nine this morning, here’s what the Lr progress bar looked like:
The two operations are the export and producing the previews. Looks like it’s going to take a lot more than 24 hours to do the export.
Here’s what the Windows Task Manager says is going on:
It looks like Lr can only get about two threads working in parallel. You’d think that an operation like this would be disk bound, but Lr’s inability to run the individual image exports in parallel is slowing things to a crawl. I’ve noticed before that If I dis three or four exports in rapid succession, Lr would parallelize them all. Seems it should be able to figure out how to do that for one big — OK, huge — export.
I wrote some Matlab code to do the downsizing. In spite of the fact that I wrote it as a single thread (I don’t have the Matlab Toolkit for parallel processing), the Matlab code runs considerably faster than Lr, although it looks like it will take about 5 hours to downsize 8000 files.
You may be wondering what all those Photoshop GPU Sniffer tasks are. Ever since I upgraded to Ps CC 2017, Ps has been spawning one of them every time it’s launched, and neglecting to kill it when it’s closed down. I have to go into the Task Manager and manually kill htem every so often.
I use Riot, it has batch processing, but I dont know how it behaves with 8k images torture
Giulio Magnifico says
Lightroom is the most non-optimized app ever…it’s fantastic to develop and organize photos (I’m using only it on my workflow) but it’s slow slow and slow! Especially on the 5k iMac (i7 32gb ram).
Thanks for your tests, I’ll follow your investigation 🙂
Tony A. says
Maybe this will be moot because next time you might capture reduced jpg images up front, but for processing large number of files I’ve found that DxO Optics Pro uses every processor core, each one dedicated to one image at a time. It was fun to watch a 6-core AMD work it’s way through a large list.
It’s not going to be moot, because I want to work with the small images only to develop the strategy to apply to the large ones.
Thanks for the tip.
You will probably find ImageMagick a highly flexible and efficient solution.
Richard Scobie says
I note a number of people on the web who are having “GPU Sniffer” issues, have fixed it by re-installing the graphics card driver.
Worth a shot if you’ve not already done it.