• site home
  • blog home
  • galleries
  • contact
  • underwater
  • the bleeding edge

the last word

Photography meets digital computer technology. Photography wins -- most of the time.

You are here: Home / The Last Word / More slit scan experiments — processing speed

More slit scan experiments — processing speed

November 16, 2016 JimK 6 Comments

This post is part of a series about some experiments I’m doing combining space and time in slit scan photographs. The series starts here.

The experiments that I’m doing can  use a lot of computer resources, if you’re not paying attention, and even if you are. 

I decided to scale the 8000 pixel-wide images down to 1000 pixels. While images of that size wouldn’t produce decent exhibition prints, I thought that I could use them to develop my computer programs, and also to find out what strategies worked best with what images.

I brought about 8000 images into Lightroom, selected them all, and exported them as 1000-pixel-wide JPEGs. That was at three yesterday afternoon. About nine this morning, here’s what the Lr progress bar looked like:

lr-after-18-hours

The two operations are the export and producing the previews. Looks like it’s going to take a lot more than 24 hours to do the export.

Here’s what the Windows Task Manager says is going on:

lr-export-parallelism

It looks like Lr can only get about two threads working in parallel. You’d think that an operation like this would be disk bound, but Lr’s inability to run the individual image exports in parallel is slowing things to a crawl. I’ve noticed before that If I dis three or four exports in rapid succession, Lr would parallelize them all. Seems it should be able to figure out how to do that for one big —  OK, huge — export.

I wrote some Matlab code to do the downsizing. In spite of the fact that I wrote it as a single thread (I don’t have the Matlab Toolkit for parallel processing), the Matlab code runs considerably faster than Lr, although it looks like it will take about 5 hours to downsize 8000 files.

You may be wondering what all those Photoshop GPU Sniffer tasks are. Ever since I upgraded to Ps CC 2017, Ps has been spawning one of them every time it’s launched, and neglecting to kill it when it’s closed down. I have to go into the Task Manager and manually kill htem every so often.

 

The Last Word

← More slit scan experiments — blending space and time More slit scan experiments — clouds →

Comments

  1. Eric says

    November 17, 2016 at 8:47 am

    I use Riot, it has batch processing, but I dont know how it behaves with 8k images torture

    http://luci.criosweb.ro/riot/

    Reply
  2. Giulio Magnifico says

    November 17, 2016 at 10:30 am

    Lightroom is the most non-optimized app ever…it’s fantastic to develop and organize photos (I’m using only it on my workflow) but it’s slow slow and slow! Especially on the 5k iMac (i7 32gb ram).

    Thanks for your tests, I’ll follow your investigation 🙂

    Reply
  3. Tony A. says

    November 17, 2016 at 1:12 pm

    Maybe this will be moot because next time you might capture reduced jpg images up front, but for processing large number of files I’ve found that DxO Optics Pro uses every processor core, each one dedicated to one image at a time. It was fun to watch a 6-core AMD work it’s way through a large list.

    Reply
    • Jim says

      November 17, 2016 at 2:41 pm

      It’s not going to be moot, because I want to work with the small images only to develop the strategy to apply to the large ones.

      Thanks for the tip.

      Jim

      Reply
  4. Richard says

    November 18, 2016 at 11:40 pm

    Hi Jim,

    You will probably find ImageMagick a highly flexible and efficient solution.

    Regards,

    Richard

    Reply
  5. Richard Scobie says

    November 19, 2016 at 5:22 pm

    Jim,

    I note a number of people on the web who are having “GPU Sniffer” issues, have fixed it by re-installing the graphics card driver.

    Worth a shot if you’ve not already done it.

    Regards,

    Richard

    Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

May 2025
S M T W T F S
 123
45678910
11121314151617
18192021222324
25262728293031
« Apr    

Articles

  • About
    • Patents and papers about color
    • Who am I?
  • How to…
    • Backing up photographic images
    • How to change email providers
    • How to shoot slanted edge images for me
  • Lens screening testing
    • Equipment and Software
    • Examples
      • Bad and OK 200-600 at 600
      • Excellent 180-400 zoom
      • Fair 14-30mm zoom
      • Good 100-200 mm MF zoom
      • Good 100-400 zoom
      • Good 100mm lens on P1 P45+
      • Good 120mm MF lens
      • Good 18mm FF lens
      • Good 24-105 mm FF lens
      • Good 24-70 FF zoom
      • Good 35 mm FF lens
      • Good 35-70 MF lens
      • Good 60 mm lens on IQ3-100
      • Good 63 mm MF lens
      • Good 65 mm FF lens
      • Good 85 mm FF lens
      • Good and bad 25mm FF lenses
      • Good zoom at 24 mm
      • Marginal 18mm lens
      • Marginal 35mm FF lens
      • Mildly problematic 55 mm FF lens
      • OK 16-35mm zoom
      • OK 60mm lens on P1 P45+
      • OK Sony 600mm f/4
      • Pretty good 16-35 FF zoom
      • Pretty good 90mm FF lens
      • Problematic 400 mm FF lens
      • Tilted 20 mm f/1.8 FF lens
      • Tilted 30 mm MF lens
      • Tilted 50 mm FF lens
      • Two 15mm FF lenses
    • Found a problem – now what?
    • Goals for this test
    • Minimum target distances
      • MFT
      • APS-C
      • Full frame
      • Small medium format
    • Printable Siemens Star targets
    • Target size on sensor
      • MFT
      • APS-C
      • Full frame
      • Small medium format
    • Test instructions — postproduction
    • Test instructions — reading the images
    • Test instructions – capture
    • Theory of the test
    • What’s wrong with conventional lens screening?
  • Previsualization heresy
  • Privacy Policy
  • Recommended photographic web sites
  • Using in-camera histograms for ETTR
    • Acknowledgments
    • Why ETTR?
    • Normal in-camera histograms
    • Image processing for in-camera histograms
    • Making the in-camera histogram closely represent the raw histogram
    • Shortcuts to UniWB
    • Preparing for monitor-based UniWB
    • A one-step UniWB procedure
    • The math behind the one-step method
    • Iteration using Newton’s Method

Category List

Recent Comments

  • JimK on Goldilocks and the three flashes
  • DC Wedding Photographer on Goldilocks and the three flashes
  • Wedding Photographer in DC on The 16-Bit Fallacy: Why More Isn’t Always Better in Medium Format Cameras
  • JimK on Fujifilm GFX 100S II precision
  • Renjie Zhu on Fujifilm GFX 100S II precision
  • JimK on Fuji 20-35/4 landscape field curvature at 23mm vs 23/4 GF
  • Ivo de Man on Fuji 20-35/4 landscape field curvature at 23mm vs 23/4 GF
  • JimK on Fuji 20-35/4 landscape field curvature at 23mm vs 23/4 GF
  • JimK on Fuji 20-35/4 landscape field curvature at 23mm vs 23/4 GF
  • Ivo de Man on Fuji 20-35/4 landscape field curvature at 23mm vs 23/4 GF

Archives

Copyright © 2025 · Daily Dish Pro On Genesis Framework · WordPress · Log in

Unless otherwise noted, all images copyright Jim Kasson.