在没有Parallel Processing Toolkit的情况下在MATLAB中对非常大的图像集进行图像处理的并行化

Question

I have apprxoimately 2,500,000 images to process on a single computer. I currently run sequentially an input image for a single output to my function (it takes ~5 seconds to compute). This obviously takes too much time. What other methods can I employ to speed up the process? I thought of starting multiple instances of MATLAB and running each on a subset of the data, but I'm not sure if I truly achieve parallelism with this method. What are better methods of increasing the overall speed?

Answer 1

As no-one seems to be helping out, I thought I would have a little attempt and see if I can get you started with some parallelisation. I do not use Windows or Matlab, so it may need some correcting... so, if anyone knows better... feel free to contribute.

You can install GNU Parallel under Cygwin in Windows - there are plenty of tutorials and blogs describing the process if you Google them.

First, I'm guessing/hoping that the following command will process one such image from the command line, so have a little experiment and see if that will work before you move to the next step

matlab.exe -nodisplay -nosplash -nodesktop -r "run('mfile.m image.jpg');exit;"

Then, to run in parallel, you will need to generate a list of all 2,500,000 JPEGs, so that will look something like this

DIR /B /S | FINDSTR /I "*JPG$"

and you will need to feed that into GNU Parallel, something like this

DIR /B /S | FINDSTR /I "*JPG$" | parallel matlab.exe -nodisplay -nosplash -nodesktop -r "run('mfile.m {}');exit;"

Obviously test this out on a dummy directory with a copy of a few of your files so that nothing gets broken or overwritten.

As @Daniel suggests, there is overhead to starting Matlab, so it would probably be better to change your code to process all images supplied as arguments, then maybe you could pass 2-8 images to each invocation of Matlab, something like this for 4 images per Matlab job:

DIR /B /S | FINDSTR /I "*JPG$" | parallel -N 4 matlab.exe -nodisplay -nosplash -nodesktop -r "run('mfile.m {1} {2} {3} {4}');exit;"