Is there a quicker way to extract floats from a 32 bit tiff than LibTIFF?

Question

I am currently using this to extract floats:

TIFF* tiff = TIFFOpen(tiffs[i].c_str(), "r");
if (tiff) {
    uint32 width, height;
    tsize_t scanlength; 
    if (TIFFGetField(tiff,TIFFTAG_IMAGEWIDTH, &width) != 1) {}
    if (TIFFGetField(tiff,TIFFTAG_IMAGELENGTH, &height) != 1) {}
    fwidth = width;
    fheight = height;
    vector<float> data;
    scanlength = TIFFScanlineSize(tiff);
    float image[height][width];
    for (uint32 y = 0; y < height; y++) {
        TIFFReadScanline(tiff,image[y],y);
    }
}

This is taking over 0.02 seconds per TIFF and I need it to be much quicker. I know that other libraries can kind of handle this, but I have only found one other that can handle 32 bit tiffs, and it was CImg, which took way longer. Even if this is as simple as using system() to do a comand line thing or call a really fast script, I would love to know if there is a faster way.

Thank you!

https://www.dropbox.com/s/5zb8spaz7cma1gx/pic.tif?dl=0

This is an example tif.

Answer 1

Mmmm, not as complete an answer as I was hoping to be able to provide, but I do have some thoughts to share. Maybe they will trigger some further thoughts from me or other folks...

---

Firstly, your images are lacking TIFF tag 262 ( "Photometric Interpretation" ) which is upsetting several tools you might otherwise use. What program generated the images - as they are not strictly compliant? Can you correct/improve the program that generated the images?

I managed to set the "Photometric Interpretation" tag to "min-is-black" with:

tiffset -s 262 0 YourImage.tif

---

Once that is set, I managed to use vips (from here) - which is exceedingly fast, and memory-efficient, to convert your file to JPEG. It has Ruby and Python bindings if you prefer those languages.

So, the command-line in Terminal to convert your file to JPEG is:

vips im_vips2jpeg YourFile.tif result.jpg

I am not convinced that works correctly though, so maybe John @user894763 (the author of vips ) would take a look.

---

Another thought, using vips is that the following command will save a raw RGB file of 3 floats per pixel which you can read straight into your own program without any decoding at all:

vips rawsave YourFile.tif image.raw

-rw-r--r--   1 mark  staff  3145728 20 Jun 16:59 image.raw

You'll note that the file size (3145728) corresponds to:

512 pixels * 512 pixels * 3 RGB values * 4 bytes of float each

---

I also used ImageMagick to convert your image to JPEG, with

convert YourImage.tif result.jpg

and got this result:

---

A further thought that occurred to me was that you could pre-warm your buffer cache before running your own TIFF extract program, by running cat on each of your files to cause them to be fetched from the NFS server:

cat *.tif > /dev/null

or maybe run parallel streams of that to reduce latency.

---

Another thought was that you could pre-fetch the files to a RAM-backed filesystem so that your files can be read with minimal latency. At 186kB per file, you could get 5,000 in a 1GB RAMdisk for much faster processing:

mkdir /tmp/RAM
sudo mount -t temps -o size=1G temps /tmp/RAM

You could also put intermediate files that I suggest in my thoughts above into the RAM filesystem.