如果图像是 tif float32，如何将图块读入张量？

Question

I am trying to run a CNN where the input images have three channels (rgb) and the label (target) images are grayscale images (1 channel). The input and label images are in float32 and tif format.

I got the list of image and label tile pairs as below:

def get_train_test_lists(imdir, lbldir):
    imgs = glob.glob(imdir+"/*.tif")
    dset_list = []
    for img in imgs:
        filename_split = os.path.splitext(img) 
        filename_zero, fileext = filename_split 
        basename = os.path.basename(filename_zero) 
        dset_list.append(basename)
    
    x_filenames = []
    y_filenames = []
    for img_id in dset_list:
        x_filenames.append(os.path.join(imdir, "{}.tif".format(img_id)))
        y_filenames.append(os.path.join(lbldir, "{}.tif".format(img_id)))
    
    print("number of images: ", len(dset_list))
    return dset_list, x_filenames, y_filenames

train_list, x_train_filenames, y_train_filenames = get_train_test_lists(img_dir, label_dir)
test_list, x_test_filenames, y_test_filenames = get_train_test_lists(test_img_dir, test_label_dir)

from sklearn.model_selection import train_test_split
x_train_filenames, x_val_filenames, y_train_filenames, y_val_filenames = 
train_test_split(x_train_filenames, y_train_filenames, test_size=0.1, random_state=42)

num_train_examples = len(x_train_filenames)
num_val_examples = len(x_val_filenames)
num_test_examples = len(x_test_filenames)

In order to read the tiles into tensor, firstly I defined the image dimensions and batch size:

img_shape = (128, 128, 3)
batch_size = 2

I noticed that there is no decoder in tensorflow for tif image based on this link . tfio.experimental.image.decode_tiff can be used but it decodes to unit8 tensor .

here is a sample code for png images :

def _process_pathnames(fname, label_path):
  # We map this function onto each pathname pair  
  img_str = tf.io.read_file(fname)
  img = tf.image.decode_png(img_str, channels=3)

  label_img_str = tf.io.read_file(label_path)

  # These are png images so they return as (num_frames, h, w, c)
  label_img = tf.image.decode_png(label_img_str, channels=1)
  # The label image should have any values between 0 and 9, indicating pixel wise
  # cropt type class or background (0). We take the first channel only. 
  label_img = label_img[:, :, 0]
  label_img = tf.expand_dims(label_img, axis=-1)
  return img, label_img

Is it possible to modify this code by tf.convert_to_tensor or any other option to get float32 tensor from tif images? (I asked this question before, but I don't know how to integrate tf.convert_to_tensor with the mentioned codes)

Answer 1

You can read almost any image format and convert it to a numpy array with the Pillow image package:

from PIL import Image
import numpy as np

img = Image.open("image.tiff")
img = np.array(img)

print(img.shape, img.dtype)
# (986, 1853, 4) uint8

You can integrate this function into your code and then convert the numpy array to a tensorflow tensor as well as doing the appropriated image conversions.

---

Side note: you can simplify a lot your get_train_test_lists function using the pathlib package (which is integrated to Python3 like os but much simpler to use).

def get_train_test_lists(imdir, lbldir):
    x_filenames = list(Path(imdir).glob("*.tif"))
    y_filenames = [Path(lbldir) / f.name for f in x_filenames]
    dset_list = [f.stem for f in x_filenames]
    return dset_list, x_filenames, y_filenames

Note that x_filenames and y_filenames are now absolute paths but this shouldn't be an issue in your code.