How Does Tensorflow.keras's ImageDataGenerator.flow_from_directory scale image values?

Question

I have a trained tensorflow model and while making the training database, I used

from tensorflow.keras.preprocessing.image import ImageDataGenerator
ImageDataGenerator.flow_from_directory(organized_dir,
                                       target_size=(image_dim, image_dim),
                                       color_mode="grayscale",
                                       batch_size=20,
                                       shuffle=True,
                                       follow_links=True)

(I'm just showing the parameters I chose to supply, the variables such as image_dim are defined elsewhere)

When I look at one of the batches using next() on the DirectoryIterator object that the function returns, the pixel values of the image seem to be scaled from their original rgb values [0,255] to grayscale [0,1]. I want it as grayscale, and my understanding is that ML models perform best with numbers between 0 and 1. So that's great!

However, now I want to use the model on different images. Opening them in cv2 , and converting to grayscale does not scale the pixel values in the same way that tensorflow does it, it just keeps the color values in [0,255] instead of [0,1]:

>>> z = cv2.imread("img.png")
>>> cv2.cvtColor(z, cv2.COLOR_BGR2GRAY)
array([[255, 255, 255, ..., 255, 255, 255],
       [255, 255, 255, ..., 255, 255, 255],
       [255, 255, 255, ..., 255, 255, 255],
       ...,
       [255, 255, 255, ..., 255, 255, 255],
       [255, 255, 255, ..., 255, 255, 255],
       [255, 255, 255, ..., 255, 255, 255]], dtype=uint8)

So, what I'am asking is how do I use the same conversion that tensorflow's dataset uses to make an image use pixel values from [0,255] to [0,1]. I found here that I could normalize the images: Normalizing images in OpenCV . But I would like to use the exact algorithm/parameters that tensorflow uses so I can maximize my real world accuracy. thanks

Answer 1

You are incorrect about the ImageDataGenerator rescaling the images pixel values. According to the documentation the generator has a parameter called rescale. For this parameter the documentation states:

rescale: rescaling factor. Defaults to None. If None or 0, no rescaling is applied,
 otherwise we multiply the data by the value provided
 (after applying all other transformations).

so to rescale the values from 0 to 1 then use

rescale=1/255

Many models used in transfer learning require the pixel values to be between -1 and +1. For that case use

rescale=1/127.5-1

It may be that the images you were reading in already had their pixel values rescaled. To test if your image is pre-scaled use

import numpy as np
import cv2
path_to_file=        #specify the full path to the file
img=cv2.imread(path_to_file,0)# read in image as grayscale
max_pixel_value=np.max(img) #  find maximum pixel value
min_pixel_value=np.min(img) # find minimum pixel value
print('max pixel value= ', max_pixel_value, '  min pixel value= ', min_pixel_value)

Answer 2

cv2.imread() reads images as np.array with data type uint8. That's why you have all the values from 0 to 255. To rescale values to [0, 1] just divide them by 255

In[1]: import cv2
  ...: import numpy as np
In[2]: img = cv2.imread('some_image.jpg')
In[3]: img
Out[3]: 
array([[[110,  89,  92],
        [110,  89,  92],
        [ 50,  29,  32],
        ...

In[4]: type(img)
Out[4]: numpy.ndarray

In[5]: img.dtype
Out[5]: dtype('uint8')

In[6]: img = (img / 255)
In[7]: np.min(img), np.max(img)
Out[7]: (0.0, 1.0)

In[8]: img
Out[8]: 
array([[[0.43137255, 0.34901961, 0.36078431],
        [0.43137255, 0.34901961, 0.36078431],
        [0.19607843, 0.11372549, 0.1254902 ],
        ...

In[9]: img.dtype
Out[9]: dtype('float64')

Here is a quote from docstring of ImageDataGenerator:

rescale: rescaling factor. Defaults to None. If None or 0, no rescaling is applied, otherwise we multiply the data by the value provided (after applying all other transformations)

So the answer for your question: just multiply your array by 1/255 and you'll get the same result as ImageDataGenerator returns