Jimp image to Tensor node.js

Question

I am using Tensorflow.js model. Model receives image in a Jimp format.

I need to convert Jimp bitmap to 4d Tensor.

So far I have tried this toTensor function:

function imageByteArray (image){
    
    const numChannels = 3;

    const numPixels = image.bitmap.width * image.bitmap.height;
    const values = new Int32Array(numPixels * numChannels);


    image.scan(0, 0, image.bitmap.width, image.bitmap.height, function(x, y, idx){

        values[y * image.bitmap.width * numChannels + x * numChannels + 0] = this.bitmap.data[idx + 0];
        values[y * image.bitmap.width * numChannels + x * numChannels + 1] = this.bitmap.data[idx + 1];
        values[y * image.bitmap.width * numChannels + x * numChannels + 2] = this.bitmap.data[idx + 2];

    });


    return values
}
  
function toTensor(image){
    const values = imageByteArray(image);
    // const values = image.data;
    const outShape = [1, image.bitmap.height, image.bitmap.width, 3];
    const input = tf.tensor4d(values, outShape, 'float32');

    return input.sub(127.5).div(128.0)

}

But when I compare original pre-processing (implemented during training stage) using python cv2 :

def process(image):

    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

    image = image.astype("float32")

    image = (image - 127.5) / 128.0


    return image.reshape((1, width, height, 3))

But there are some small differences in input.

Is there any correct method to convert jimp image to RGB tensor

Answer 1

tf.node can allow to decode the bitmap encoded image as already indicated in this answer

const img = fs.readFileSync("path/of/image");
const tensor = tf.node.decodeImage(img)

Answer 2

I have found a way to convert jimp image to tfnode.Tensor :

function preProcess(image){
    // const values = imageByteArray(image);
    const values = image.bitmap.data;
    const outShape = [1, image.bitmap.width, image.bitmap.height, 4];
    var input = tf.tensor4d(values, outShape, 'float32');
    
    // Slice away alpha
    input = input.slice([0, 0, 0, 0], [1, image.bitmap.width, image.bitmap.height, 3]);


    return input;

}

Jimp image usually contains alpha values as well, So I maked 4D Tensor containing alpha values as well, then sliced RGB only values.

As @edkeveked said. I can use tf.node.decodeImage functionality, but my main preprocessing (during training) was on opencv, so I needed to make sure it was as close to opencv implementation.

I also found some problems with tensorflow image functions.

So I opt to not use tensorflow image functions.