在 JavaScript 中将编码的 JPEG 图像转换为 Uint8Array

Question

I have a python server that sends a frame of RGB data to my JS script via a websocket. The frame is encoded using simplejpeg python package. The code looks like this:

jpeg_frame = simplejpeg.encode_jpeg(image=color_frame, quality=85,colorspace='RGB', colorsubsampling='444', fastdct=True)

jpeg_frame is passed to the websocket and sent to the JS script.

On the JS side however, I would like to decompress the image and have it in the form of a Uint8Array so that I can work with the data. The image does not have to be viewed.

The data recieved is in the form of ArrayBuffer.

This is what I have so far.

socketio.on('colorFrame',(data)=>{
    var mime = 'image/jpeg';
    var a = new Uint8Array(data);
    var nb = a.length;
    if (nb < 4)
        return null;
    var binary = "";
    for (var i = 0; i < nb; i++)
        binary += String.fromCharCode(a[i]);
    var base64 = window.btoa(binary);
    var image = new Image();
    image.src = 'data:' + mime + ';base64,' + base64;
    
    var canvas = document.createElement('canvas');
    var ctx = canvas.getContext('2d');

    ctx.drawImage(image,0,0)

    image.onload = function(){
        var image_data = ctx.getImageData(0,0, 960, 540);
        console.log(image_data);
    };
});

So far I could not yet figure out how I can decompress the image. I dont mind the inaccuracy of the lossy compression, I just want the image back to its original resolution and be able to convert it to a Uint8Array.

What is the simplest way to get JPEG decoding working in a JS scrip?

Answer 1

There is an excellent article "Decoding a PNG Image in JavaScript" that answers the literal question. It's a non-trivial answer. (Note: I am not the author of the article)

However

Based on your code, it looks like you might not care about actually decoding the Image yourself. If you just want to display it, take a look at this StackOverflow question