PNG or JPG (not rgb) over websocket with ArrayBuffer without base64

Question

Is there a way to render a PNG image to canvas without having to encode it to base64?

Server sends a PNG in binary, client receives it in an ArrayBuffer and displays it on the canvas. Only way I could get this to work is by encoding the data to base64 - on the server side - as I need it to be fast. On the client side, I created an image obj with data:image/png;base64 tag.

I know you can create a blob and a file reader but I could not get that to work.

This is the blob version:

var blob  = new Blob([image.buffer],{type: "image/png"});
var imgReader = new FileReader();

imgReader.onload = function (e) {
  var img = new Image();
  img.onload = function (e) {
    console.log("PNG Loaded");
    ctx.drawImage(img, left, top);
    window.URL.revokeObjectURL(img.src);    
    img = null;  
  };

  img.onerror = img.onabort = function () {         
    img = null;
  };
  img.src = e.target.result;              
  imgReader = null;
}
imgReader.readAsDataURL(blob);  

image is Uint8Array. I create a blob from it. The rest is self-explanatory.

Images are correct and valid PNG images. When I send it from the server, I wrote them to a file on the server side and they render fine with an image viewer.

Answer 1

You can create a blob url with createObjectURL without having to do any base64 encoding, just pass the blob you crated to it and you will have a url you can set as img.src

  var blob  = new Blob([image],{type: "image/png"});
  var img = new Image();
  img.onload = function (e) {
    console.log("PNG Loaded");
    ctx.drawImage(img, left, top);
    window.URL.revokeObjectURL(img.src);    
    img = null;  
  };

  img.onerror = img.onabort = function () {         
    img = null;
  };
  img.src = window.URL.createObjectURL(blob); 

Answer 2

I've only seen it used in this way. If you don't want to send the base64 via the network, then, you can use the btoa to convert the binary data to a base64 on the client side.

Looking at MDN, drawImage takes a CanvasImageSource object. CanvasImageSource represents any object of type HTMLImageElement , ImageBitmap and few others.

On further searching, I found some information related to ImageBitmap , but, not enough to provide a solution.

I could have added this to the comment, but, it would have become a massive comment and lose all the clarity.