Is it possible to transfer ImageBitmap directly from JS to WebAssembly and convert it to ImageData there?

Question

Is it possible to transfer ImageBitmap directly from JS to WebAssembly without converting it to a typed array? Can ImageBitmap be some kind of "transferable" object between JS and WebAssembly as it is between the main thread and web worker?

The idea is to replace CanvasRenderingContext2D.getImageData() with something faster. Currently, I do this to apply filters on video:

1. context.drawImage(imageBitmap, 0, 0);
2. let imageData = context.getImageData();
3. loop through imageData and apply chroma key filter.

I want to try to replace this with the next (if it's possible):

1. transfer ImageBitmap to WebAssembly;
2. convert ImageBitmap to ImageData;
3. loop through ImageData and apply chrome key filter;
4. transfer changed ImageData back to JS and draw it on canvas.

I'm unfamiliar with the possibilities of WebAssembly, so maybe sorry for the silly question.

Thanks.

Answer 1

I'm not very familiar with the WebAssembly side of the force either, but I really don't think there is any way to consume an ImageBitmap object from there.

But anyway, what's slow about reading the pixel data from an ImageBitmap object is the read-back operation from the GPU to the CPU. WebAssembly won't really help here.

You may want to look at WebGL instead which IIRC has pathes to both render then read-back with minimal operations in between, that might be a bit faster than with a 2D context. Another place to look at, with a bit less support is the incoming WebGPU context. Once again I'm not very familiar with it but it's definitely worth a look at it if you wish the best perfs.

However, the best in your case might be to come back a step before, right when you generate that ImageBitmap object. You don't say where it comes from, but certainly the source that was used to generate it can give access to its pixel data already, maybe faster than going by a bitmap. For instance, if you've got a Blob representation of an image file, you could pass its inner ArrayBuffer data to your WebAssembly and do the decoding there. If you have a video, you could use the WebCodecs API to extract the YUV planes. etc.