Is it conceivable to use a mobile device's GPU to accelerate runtime image compression?

Question

I'm not necessarily looking for an immediate practical solution, but was curious as to whether the state of the art has been taken to practical limits.

I understand low level GPU programming is a black art only a few have mastered and was curious if a mobile device's GPU can be used for such general processing tasks, it's all way over my head. (I recall hearing that the export of 3D game consoles to some countries is forbidden, due to the possibility of employing them to do Supercomputer power computing)

It would be great if there was a Flash native extension that could quickly compress dynamically generated images to GPU texture map formats. I'm guessing some trickery, like using the GPU in the compression process may be needed to get it done in an acceptable time. My particular vector art project could tolerate simple color palette reduction techniques like PVRTC4 which might be? doable in AS3.

I'm working on a project that dynamically generates large background texture maps at runtime from Flash vectors, in a project using Starling (the tools that allows mortals to utilize the GPU via stage3D) It would be a big plus to be able to generate compressed textures that occupy less GPU memory, it seems that the only current solution for minimizing texture map memory size is using pre-compressed external images, like an Adobe ATF container. Although Flash may already be beyond it's projected lifespan, I can still get distribution files 10X smaller using flash vectors instead of any of the alpha channel bitmap compression methods.

Adobe had added the facility "Flash Player 11.4 and AIR 3.4 support runtime texture compression" (but it turns out to currently only work on desktop systems)

Mentioned as an aside in this article: http://help.adobe.com/en_US/as3/dev/WS5b3ccc516d4fbf351e63e3d118a9b90204-7d5b.html

To access the functionality: "Create the texture object by calling the Context3D.createTexture() method, passing flash.display3D.Context3DTextureFormat.COMPRESSED"

The subject was brought up here: stackoverflow.com/questions/18070813/how-to-implement-the-runtime-compression-of-texture

The maker of starling worked on incorporating the Adobe runtime image compression facilities, only to learn from Adobe that it doesn't presently work on mobile devices. https://github.com/PrimaryFeather/Starling-Framework/issues/153

I assume it would be a huge challenge for someone to create the code to compress either PVRTC, ETC1 or DXT5 format depending on the device, just to support the few generating bitmaps on the fly.

Here's a "real time trans-coding" C++ library that may be relevant:

code.google.com/p/crunch/

and a javascript translation: www-cs-students.stanford.edu/~eparker/files/crunch/more_info.html

Answer 1

I was hoping somebody with actual experience would answer this .. oh well, here goes.

"Low level GPU programming" isn't that much of a black art. You're writing shaders in a dialect of C. You can do bit masking and integer shifts as well as floating point math. The hardest part is wrapping your head around how a GPU does parallel processing and how the data gets fed into your shaders. The OpenGL SuperBible should be your starting point.

In my limited experience, GPU shaders are pretty good at decompressing images, eg I'v used shaders to convert from Bayer RGGB digital camera pixel form into RGB. They're not so good at compressing images, because the output from compression is a variable length stream of data, not an image. If I were trying it, I'd write a vertex shader with the uncompressed image as a texture input and the output into a transform feedback buffer. But I'm not sure transform feedback is available on many mobile devices.

Hope this helps.