MTLTexture vs CGImageRef

Question

What is the main difference between MTLTexture vs CGImageRef ? When do we need to use MTLTexture instead of CGImageRef (and vice versa)?

I have an app (say a video game) that draw everything by itself on a dedicated surface. this includes animation at 60fps (so I need to redraw the surface every 16ms). I don't know the most efficient way to do my app using Metal

Answer 1

First of all, MTLTexture comes from a low-level graphics API. MTLTexture refers to an "image" that resides in memory accessible to GPU (no necessarily on GPU itself). You can then write a program that uses Metal, specifically render ( MTLRenderPipelineState ) or compute ( MTLComputePipelineState ) pipeline states that contain shader (programs that run on GPU) to read textures, sample them, write to them and use them as attachments (output rendering results to them). Textures can also be copied to buffers ( MTLBuffer ) and other textures, if you want to read back texture data on the CPU. But MTLTexture is mostly intended to be used by GPU rather than CPU. Also, MTLTexture is not limited to being 2D, it can also be a cube texture or even a 3D texture.

CGImage , on the other hand, comes from a higher-level API (Core Graphics or Quartz 2D) that is intended for 2D use. You don't need shaders or GPU pipeines to create or modify CGImage s and there are many functions to work with these images "out of the box".

I would say, if you have a 3D video game, you can check out Metal, but it's a low level API, and setting up Metal is a much more involved process than setting up OpenGL, for example. You can't use Core Graphics for 3D games as-is. If Metal seems too hard, you can check out higher-level APIs from Apple, such as SceneKit, which are also intended for game development.

I can't say much about 2D game development, but you can definitely use Metal for it, it might just be a bit "overkill".

In conclusion, you need to find a balance between complexity and control and chose what best suits you.