ARKit, metal shader for ARSCNView

Question

Trying to figure out how to solve my issue of applying shaders to my ARSCNView. Previously, when using a standard SCNView, i have successfully been able to apply a distortion shader the following way:

if let path = Bundle.main.path(forResource: "art.scnassets/distortion", ofType: "plist") {
    if let dict = NSDictionary(contentsOfFile: path)  {
        let technique = SCNTechnique(dictionary: dict as! [String : AnyObject])

        scnView.technique = technique
    }
}

Replacing SCNView with ARSCNView gives me the following error(s):

"Error: Metal renderer does not support nil vertex function name"

"Error: _executeProgram - no pipeline state"

I was thinking it's because that ARSCNView uses a different renderer than SCNView. But logging ARSCNView.renderingAPI tells me nothing about the renderer, and i can't seem to choose one when i construct my ARSCNView instance. I must be missing something obvious, because i can't seem to find a single resource when scouring for references online.

My initial idea was instead use a SCNProgram to apply the shaders. But i can't find any resources of how to apply it to an ARSCNView, or if it's even a correct/possible solution, SCNProgram seems to be reserved for materials.

Anyone able to give me any useful pointers of how to solve vertex+fragment shaders for ARSCNView?

Answer 1

SCNTechnique for ARSCNView does not work with GLSL shaders, instead Metal functions need to be provided in the technique's plist file under the keys metalVertexShader and metalFragmentShader .

To the contrary, documentation says any combination of shader should work:

You must specify both fragment and vertex shaders, and you must specify either a GLSL shader program, a pair of Metal functions, or both. If both are specified, SceneKit uses whichever shader is appropriate for the current renderer.

So it might be a mistake, but I guess the documentation is outdated. Since all ARKit running devices also run Metal, GLSL support has not been added to ARSCNViews. As iOS12 deprecates OpenGL this looks like planned.

Answer 2

I had this issue in ARKit iOS11.4 and 12 and it came down to a series of miss-spelt shaders. I hope this might help someone.