如何从iOS上的Grand Central Dispatch Queue异步绘制到GLKit的OpenGL ES上下文

Question

I'm trying to move lengthy OpenGL draw operations into a GCD queue so I can get other stuff done while the GPU grinds along. I would much rather prefer to do this with GCD vs. adding real threading to my app. Literally all I want to do is be able to

• Not block on a glDrawArrays() call so the rest of the UI can stay responsive when GL rendering gets very slow.
• Drop glDrawArrays() calls when we aren't finishing them anyway (don't build up a queue of frames that just grows and grows)

On Apple's website, the docs say:

GCD and NSOperationQueue objects can execute your tasks on a thread of their choosing. They may create a thread specifically for that task, or they may reuse an existing thread. But in either case, you cannot guarantee which thread executes the task. For an OpenGL ES app, that means:

• Each task must set the context before executing any OpenGL ES commands.
• Two tasks that access the same context may never execute simultaneously.
• Each task should clear the thread's context before exiting.

Sounds pretty straightforward.

For the sake of simplicity in this question, I'm starting with a new, bone-stock version of the Apple template that comes up in the "New Project" dialog for "OpenGL ES" game. When you instantiate it, compile, and run, you should see two cubes rotating on a grey field.

To that code, I have added a GCD queue. Starting with the interface section of ViewController.m :

dispatch_queue_t openGLESDrawQueue;

Then setting those up in ViewController viewDidLoad :

openGLESDrawQueue = dispatch_queue_create("GLDRAWINGQUEUE", NULL);

Finally, I make these very small changes to the drawInRect method that CADisplayLink ends up triggering:

- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect
{
void (^glDrawBlock)(void) = ^{
    [EAGLContext setCurrentContext:self.context];
    glClearColor(0.65f, 0.65f, 0.65f, 1.0f);
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    glBindVertexArrayOES(_vertexArray);

    // Render the object with GLKit
    [self.effect prepareToDraw];

    glDrawArrays(GL_TRIANGLES, 0, 36);

    // Render the object again with ES2
    glUseProgram(_program);

    glUniformMatrix4fv(uniforms[UNIFORM_MODELVIEWPROJECTION_MATRIX], 1, 0, _modelViewProjectionMatrix.m);
    glUniformMatrix3fv(uniforms[UNIFORM_NORMAL_MATRIX], 1, 0, _normalMatrix.m);

    glDrawArrays(GL_TRIANGLES, 0, 36);
};
dispatch_async(openGLESDrawQueue, glDrawBlock);
}

This does not work. The drawing goes crazy. Drawing with the same block with dispatch_sync() works fine, though.

Let's double check Apple's list:

• Each task must set the context before executing any OpenGL ES commands.
  • Ok. I'm setting the context. It's Objective-C object pointers that have a lifetime longer than the block anyway, so they should get closed over fine. Also, I can check them in the debugger and they are fine. Also, when I draw from dispatch_sync, it works. So this does not appear to be the problem.
• Two tasks that access the same context may never execute simultaneously.
  • The only code accessing the GL context once it's set up is the code in this method, which is in turn, in this block. Since this a serial queue, only one instance of this should ever be drawing at a time anyway. Further, if I add a synchronized(self.context){} block, it doesn't fix anything. Also, in other code with very slow drawing I added a semaphore to skipping adding blocks to the queue when the previous one hadn't finished and it dropped frames fine (according to the NSLog() messages it was spitting out), but it didn't fix the drawing. HOWEVER, there is the possibility that some of the GLKit code that I can't see manipulates the context in ways I don't understand from the main thread. This is my second-highest rated theory right now, despite the fact that synchronized() doesn't change the problem and OpenGL Profiler doesn't show any thread conflicts.
• Each task should clear the thread's context before exiting.
  • I'm not totally clear on what this means. The GCD thread's context? That's fine. We're not adding anything to the queue's context so there's nothing to clean up. The EAGLContext that we're drawing to? I don't know what else we could do. Certainly not actually glClear it, that will just erase everything. Also, there's some code in Sunset Lake's Molecules that renders that looks like this:

Code:

dispatch_async(openGLESContextQueue, ^{
    [EAGLContext setCurrentContext:context];        
    GLfloat currentModelViewMatrix[9];
    [self convert3DTransform:&currentCalculatedMatrix to3x3Matrix:currentModelViewMatrix];
    CATransform3D inverseMatrix = CATransform3DInvert(currentCalculatedMatrix);
    GLfloat inverseModelViewMatrix[9];
    [self convert3DTransform:&inverseMatrix to3x3Matrix:inverseModelViewMatrix];

    GLfloat currentTranslation[3];
    currentTranslation[0] = accumulatedModelTranslation[0];
    currentTranslation[1] = accumulatedModelTranslation[1];
    currentTranslation[2] = accumulatedModelTranslation[2];

    GLfloat currentScaleFactor = currentModelScaleFactor;

    [self precalculateAOLookupTextureForInverseMatrix:inverseModelViewMatrix];
    [self renderDepthTextureForModelViewMatrix:currentModelViewMatrix translation:currentTranslation scale:currentScaleFactor];
    [self renderRaytracedSceneForModelViewMatrix:currentModelViewMatrix inverseMatrix:inverseModelViewMatrix translation:currentTranslation scale:currentScaleFactor];

    const GLenum discards[]  = {GL_DEPTH_ATTACHMENT};
    glDiscardFramebufferEXT(GL_FRAMEBUFFER, 1, discards);

    [self presentRenderBuffer];

    dispatch_semaphore_signal(frameRenderingSemaphore);
});

This code works, and I don't see any additional cleanup. I can't figure out what this code is doing differently than mine. One thing that's different is it looks like literally everything that touches the GL context is being done from the same GCD dispatch queue. However, when I make my code like this, it doesn't fix anything.

The last thing that's different is that this code does not appear to use GLKit. The code above (along with the code I'm actually interested in) does use GLKit.

At this point, I have three theories about this problem: 1. I am making a conceptual error about the interaction between blocks, GCD, and OpenGL ES. 2. GLKit's GLKViewController or GLKView do some drawing to or manipulation of the EAGLContext in between calls to drawInRect . While my drawInRect blocks are being worked on, this happens, messing things up. 3. The fact that I'm relying on the - (void)glkView:(GLKView *)view drawInRect:(CGRect)rect method is ITSELF the problem. I think of this method as, "Hey, you automatically have a CADisplayLink configured, and every time it wants a frame, it'll hit this method. Do whatever the hell you want. I mean, in normal code here you just issue glDrawArrays commands. It's not like I'm passing back a framebuffer object or a CGImageRef containing what I want to end up on the screen. I'm issuing GL commands. HOWEVER, this could be wrong. Maybe you just can't defer drawing in this method in anyway without causing problems. To test this theory, I moved all the draw code into a method called drawStuff and then replaced the body of the drawRect method with:

[NSTimer scheduledTimerWithTimeInterval:10 target:self selector:@selector(drawStuff) userInfo:nil repeats:NO];

The app comes up, displays the color the view is glClear 'd to for ten seconds, and then draws like normal. So that theory doesn't look too strong either.

There is a similar question posted here that has one answer, which is upvoted and accepted:

The code in the dispatch block isn't going to work. By the time it gets executed, all of the OpenGL state for that frame will have long since been destroyed. If you were to put a call to glGetError() in that block, I'm sure it would tell you the same. You need to make sure that all your drawing code is done in that glkView method for the OpenGL state to be valid. When you do that dispatch, you're essentially shunting the execution of that drawing code out of the scope of that method.

I don't see why this should be true. But:

1. I'm only closing over references to things in the block that are going to outlive the block, and they're things like Objective-C pointers from the enclosing object scope.
2. I can check them in the debugger, they look fine.
3. I inserted a getGLError() call after every GL operation and it never returns anything but zero.
4. Drawing from a block with dispatch_sync works.
5. I tried a think where in the drawInRect method, I save the block to an ivar and then set a NSTimer to call drawStuff . In drawStuff I just invoke the block. Draws fine.

The NSTimer case draws asynchronously, but it does not involve drawing from another thread since AFAIK NSTimer invocations just get scheduled on the setting thread's runloop. So it has to do with threads.

Can anyone clue me in on what I'm missing here?

Answer 1

This is not working because as borrrden says, GLKit is calling presentRenderbuffer: immediately after - (void)glkView:(GLKView *)view drawInRect:(CGRect)rect completes.

This is working in your case of using a timer, because the drawStuff method is being called on the main thread at the beginning of a draw cycle. Your - (void)glkView:(GLKView *)view drawInRect:(CGRect)rect effectively does nothing but schedule this to happen on the main thread again in another 10 seconds, and the previously scheduled drawing call is then rendered at the end of the drawInRect: method. This is doing nothing except delaying the drawing by 10 seconds, everything is still occurring on the main thread.

If you want to go the route of rendering off the main thread, GLKit is not going to be a good match. You are going to need to set up your own thread with a runloop, hook up a CADisplayLink to this runloop, and then render from this queue. GLKViewController is configured to use the main runloop for this, and will always present the render buffer at the end of each frame, which will cause havoc with whatever you are doing on a different thread.

Depending on your GL needs you may find it simpler doing all the GL stuff on the main thread, and doing the "other stuff" off the main thread.