Qt GUI event recording and playback

Question

I'm attempting to implement a simple, lightweight system for recording Qt GUI events and playing them back from a script. I thought this would be fairly straightforward using the magic of Qt's event system, but I'm running into a problem I don't understand.

Here's quick summary of what I'm doing:

RECORDING:

I use QApplication.instance().eventFilter() to capture all GUI events I'm interested in* and save them to a Python script, in which each step looks something like this:

obj = get_named_object('MainWindow.my_menubar')
recorded_event = QMouseEvent(2, PyQt4.QtCore.QPoint(45, 8), 1, Qt.MouseButtons(0x1), Qt.KeyboardModifiers(0x0))
post_event(obj, recorded_event)

PLAYBACK:

I simply execute the script above, in a worker ( non-GUI ) thread. (I can't use the GUI thread because I want to keep sending scripted events to the application, even if the 'main' eventloop is blocked while a modal dialog eventloop is running.)

The important stuff happens in my post_event() function, which needs to do two things:

• First, call QApplication.postEvent(obj, recorded_event)
• Wait for all events to finish processing:**
  • Post a special event to the same eventloop that obj is running in.
  • When the special event is handled:
    • Call QApplication.processEvents()
    • Set a flag that tells the playback thread it's okay to continue

After the second part is complete, my expectation is that all effects of the first part (the recorded event) have completed , since the special event was queued after the recorded event.

The whole system mostly seems to work just fine for mouse events, key events, etc. But I'm having a problem with QAction handlers when I attempt to playback events for my main QMenuBar .

No matter what I try, it seems that I can't force my playback thread to block for the completion of all QAction.triggered handlers that result from clicking on my QMenu items. As far as I can tell, QApplication.processEvents() is returning before the QAction handler is complete.

Is there something special about QMenu widgets or QAction signals that breaks the normal rules for QApplication.postEvent() and/or QApplication.processEvents() ? I need a way to block for the completion of my QMenu 's QAction handlers.

[*] Not every event is recorded. I only record spontaneous() events, and I also filter out a few other types (eg Paint events and ordinary mouse movements).

[**] This is important because the next event in the script might refer to a widget that was created by the previous event.

Answer 1

I think your problem might best be served by using QFuture and QFutureWatcher (that is, if you're using the QtConcurrent namespace for threads, and not QThreads). Basically, the Qt Event handling system does NOT necessarily handle events in the order they're posted. If you need to block until a certain action is completed, and you're doing that action in a separate thread, you can use the QFuture object returned by QtConcurrent::run() with a QFutureWatcher to block until that particular thread finishes its processing.

Something else to consider is the way you handle events. When you use QApplication.postEvent(), the event you create gets added to the receiver's event queue to be handled later. Behind the scenes, Qt can reorder and compress these events to save processor time. I suspect this is more your problem.

In your function which handles playback, consider using QCoreApplication::processEvents(), which will not return until all events have finished processing. Documentation for QCoreApplication is here.

Answer 2

QMenu widgets and QAction signals are a special case. QMenu has an exec() function, normally used for popups. I suspect (but I don't know for sure) that QMenuBar would use this mechanism when it opens a regular pull-down menu. The docs are not clear about this, but Menus act a lot like dialog boxes in that they block all other user activity - how would Qt do this except by giving menus their own event loop? I can't fill in all the blanks from the information in your post, but I don't see how your playback thread would cope with a new event loop.