Python：QML 布局内的 matplotlib 绘图

Question

Consider the following python3 PyQt code to display an interactive matplotlib graph with toolbar

import sys, sip
import numpy as np
from PyQt5 import QtGui, QtWidgets
from PyQt5.Qt import *

from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.backends.backend_qt5agg import NavigationToolbar2QT as NavigationToolbar
import matplotlib.pyplot as plt

app = QApplication(sys.argv)
top = QWidget()

fig = plt.figure()
ax = fig.gca()
x = np.linspace(0,5,100)
ax.plot(x,np.sin(x))
canvas = FigureCanvas(fig)
toolbar = NavigationToolbar(canvas, top)

def pick(event):
    if (event.xdata is None) or (event.ydata is None): return
    ax.plot([0,event.xdata],[0,event.ydata])
    canvas.draw()

canvas.mpl_connect('button_press_event', pick)

layout = QtWidgets.QVBoxLayout()
layout.addWidget(toolbar)
layout.addWidget(canvas)
top.setLayout(layout)
top.show()

app.exec_()

Now I'd like to achieve the same by using PyQt with QML instead. I have some experience with creating QML GUIs in C++ and I really like the fact that the layout code is nicely separated from the core logic of the code. I have found several examples on how to show plots in PyQt and on how to use Python with QML, but nothing that combines the two.

To start off, my python and QML snippets look as follows:

Python:

import sys, sip
import numpy as np
from PyQt5 import QtGui, QtWidgets
from PyQt5.Qt import *

from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.backends.backend_qt5agg import NavigationToolbar2QT as NavigationToolbar
import matplotlib.pyplot as plt

app = QApplication(sys.argv)
engine = QQmlApplicationEngine()
engine.load(QUrl('layout.qml'))
root = engine.rootObjects()[0]
root.show()
sys.exit(app.exec_())

Layout:

import QtQuick 2.7
import QtQuick.Controls 1.4

ApplicationWindow {

    visible: true
    width: 400
    height: 400

    Canvas {
        // canvas may not be the right choice here
        id: mycanvas
        anchors.fill: parent
    }

}

But I am quite lost on how to continue.

More concretely, the question would be: Is there a way to display an interactive matplotlib plot in QML (by interactive I mean not just a figure that has been saved as an image, ideally with the standard toolbar for zoom etc.)

Can anyone help? Or is the combination of QML and plots just simply discouraged ( this question suggests python and QML should work together quite well)?

Answer 1

I don't have a full solution, but if you're OK with just displaying charts and the fact that you'll have to provide any interactive controls by yourself, then there's a reasonably simple way to do that.

First of all, you will need to convert your matplotlib chart into a QImage. Fortunately doing so is surprisingly easy. The canonical backend (renderer) for matplotlib is *Agg`, and it allows you to render your Figure into a memory. Just make a suitable Canvas object for you Figure, then call .draw(). The QImage constructor will take generated data directly as inputs.

canvas = FigureCanvasAgg(figure)
canvas.draw()
    
img = QtGui.QImage(canvas.buffer_rgba(), *canvas.get_width_height(), QtGui.QImage.Format_RGBA8888).copy()

The Qt way to provide that image into QML is to use QQuickImageProvider. It will get "image name" as input from QML and should provide a suitable image as output. This allows you to serve all matplotlib charts in your app with just one Image provider. When I was working on a small visualization app for internal use, I ended up with a code like this:

import PyQt5.QtCore as QtCore
import PyQt5.QtGui as QtGui
import PyQt5.QtQuick as QtQuick
import PyQt5.QtQml as QtQml

from matplotlib.figure import Figure
from matplotlib.backends.backend_agg import FigureCanvasAgg

class MatplotlibImageProvider(QtQuick.QQuickImageProvider):
    figures = dict()

    def __init__(self):
        QtQuick.QQuickImageProvider.__init__(self, QtQml.QQmlImageProviderBase.Image)

    def addFigure(self, name, **kwargs):
        figure = Figure(**kwargs)
        self.figures[name] = figure
        return figure

    def getFigure(self, name):
        return self.figures.get(name, None)

    def requestImage(self, p_str, size):
        figure = self.getFigure(p_str)
        if figure is None:
            return QtQuick.QQuickImageProvider.requestImage(self, p_str, size)

        canvas = FigureCanvasAgg(figure)
        canvas.draw()

        w, h = canvas.get_width_height()
        img = QtGui.QImage(canvas.buffer_rgba(), w, h, QtGui.QImage.Format_RGBA8888).copy()

        return img, img.size()

Whenever I need to draw a plot in python code, I just create Figure using this addFigure to give it a name and let the Qt to know about it. Once you got Figure, rest of matplotlib drawing happens exactly as usual. Make axes and plot.

self.imageProvider = MatplotlibImageProvider()
figure = self.imageProvider.addFigure("eventStatisticsPlot", figsize=(10,10))
ax = figure.add_subplot(111)
ax.plot(x,y)

Then in QML code I can simply refer matplotlib image by name ("eventStatisticsPlot")

Image {
    source: "image://perflog/eventStatisticsPlot"
}

Note that URL is prefixed by "image://" to tell QML that we need to get image from QQuickImageProvider and includes name ("perflog") of a particular provider to use. For this stuff to work we need to register our provider during QML initialization with a call to addImageProvider. For example,

engine = QtQml.QQmlApplicationEngine()
engine.addImageProvider('perflog', qt_model.imageProvider)
engine.load(QtCore.QUrl("PerflogViewer.qml"))

At this point you should be able to see static graphs shown, but they will not be updated properly because Image component in QML assumes that image that we provide does not change. I found no good solution for it, but an ugly workaround is fairly simple. I added a signal called eventStatisticsPlotChanged to my helper class that exposes Python app data to QML and .emit() it whenever the relevant plot is changed. Eg here's a chunk of code where I get data from QML on a time interval selected by user.

@QtCore.pyqtSlot(float, float)
def selectTimeRange(self, min_time, max_time):
    self.selectedTimeRange = (min_time, max_time)
    _, ax, _ = self.eventStatisticsPlotElements
    ax.set_xlim(*self.selectedTimeRange)
    self.eventStatisticsPlotChanged.emit()

See that .emit() in the end? In QML this event forces image to reload URL like this:

Image {
    source: "image://perflog/eventStatisticsPlot"

    cache: false
    function reload() { var t = source; source = ""; source = t; }
}

Connections {
    target: myDataSourceObjectExposedFromPython
    onEventStatisticsPlotChanged: eventStatisticsPlot.reload()
}

So whenever user moves a control, following happens:

• QML sends updated time interval to my data source via selectTimeRange() call
• My code calls .set_xlim on appopriate matplotlib object and emit() a signal to notify QML that chart changed
• QML queries my imageProvider for updated chart image
• My code renders matplotlib chart into new QImage with Agg and passes it to Qt
• QML shows that image to user

It might sound a bit complicated, but its actually easy to design and use.

Here's an example of how all this looks in our small visualization app. That's pure Python + QML, with pandas used to organize data and matplotlib to show it. Scroll-like element on bottom of the screen essentially redraws chart on every event and it happens so fast that it feels real-time.

I also tried to use SVG as a way to feed vector image into QML. It's also possible and it also works. Matplotlib offers SVG backend (matplotlib.backends.backend_svg) and Image QML component support inline SVG data as a Source. The SVG data is text so it can be easily passed around between python and QML. You can update (source) field with new data and image will redraw itself automatically, you can rely on data binding. It could've worked quite well, but sadly SVG support in Qt 4 and 5 is poor. Clipping is not supported (charts will go out of the axes); resizing Image does not re-render SVG but resizes pixel image of it; changing SVG causes image to blink; performance is poor. Maybe this will change one day later, but for now stick to agg backend.

I really love design of both matlpotlib and Qt. It's smart and it meshes well without too much effort or boilerplate code.

Answer 2

It's in a fairly basic state, but https://github.com/jmitrevs/matplotlib_backend_qtquick provides a workable model to start from.

To quickly summarize the gist of the example provided with it:

• The library provides the types FigureCanvasQtQuickAgg and NavigationToolbar2QtQuick .
• The FigureCanvasQtQuickAgg type is registered with QML:

   QtQml.qmlRegisterType(FigureCanvasQtQuickAgg, "Backend", 1, 0, "FigureCanvas")

• This allows you to use it from within QML:

   FigureCanvas { id: mplView objectName : "figure" dpi_ratio: Screen.devicePixelRatio anchors.fill: parent }

  The objectName property allows the canvas instance to be found from within the Python code.
• The toolbar is made out of QML buttons,
• In the demo, they provide a DisplayBridge Python class that is linked to the canvas and is responsible for the actual plotting and for forwarding events from the various toolbar buttons

---

Internally, the FigureCanvasQtQuickAgg backend is a QQuickPaintedItem . In its paint function it copies data from the renderer attribute of the matplotlib FigureCanvasAgg base class into a QImage and the QImage is then painted. This is a fairly similar design to how the QWidget version of matplotlib works.