How to access the raw widget pixels rendered on the screen?

Question

I am trying to extract the pixel data from every frame, but when I try to get the pixmap it returns null. I thought that the pixmap would return the actual pixel data of what was on the screen, similar to glReadPixels but I think I am mistaken. I think it's meant to access the pixel map of the result of setPixmap .

Is there a way to access the raw pixels rendered on the screen? With the below example, "Hello World" is rendered on the screen and I want the actual pixel data of that label.

QWidget window;
window.resize(1280, 720);
window.show();
window.setWindowTitle(QApplication::translate("toplevel", "Top-Level Widget"));

QLabel *label = new QLabel(QApplication::translate("label", "Hello World"), &window);
label->move(500, 500);
label->raise();
label->show();
QPixmap pixmapVal = label->pixmap(Qt::ReturnByValue);

Answer 1

Cause

QPixmap :

The QPixmap class is an off-screen image representation that can be used as a paint device

In other words, it is a drawing canvas and, as any other canvas, if you have not drawn on it, it would be empty.

On the other hand QLabel serves as a view for the pixmap, not as its content, and when you try to access the pixmap of the label without having set one, it returns null .

Solution

There is of course a way to make a widget, the label in your case, content of the pixmap and access the pixel data. My approach to the this problem would be like this:

1. Create an empty pixmap with the size of the widget whos pixel content you want to access, eg:

    QPixmap pixmap(widget->size()); pixmap.fill(Qt::transparent);

2. Use QWidget::render to render the content of the widget onto the pixmap:

    widget->render(&pixmap);

3. Convert the pixmap to QImage and use QImage::pixel or QImage::pixelColor to access the rgb data of the pixel at (pixelX, pixelY) like this:

    pixmap.toImage().pixelColor(pixelX, pixelY);

Example

Here is an example I have prepared for you to demonstrate how the proposed solution could be implemented:

#include "MainWindow.h"
#include <QApplication>
#include <QLabel>
#include <QDebug>

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);
    QWidget w;
    auto *label = new QLabel(QObject::tr("Hello World"), &w);

    label->move(500, 500);
    label->raise();

    w.setWindowTitle(QObject::tr("Top-Level Widget"));
    w.resize(1280, 720);
    w.show();

    QPixmap pixmap(label->size());

    pixmap.fill(Qt::transparent);

    label->render(&pixmap);

    int pixelX = 10;
    int pixelY = 5;

    // Access image pixels
    qDebug() << pixmap.toImage().pixelColor(pixelX, pixelY);

    return a.exec();
}

Result

For the pixel at (10, 5) the example produces the following result:

QColor(ARGB 1, 0, 0, 0.156863)