Dynamically add Items to a QListWIdget

Question

I have this function

def search(self):
    
    #creating a list just ignore
    self.listWidget.clear()
    self.gogo.keywordSetter(self.lineEdit.text())
    self.gogo.linkSetter()
    self.gogo.resultsGetter()
    
    #iterarting through the generated lsit
    for x in self.gogo.resultsContainer:
        self.listWidget.update()
        #print(x[2])
        #x[2] is the url to an image
        url = x[2]
        print(url)
        
        req = Request(url, headers={'User-Agent': 'Mozilla/5.0'})
        webpage = urlopen(req).read()
        
        pixmap = QPixmap()
        pixmap.loadFromData(webpage)
        icon = QIcon(pixmap)
        #x[1] is a string I want to display it's a title basically
        item = QListWidgetItem(icon, x[1])
        size = QSize()
        size.setHeight(100)
        size.setWidth(400)
        item.setSizeHint(size)
        #item.iconSize(QSize(100, 400))
        self.listWidget.addItem(item)

It works, my problem is that it displays everything only after it iterates through every item. What I mean is that I can see it using the print statement that it IS going through the list and creating the items but no items is being displayed. They get all displayed at once after it completely iterates through the list. It's very slow. I know part of it is due to the image download and I can do nothing about it. But adding the items dynamically would at least make it a little more bearable. tried to use update() and it didn't really work. another weird behaviour is despite the clear() being the first instruction it doesn't clear the listWidget as soon as the function is called, it looks like it's due to the same thing that leads to everything being displayed at one.

Answer 1

UI systems use an "event loop", which is responsible of "drawing" elements and allow user interaction; that loop must be left free to do its job, and functions that operate within it must return as soon as possible to prevent "freezing" of the UI: the window is not refreshed or properly displayed, and it seems unresponsive to keyboard or mouse events.

Your function does exactly that: it blocks everything until it's finished.
Calling update() won't be enough, as it only schedules a repainting, which will only happen as soon as the main loop is able to process events. That's why long or possibly infinite for/while loops should always be avoided, as much as any blocking function like time.sleep , even for short amounts of time.

"I can do nothing about it."

Actually, not only you can, but you have to.

A possibility is to use threading, specifically QThread , which is an interface that allows to execute calls in a separate thread while providing Qt's signal/slot mechanism that can work asynchronously between thread. Using QThread and signals to communicate with the main thread is extremely important, as UI elements are not thread-safe, and must never be accessed (nor created) from external threads.

class Downloader(QThread):
    imageDownloaded = pyqtSignal(int, object)
    def __init__(self, parent, urlList):
        super().__init__(parent)
        self.urlList = urlList

    def run(self):
        for i, url in enumerate(self.urlList):
            req = Request(url, headers={'User-Agent': 'Mozilla/5.0'})
            webpage = urlopen(req).read()
            self.imageDownloaded.emit(i, webpage)


class YourWidget(QWidget):
    # ...
    def search(self):
        self.listWidget.clear()
        self.gogo.keywordSetter(self.lineEdit.text())
        self.gogo.linkSetter()
        self.gogo.resultsGetter()
        
        #iterating through the generated list
        urlList = []
        for x in self.gogo.resultsContainer:
            url = x[2]
            urlList.append(url)
            
            item = QListWidgetItem(x[1])
            item.setSizeHint(QSize(400, 100))
            self.listWidget.addItem(item)

        # the parent argument is mandatory, otherwise there won't be any
        # persistent reference to the downloader, and it will be deleted
        # as soon as this function returns; connecting the finished signal
        # to deleteLater will delete the object when it will be completed.
        downloadThread = Downloader(self, urlList)
        downloadThread.imageDownloaded.connect(self.updateImage)
        downloadThread.finished.connect(downloadThread.deleteLater)
        downloadThread.start()

    def updateImage(self, index, data):
        pixmap = QPixmap()
        if not pixmap.loadFromData(data) or index >= self.listWidget.count():
            return
        self.listWidget.item(index).setIcon(QIcon(pixmap))

^{Note: the code above is untested, as it's not clear what modules you're actually using for downloading, nor what self.gogo is.}

A slightly different alternative of the above is to use a persistent "download manager" thread, and queue requests using a python Queue , which will be read in the run() implementation.

---

Consider that Qt provides the QtNetwork module that already works asynchronously , and implements the same concept.

You have to create a QNetworkAccessManager instance (one is usually enough for the whole application), then create a QNetworkRequest for each url, and finally call the manager's get() with that request, which will return a QNetworkReply that will later be used for retrieving the downloaded data.

In the following example, I'm also setting a custom property for the reply, so that when the reply will be received, we will know to what index it corresponds to: this is even more important than what done above, as QNetworkAccessManager can download in parallel, and downloads can be completed in a different order than they were requested (for instance, if the images have different sizes, or are being downloaded from different servers).

Note that the index must be set as a Qt property, and cannot (or, better, should not) be set as a python attribute, like reply.index = i . This is because the reply we're using in python is just a wrapper around the actual Qt object, and unless we keep a persistent reference to that wrapper (for instance, by adding it to a list), that attribute would be lost.

from PyQt5.QtNetwork import *

class YourWidget(QWidget):
    def __init__(self):
        # ...
        self.downloadManager = QNetworkAccessManager()
        self.downloadManager.finished.connect(self.updateImage)

    # ...
    def search(self):
        self.listWidget.clear()
        self.gogo.keywordSetter(self.lineEdit.text())
        self.gogo.linkSetter()
        self.gogo.resultsGetter()
        
        for i, x in enumerate(self.gogo.resultsContainer):
            item = QListWidgetItem(x[1])
            item.setSizeHint(QSize(400, 100))
            self.listWidget.addItem(item)

            url = QUrl(x[2])
            request = QNetworkRequest(url)
            request.setHeader(request.UserAgentHeader, 'Mozilla/5.0')
            reply = self.downloadManager.get(request)
            reply.setProperty('index', i)

    def updateImage(self, reply):
        index = reply.property('index')
        if isinstance(index, int):
            pixmap = QPixmap()
            if pixmap.loadFromData(reply.readAll()):
                item = self.listWidget.item(index)
                if item is not None:
                    item.setIcon(QIcon(pixmap))
        reply.deleteLater()