Dash running in a PyQt QThread() - most appropriate way to re-initialize with new data

Question

At the moment I have a PyQt5 app running Dash in a QThread like so:

class MapView(QObject):

    message = pyqtSignal(str)
    shutting_down = pyqtSignal(bool)

    def __init__(self, data):
        super().__init__()
        self.app = dash.Dash(__name__)
        self.data = data
    
        #manual callbacks
        self.app.callback(
            Output('hover-data', 'children'),
            Input('basic-interactions', 'hoverData'))(self.display_hover_data)

        self.app.callback(
            Output('page-content', 'children'),
            Input('url', 'pathname'))(self.shutdown)
        
#...more callbacks

    def shutdown(self, pathname):
        if pathname != '/shutdown':
            return
        print("Trying to shutdown map dash")
        func = request.environ.get('werkzeug.server.shutdown')
        if func is None:
            raise RuntimeError('Not running with the Werkzeug Server')
        func()
        self.shutting_down.emit(True)

#.....lots of stuff like graphs, the layout, a function to start the server etc

and then the threading code in my main pyqt5 app is:

def show_map(self):
        
        self.mapthread = QThread()
        self.map = map_dash.MapView(self.data) 
        self.map.moveToThread(self.mapthread)
        self.map.message.connect(self.update_output)
        self.map.shutting_down.connect(self.close_map_thread)
        self.mapthread.finished.connect(self.open_project)
        self.mapthread.started.connect(self.map.run)
        self.mapthread.start()
        self.browser.setUrl(QUrl('http://127.0.0.1:8050'))
        self.update_output("Map plotted.")

My initial problem here was that if I tried to run show_map() with new data when Dash was already running (the user goes to File -> Open when a project is already running) , I would get the fatal Qthread: destroyed while thread is still running . So I embarked along the path of shutting down the flask server, then closing the thread, then going back to the function that opens a new project. It's like this:

1. User goes to File -> Open
2. def open_project() checks mapthread.isRunning()
3. It's not running so it opens a new project, creates a new QThread and new MapView instance
4. User goes to File -> Open again
5. The check in (2) returns True so the flask server is asked to shut down
6. After the server has shut down, the shutting_down signal causes the thread to be asked to quit() (I'm not sure how robust this is because it isn't a signal from flask, just a line after I've asked it to shut down. But it seems to work for now).
7. Once the thread has finished, the thread emits finished() which calls open_project() again
8. open_project this time sees that the thread is not running and allows the user to open a new file.

4 to 8 doesn't take too long to run but it all seems a bit complicated and the Dash layout glitches a bit for whatever reason. As it stands, when the QThread finishes under any other circumstances it would call open_project (although I could probably work around that). Is there a better way to do this? Can I feed the existing map instance new data somehow? I have read the documentation on dcc.Interval but that doesn't seem a great way to do it either...

Update (as per comments below):

Now what I'm doing is passing new data to the thread self.map.data = new_data , then using a callback on url to re-draw the map and refresh the layout. Exactly like this:

    elif pathname == '/refresh':
            self.draw_map()
            self.set_layout()

But here is the problem. self.set_layout() refreshes dash with the OLD figure. I have verified that self.draw_map() is drawing a new figure. But calling /refresh a second time does cause dash to use the new figure. So is dash storing the old figure in a cache that hasn't been updated in the fraction of a second it takes to set the layout again? How can I make dash wait for the cache to be updated?

Answer 1

One way to do this is to pass the new data into the dash instance

self.map.data = new_data

but this does not refresh the layout and you can't simply call

self.map.set_layout() from the main thread. Instead you have to use the url callback like this:

def shutdown(self, pathname):
        if pathname == '/shutdown':
            print("Trying to shutdown map dash")
            func = request.environ.get('werkzeug.server.shutdown')
            if func is None:
                raise RuntimeError('Not running with the Werkzeug Server')
            func()
            self.shutting_down.emit(True)
        elif pathname == '/refresh':
            self.draw_map()
            self.set_layout()

of course it would be more appropriate to call that method something like handle_url() now.

But there is a further problem. I'm guessing due to some caching issue, dash displays the old version of the figure unless the user manually calls refresh again. The solution is to add a dcc.Interval callback that updates the figure (it calls draw_map() ). It doesn't need to be doing this every second, you can set the interval to 999999, for example. In fact you can set 'max_intervals=1' and it still works. As long as the callback is there, it will be called the first time the page refreshes and the figure will be updated.

In fact, with that in place, the draw_map() and set_layout() functions don't even need to be called in the url callback. So the code now looks like this:

def shutdown(self, pathname):
        if pathname == '/shutdown':
            print("Trying to shutdown map dash")
            func = request.environ.get('werkzeug.server.shutdown')
            if func is None:
                raise RuntimeError('Not running with the Werkzeug Server')
            func()
            self.shutting_down.emit(True)
        elif pathname == '/refresh':
            print("I'm doing nothing")

And it works. But the url callback itself IS necessary.