how to call method in the main process from a multiprocessing.Process without getting a new PID?

Question

I'm working in a python app where the user should be able to enter a query and get a list of results from an api. Both input and results should happen in a gui.

This should happen as the user types (with some debounce in the future), so the user should be able to query for "foo" and a call to the api should be triggered, but if the user changes his mind and now wants to query "bar", he should be able to change his query whether the previous api call ended or not.

I was able to get It working synchronously, where the api call blocks the entire app until It's finished. I did It like this (simplified):

class Api:
    def get_results(self, query):
        return self.api.results(query)
    ...


class App:
    def __init__(self):
        self.gui = None
        self.api = None

    def get_results(self, query):
        self.api = Api()
        results = self.api.get_results
        self.gui.render(results)
    ...


class Gui(tk.Frame):
    def __init__(self, root, app, *args, **kwargs):
        tk.Frame.__init__(self, root, *args, **kwargs)
        self.root = root
        self.root.attributes('-type', 'dialog')
        self.app = app
        self.app.gui = self

    def render(self, results)
        # render results

In order to get It working asynchronously, I figured I should run the call in a separate thread or process and then kill It and spawn a new one every time the user changes the query in the gui. So I changed my Api class to inherit from multiprocessing.Process , initializing It with a reference to the app instance and adding a method to run a callback in the app instance that initialized It. It's a little like this:

class Api(multiprocessing.Process):
    def __init__(self, app, query, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.app = app
        self.query = query
        self.start()

    def get_results(self, query):
        return self.api.results(query)
    
    def run(self):
        results = self.get_results
        self.app.callback(results)


class App:
    def __init__(self):
        self.gui = None
        self.api = None

    def get_results(self, query):
        if self.api:
            self.api.kill()
        self.api = Api(self, query)
    
    def callback(self, results):
        self.gui.render(results)


class Gui(tk.Frame):
    def __init__(self, root, app, *args, **kwargs):
        tk.Frame.__init__(self, root, *args, **kwargs)
        self.root = root
        self.root.attributes('-type', 'dialog')
        self.app = app
        self.app.gui = self

    def render(self, results)
        # render results

If I run this code with some print statements, I can see that It does work properly, calling the callback with the correct values, but the gui isn't updated for some reason. After some debugging I verified that the PIDs of the running code change. The gui PID changes and the app PID changes as well once the callback function is called, so I believe that It's sort of working, but I don't know to approach this problem.

Judging by the time that I'm trying to solve this problem I believe that I'm overlooking some pretty simple way of achieving my goal here. Thanks in advance!

Answer 1

I believe that your problem is that the callback is happening on the other thread. That other thread knows nothing about your GUI. All changes to the GUI need to happen on the main thread.

Try having Api just returning a value. Your other code needs to wait for that value to be returned, and then update the GUI.

GUIs and asynchronous code get a bit ugly.

Answer 2

I ended up having a Queue and a Thread together with the Process, so now the thread keeps getting results from the Queue while True and calling the callback if the value has changed. Quite a journey.

Not an elegant solution, but solved my problems for now.