Using Python websocket-client multi threaded

Question

Using Python websocket-client along with multiprocessing.Pool I'm able to spawn 3 instances of the websocket-client with each connected to a different URL and writing the data to 3 separate dictionaries.

My goal is to have a 4th process running that would be able to READ the data from all 3 dictionaries as they are updated.

What is the best way to share data between these processes?

I've looked at the multiprocessing examples but because a websocket.WebSocketApp takes a parameter that includes a function name to call (on_message) and it doesn't accept additional custom parameters, I'm not clear that the on_message functions would be able to be passed an instance of a Value or Pipe, etc.

Answer 1

Multiprocessesing doesn't share data, so you would have to use a database of some sort.

I use gevent and the gevent-websocket library.

https://pypi.python.org/pypi/gevent-websocket/

You have two options, you can just run as many as you want asynchronously, OR you can use the GIPC module which actually multiprocesses gevent for you with the added benefit of being able to talk to each other through read write pipes.

https://gehrcke.de/gipc/