Creating Unique threads for unique ID in python flask threading in socketio

Question

I have a python flask application which can receive data from a json request and then process.

A sample of my code is as below: -

# Start with a basic flask app webpage.
from flask_socketio import SocketIO, emit
from flask import Flask, render_template, request, url_for, copy_current_request_context
from time import sleep
from threading import Thread, Event

__author__ = 'shark'

app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
app.config['DEBUG'] = True

# turn the flask app into a socketio app
socketio = SocketIO(app, async_mode=None, logger=True, engineio_logger=True)

thread = Thread()
thread_stop_event = Event()


@app.route('/platform-data', methods=['POST'])
def platformData():
    """
    Generate a random number every 1 second and emit to a socketio instance (broadcast)
    Ideally to be run in a separate thread?
    """
    # infinite loop of magical random numbers
    print("Receiving platform data")
    while not thread_stop_event.isSet():
        req_data = request.get_json()

        id = req_data['id']
        latitude = req_data['coordinates'][1]
        longitude = req_data['coordinates'][0]
        speed = req_data['speed']
        angle = req_data['angle']
        length = req_data['dimensions'][0]
        width = req_data['dimensions'][1]
        laneW = req_data['lane_width']
        spdLmt = req_data['speed_limit']

        # return testProcess(speed)

        #        print(id, latitude, longitude, speed, angle, length, width, laneW, spdLmt)

        def testProcess(id,speed):
            if speed > 30:
                print(id, " ", "slow down")
            else:
                print(id," ", "ok")

        testProcess(id,speed)

        # return {"speed": speed}

        # socketio.emit('speed', {'speed': speed}, namespace='/test')
        socketio.sleep(1)


@app.route('/')
def index():
    # only by sending this page first will the client be connected to the socketio instance
    return render_template('index.html')


@socketio.on('connect', namespace='/test')
def test_connect():
    # need visibility of the global thread object
    global thread
    print('Client connected')

    # Start the random number generator thread only if the thread has not been started before.
    if not thread.isAlive():
        print("Starting Thread")
        thread = socketio.start_background_task(platformData)


@socketio.on('disconnect', namespace='/test')
def test_disconnect():
    print('Client disconnected')


if __name__ == '__main__':
    socketio.run(app)

I need to create this app in a way that it creates a separate thread for requests coming to it in realtime, based on the id of each request. The requests with the same id will be directed to the thread running that particular id.

My json request is as follows: -

{
    "id" : "1"
    "speed" : 20
}

I want to create a unique thread for each unique id in testProcess() and provide output based on that id's speed. Currently, when I pass two different speeds for the same id, 2 separate threads are created. But I need to update the change in the same thread created uniquely for each unique id.

Any idea on how to do this?

Answer 1

I hope I got now your question right. You want a Process, with a unique ID and a setting (speed) with can be changed will running? Please correct me if I'm wrong.

Then I would use the multiprocessing Tool of Python.

There you can create a Process and start it. The advantage of multiprocessing is, that you can define variables with can be shared between processes. Then you have to be careful with synchronizing the access, but that is a general Problem of multiprocessing/threading.

Small Example:

from multiprocessing import Process, Value
import time

speed_var = Value('i', 0)


def testProcess(speed_var):
    while True:
        speed = speed_var.value
        if speed >= 30:
            print("It's running fast! Perfect")
        else:
            print("It's so slow, even my grandma would be faster")
        print(speed)
        time.sleep(0.05)

id_1 = Process(target=testProcess, args=(speed_var, ))
id_1.start()

speed = 0

while speed >= 40:
   speed += 3
   speed_var.value = speed
   time.sleep(0.2)
id_1.terminate()

A different approach would be to work with the queue and not with Values. Then you can just wait for an element in the queue and start Processing it. For further Information just ask or read the docs: https://docs.python.org/2/library/multiprocessing.html#shared-ctypes-objects

If you want to run multiple Processes with your own Ids, I would manage it in an own written class. There you can link the Ids to the Processes and manage there running-state as well as there IDs and description.