Message Queue in PHP

Question

I have developed a rest api which accepts some data and store it in a message queue (Redis List). Now from redis this data is pushed into MySQL database. The problem is client needs to wait till the data is written into mysql.

I want that the client should wait till the data is written into message queue (Redis List) and the function to push the data into MySQL should execute Asynchronously. How can I do it ? My entire code base is in PHP, So I would prefer it in PHP.

I have read this but havn't tried.

Distributed queue example in PHP using Redis

I have confusion that how slave.php ( mentioned in the link) will be executed. I mean When a new message arrives in the queue, how slave.php will find this.

I dont want to use cronjob for this. Instead when a new message arrives slave.php should get executed asynchronously. How to do it?

Answer 1

Yes, this is very much possible using Memqueue, Redis etc.

Using Redis, this is how one can do it:

This PHP file pushes the message into queue when it gets it:

/*
    Code logic
*/
$redis->lPush("message_queue", "message 1");

A slave.php which does a "blocking pop" using brPop :

$redis = new Redis();
$redis->pconnect();

while (true) {
    list($queue, $message) = $redis->brPop(["message_queue"], 0);
    /*
    Logic to insert $message to MySQL
    */
}

Now whenever a new message arrives, the slave.php will catch it and push it to MySQL.

Don't be confused by the while(true) - above code is not running in an infinite loop. The brPop asynchronously waits till there is a new message in the queue.

I'm using PHP 5.4 and connecting to Redis 2.6 and above works fine. Better, you can have multiple slave.php files and the load gets distributed.

For more details: http://redis.io/commands/blpop

Answer 2

Your question is far from clear on how your system currently works.

when a new message arrives slave.php should get executed asynchronously.

This statement is an oxymoron. If processing of a message occurs asynchrhnously then it is by definition not tied to the creation of the mesage.

I suspect that the problem is that you've got no daemon handling the dequeueing of messages - but explaining how to write such a program is way beyond the scope of an answer here.