如何在PHP中打开多个套接字连接并执行回调

Question

I'm writing some code which processes a queue of items. The way it works is this:

1. Get the next item flagged as needing to be processed from the mysql database row.
2. Request some info from a google API using Curl, wait until the info is returned.
3. Do the remainder of the processing based on the info returned.
4. Flag the item as processed in the db, move onto the next item.

The problem is that on step # 2. Google sometimes takes 10-15 seconds to return the requested info, during this time my script has to remain halted and wait.

I'm wondering if I could change the code to do the following instead:

1. Get the next 5 items to be processed as usual.
2. Request info for items 1-5 from google, one after the other.
3. When the info for item 1 is returned, a 'callback' should be done which calls up a function or otherwise calls some code which then does the remainder of the processing on items 1-5.
4. And then the script starts over until all pending items in db are marked processed.

How can something like this be achieved?

Answer 1

You can split this in 2 process types.

1. Worker process (there are many of them running): knows the database row being processed, makes and waits for Google API call, and then does the job, and saves the results to the database.

2. Scheduler (one and only): periodically (say, every few seconds) checks if there's work to do, and makes sure that there are N (5 or whatever is optimal) workers running. If less then N workers are running, starts more workers (with exec ) to keep it N, until all the work is done.

Answer 2

I really don't know if this is an elegant approach, but in theory you could use fork() to fork the PHP process for each item. This would allow all code to be in one file.

// Get items from DB
$items = get_items_from_db();
foreach($items as $item) {
    $pid = pcntl_fork();
    if($pid == -1) die("Couldn't fork!");
    if(!$pid) {
      // Process the item in the child process
      process_item($item);
      exit();
    }
}
// Wait for all child processes to end
pcntl_wait();
// We're done!

But yes, this solution will most likely make some people scream ;)

Answer 3

I think you can create a master script that will call a child script on same machine for a particular item. The child script will send request to google API and work accordingly.