我们如何异步同时发出多个 php http 请求？

Question

I'm currently working on a project with my friends, so let me explain:

We have a mySql database filled with english postcode from London, one table with universities, and one with hosts, what we want is to actually calculate the public transport travel time between all the host and the universities and save it into another table of the database that will have the host postcode, the university post code and the travel time between the both on one line, and etc...

For that we are doing http request to the tfl API that return to us a JSON with all the travel details (and of course the travel time), that we then decode and keep only what we want (travel time).

The problem is that we have a quite big database with almost 250 host and 800 universities that give us around 200 000 request and a too long process time to be used (with the api response time and the php treatment, around 19h)

We tried to see if we could use the cURL method to split the process between multiple loop so that we can divide the process time by the number of cURL we made but we can't manage to figure how we can do that...

The final goal is to make a small local app that when we select one university it give us the nearests 10 hosts in public transport.

Does anyone have any experience with that kind of things and can help us ?

Here is what we have right now :

//postCodeUni list contains all the universites objects
foreach ($postCodeUni as $uniPostCode) {

    //here we take the postcode from the university object
    $uni = $uniPostCode['Postcode'];

    //postCodeHost list contains all the host objects
    foreach ($postCodeHost as $hostPostCode) {

        //here we take the postcode from the host object
        $host = $hostPostCode['Postcode'];

        //here we make an http request to the tfl api that return us a journey between the two post codes (a json with all the journey details)
        $data =  json_decode(file_get_contents('https://api.tfl.gov.uk/journey/journeyresults/' . $uni . '/to/' . $host . '?app_key=a59c7dbb0d51419d8d3f9dfbf09bd5cc'), true);

        //here we save the multiple duration times (because there is different ways to travel between two point with public transport)
        $duration = $data['journeys'];

        $tableTemp = [];

        foreach ($duration as $durations) {
            $durationns = $durations['duration'];
            array_push($tableTemp, $durationns);
        }

        //We then take the shorter one
        $min = min($tableTemp);

        echo "Shorter travel time : " . $min . " of travel between " . $uni . " and ". $host . " . <br>";

        echo "<br>";

        //We then save this time in a table that will contain the travel time of all the journeys to do comparaison
        array_push($tableAllRequest, array($uni . " and ". $host => $min));
    }
}

Answer 1

There are many ways to achieve this however the easiest imo would be to use Guzzle Async (cURL multi interface under the hood). Take a look at this answer - Guzzle async requests not really async? example below,

<?php
use GuzzleHttp\Promise;
use GuzzleHttp\Client;

$client = new Client(['base_uri' => 'http://httpbin.org/']);

// Initiate each request but do not block
$promises = [
    'image' => $client->getAsync('/image'),
    'png'   => $client->getAsync('/image/png'),
    'jpeg'  => $client->getAsync('/image/jpeg'),
    'webp'  => $client->getAsync('/image/webp')
];

// Wait on all of the requests to complete. Throws a ConnectException
// if any of the requests fail
$results = Promise\unwrap($promises);

// Wait for the requests to complete, even if some of them fail
$results = Promise\settle($promises)->wait();

// Loop through each response in the results and fetch data etc
foreach($results as $promiseKey => $result) {

    // Data response
    $dataOfResponse = ($result['value']->getBody()->getContents());

    // Status
    echo $promiseKey . ':' . $result['value']->getStatusCode() . "\r\n";
}