PHP script for cron job timing out

Question

so I've been writing scripts for a short while and the few that ive written seem to run routinely and without timing out. However this one is and im wondering if you guys have possible suggestions on optimizing the speed of this script.

Its job is to grab ids from an existing database and use them in an API lookup to pull item data. Problem is its like 45000 items i have to look up and im already using a multi handle via curl to pipeline the process.

I'm kind of stuck and was wondering if you guys might have some ideas to make this script run faster to avoid timeout.

NOTE: my db connection info is hidden but i am connecting just fine

<?php
$s = microtime(true); //Time request variable

//CONNECT TO THE DATABASE
$DB_NAME = 'database';
$DB_HOST = 'mysql.myhost.com';
$DB_USER = 'myusername';
$DB_PASS = 'mypass';

$con = new mysqli($DB_HOST, $DB_USER, $DB_PASS, $DB_NAME);

if (mysqli_connect_errno()) {
    printf("Connect failed: %s\n", mysqli_connect_error());
    exit();
}
//END OF DB CONNECT

//TP UTIL
function array_2d_to_1d($input_array) {
    $output_array = array();

    for ($i = 0; $i < count($input_array); $i++) {
      for ($j = 0; $j < count($input_array[$i]); $j++) {
        $output_array[] = $input_array[$i][$j];
      }
    }
    return $output_array;
}

function tableExists($con, $table) {
    $show = "SHOW TABLES LIKE '$table'";
    $result = $con->query($show) or die($con->error.__LINE__);
    if($result->num_rows == 1) return true;
    else return false;
}
//END TP UTIL

//++++++++++GET ITEM IDS++++++++++++//
$table = "shinies_primitiveitems_table_NEW";

if(tableExists($con, $table)){
    $query = "SELECT * FROM $table";
    $result = $con->query($query) or die($con->error.__LINE__);
    $index = 0;

    if($result->num_rows > 0) {
        while($row = $result->fetch_assoc()) {
            $urls[$index] = "https://api.guildwars2.com/v2/items/".stripslashes($row['ItemID']);
            //echo $urls[$i]."<br />";
            $index++;
        } //end while loop
    } //end if
}
//++++++++++END GET ITEM IDS++++++++++++//


//++++++++++MULTI CURL REQUESTS FOR API+++++++++++//
// Define the URLs
//$urls = $apiURLArray;

// Create get requests for each URL
$mh = curl_multi_init();
foreach($urls as $i => $url)
{
    //echo $url."<br />";
    $ch[$i] = curl_init($url);
    curl_setopt($ch[$i], CURLOPT_RETURNTRANSFER, 1);
    curl_multi_add_handle($mh, $ch[$i]);
}

// Start performing the request
do {
    $execReturnValue = curl_multi_exec($mh, $runningHandles);
} while ($execReturnValue == CURLM_CALL_MULTI_PERFORM);
// Loop and continue processing the request
while ($runningHandles && $execReturnValue == CURLM_OK) {
  // Wait forever for network
  $numberReady = curl_multi_select($mh);
  if ($numberReady != -1) {
    // Pull in any new data, or at least handle timeouts
    do {
      $execReturnValue = curl_multi_exec($mh, $runningHandles);
    } while ($execReturnValue == CURLM_CALL_MULTI_PERFORM);
  }
}

// Check for any errors
if ($execReturnValue != CURLM_OK) {
  trigger_error("Curl multi read error $execReturnValue\n", E_USER_WARNING);
}

// Extract the content
foreach($urls as $i => $url)
{
  // Check for errors
  $curlError = curl_error($ch[$i]);
  if($curlError == "") {
    $res[$i] = curl_multi_getcontent($ch[$i]);
  } else {
    print "Curl error on handle $i: $curlError\n";
  }
  // Remove and close the handle
  curl_multi_remove_handle($mh, $ch[$i]);
  curl_close($ch[$i]);
}
// Clean up the curl_multi handle
curl_multi_close($mh);

//var_dump(json_decode($res, true));
//echo count($res)."<br />";

//Decode data
for($i=0;$i<count($res);$i++){
    $dataArray[$i] = json_decode($res[$i], true);
    //echo "$i: ".json_decode($res[$i], true)."<br /><br />";
}
//echo count($dataArray)."<br />";
//var_dump($dataArray);


//$data = array_2d_to_1d($dataArray);

//echo count($data)."<br />";
/*
//Find attributes of each item
for($i=0;$i<count($data);$i++){
    echo $data[$i]['name']." - ".$data[$i]['icon'];
}*/

//turn dataArray into a single dimensional data array
//$data = array_2d_to_1d($dataArray);

//print_r($data);
//++++++++++END REQUEST+++++++++++//


// Print the response data - DEBUG
echo "<p>Total request time: ".round(microtime(true) - $s, 4)." seconds.";
?>

Answer 1

Is the script timing out due to max executing time limit? If so, perhaps you can set the timeout to something larger with http://php.net/manual/en/function.set-time-limit.php or something?

If it's timing out in the sense that all of your curl items stop returning, perhaps the remote end has rate limiting on requests or there's some other limit you're hitting if you're trying to launch 45,000 simultaneous TCP requests or something?