Importing csv timeout issue

Question

I'm trying to modify a csv import function which times out after 60 sec of importing. For every line there are images that are resized and some other code is executed.

I know the vps can handle this but in batches because I have another website on the same server that runs a different csv program but does the same thing. That program can import 8000 lines and resize images as well. The settings there are: process 10 lines and wait 3 sec, repeat.

Settings I raised:

• set_time_limit
• max_execution_time
• Browser http keep alive timeout

I have tried sleep() for every 10th line but this only makes the process import fewer lines

if( (($current_line % 10) == 0) && ($current_line != 0) )
{
  sleep(3);
}

This is how the script loops through the file

for ($current_line = 0; $line = fgetcsv($handle, MAX_LINE_SIZE, Tools::getValue('separator')); $current_line++)
{
//code here
}

Server:

• Apache
• PHP 5.3.3
• MYSQL
• Varnish cache

What can I do to make this work?

Answer 1

The first thing to try when your script times out is to run it using the php-cli . There is no execution time limit to scripts that are run through the command line.

If this doesn't solve your problem, then you know it wasn't the execution time limit.

The second thing to try is to print out regular status messages, including from memory_get_usage() so that you can eliminate memory leaks as a cause for your script crash. This may help you identify whether your script was dying on some input.

Answer 2

Try outputting something to the browser to keep the browser alive. IE times out after 1 minute if inactivity FF is 3 minutes.

<?
if( (($current_line % 10) == 0) && ($current_line != 0) )
{
  sleep(3);
  echo '. ';
}
?>

Answer 3

you can over-write the default timeout time. set_time_limit (0) ;

using sleep will make it import fewer lines, basically the script is timeing out because it is taking over 60 seconds. by adding sleep it just gets less done in 60 seconds.

If this is a critical script i'd look at moving it to another programing language that can execute this faster. if its just a one off, or not mission critical try set_time_limit(0) which makes it never time out. also try typing php scriptname into the browser to run the script in command line.