Laravel Out of memory on localhost when trying to upload very big file

Question

I am trying to upload csv file so that I can upload those data into the database. The code for the same is as bellow

public function upload(Request $request){
        set_time_limit(0);
        ini_set('MAX_EXECUTION_TIME', 36000);
        $file = Input::file('file');
        $filePath = $file->getRealPath();

       $handle = fopen($filePath, "r");
        while(!feof($handle))
        {
           << DO THE DATABASE OPERATION >>
        }
        fclose($handle);

  return redirect()->back()->with('success', 'File uploaded successfully');
}

This works fine if the file size is less lets say about 100 or 200mb. But when the file size is big like 2GB. It closes the local server. In the console, it says out of memory

My php.ini settings are :

post_max_size=10000M
upload_max_filesize=10000M

My system config:

Window machine 64 bit

Problem

It's not only failing to upload but also closing the development server ie localhost:8000

Can anyone please tell me why this is happening and how can i fix it. I did follow a couple of threads on StackOverflow like this

phpMyAdmin: Can't import huge database file, any suggestions?

Large file uploads failing php

Laravel out of memory issue?

Uploading a file larger than 2GB using PHP

But unfortunately, these solutions did not help.

Answer 1

I would check your Windows Event Viewer to try and track down why the server is crashing. That would hopefully give you some insight into what the issue is.

If you can't find the source of the crash and since you are using 7.1 and large files, try processing the file with a generator. You're loading the whole thing into memory. Even with that large value for memory_limit, I still think that loading that whole file into memory might be your issue.

http://php.net/manual/en/language.generators.php

This will stream it line by line, but you can find other examples that allow you to chunk the file.

public function read_file($file)
{
    $fp = fopen($file, 'r');

    while(($line = fgets($fp)) !== false)
        yield $line;

    fclose($fp);
}  

public function upload(Request $request){
            set_time_limit(0);
            ini_set('MAX_EXECUTION_TIME', 36000);
            $file = Input::file('file');
            $filePath = $file->getRealPath();

            foreach(read_file($filePath) as $line)
            {
               << DO THE DATABASE OPERATION >>
            }

      return redirect()->back()->with('success', 'File uploaded successfully');
    }