Issue downloading file with PHP

Question

I am trying to download a file using PHP and cURL. I obtain the url to download the file through a series of cURL requests.

When I try to download the file I get an empty file. See below the relevant piece of code:

...
$output = curl_exec($ch); 
curl_close($ch);
$regex = '/\b(https?|ftp|file):\/\/resources\.lendingclub\.com\/secure[-A-Z0-9+&@#\/%?=~_|$!:,.;]*[A-Z0-9+&@#\/%=~_|$]/i';
preg_match_all($regex, $output, $parts);
$url3a = $parts[0][0];
OutputMsg($url3a); //print the content of $url3a
DownloadFile($url3a, 'testfile.zip'); //downloads the file using a cURL request
...

This creates the file testfile.zip with ZERO bytes. And outputs the following to the screen:

https://resources.lendingclub.com/secure/LoanStats3a_securev1.csv.zip?signature=cmu73mJsyNhznZMBH6B%2FsFjoNuE%3D&issued=1459663950631

If I add a line (see below)

...
$output = curl_exec($ch); 
curl_close($ch);
$regex = '/\b(https?|ftp|file):\/\/resources\.lendingclub\.com\/secure[-A-Z0-9+&@#\/%?=~_|$!:,.;]*[A-Z0-9+&@#\/%=~_|$]/i';
preg_match_all($regex, $output, $parts);
$url3a = $parts[0][0];
//line added below
$url3a = 'https://resources.lendingclub.com/secure/LoanStats3a_securev1.csv.zip?signature=cmu73mJsyNhznZMBH6B%2FsFjoNuE%3D&issued=1459663950631';
OutputMsg($url3a); //print the content of $url3a
DownloadFile($url3a, 'testfile.zip'); //downloads the file using a cURL request
...

the file is downloaded correctly and the screen outputs the same as before:

https://resources.lendingclub.com/secure/LoanStats3a_securev1.csv.zip?signature=cmu73mJsyNhznZMBH6B%2FsFjoNuE%3D&issued=1459663950631

I am at a loss why the first example above does not work and the second does. The only thing I did was to type the url and assign it to the variable.

I can provide more of the code, including the DownloadFile function.

I haven't figured out a way to make it work without me having to type in the url in the source code.

Thanks.

Answer 1

I finally figured out the issue. I am posting the solution here for future reference.

In my first try, I was getting the url for the file I wanted to download by scrapping the html page returned by a cURL request.

In my second try, I typed the url directly in the php source code.

When I echoed both to the screen they looked exactly the same.

However after playing with strlen and strpos I realized that the first url had the "&" stored as "&amp". Both looked exactly the same on my screen, but they were stored differently.

To solve the issue I replaced the line:

$url3a = $parts[0][0];

with

$url3a = htmlspecialchars_decode($parts[0][0]);

This htmlspecialchars_decode function replaces all the "&amp" with regular "&".

I hope this helps someone else.

Thanks.