Php curl missing content from page

Question

$url = "http://www.ksl.com/index.php?nid=231&sid=74268&cat=&search=&zip=&distance=&min_price=&max_price=&type=&category=&subcat=&sold=&city=&addisplay=%5BNOW-1HOURS+TO+NOW%5D&sort=1&userid=&markettype=sale&adsstate=&nocache=1&o_facetSelected=true&o_facetKey=ad+posted&o_facetVal=Last+Minute&viewSelect=list&viewNumResults=48&sort=1";
$html = file_get_contents($url);

This returns some of the page content. I think that because the page I am trying to curl uses jquery to insert the listings the curl executes before the jquery populates the page?

Any ideas on how to get the full contents of the search curl?

Answer 1

Try this:

 $url = 'http://www.ksl.com/index.php?nid=231&sid=74268&cat=&search=&zip=&distance=&min_price=&max_price=&type=&category=&subcat=&sold=&city=&addisplay=%5BNOW-1HOURS+TO+NOW%5D&sort=1&userid=&markettype=sale&adsstate=&nocache=1&o_facetSelected=true&o_facetKey=ad+posted&o_facetVal=Last+Minute&viewSelect=list&viewNumResults=48&sort=1';


/* gets the data from a URL */
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}

$returned_content = get_data($url);

echo $returned_content;

Answer 2

No they are not getting the listings via jQuery, AJAX. There are no XHR requests.

A view source shows the listings.

I just used this code to get

header('Content-Type: text/plain; charset=utf-8');
$url = 'http://www.ksl.com/index.php?nid=231&sid=74268&cat=&search=&zip=&distance=&min_price=&max_price=&type=&category=&subcat=&sold=&city=&addisplay=%5BNOW-1HOURS+TO+NOW%5D&sort=1&userid=&markettype=sale&adsstate=&nocache=1&o_facetSelected=true&o_facetKey=ad+posted&o_facetVal=Last+Minute&viewSelect=list&viewNumResults=48&sort=1';
$data = file_get_contents($url);
echo $data;

The Result

Partial edited

Black Xbox 360 320gb Hard Drive Plus controllers and G...</a>
<span style="color: #555;">Magna,&nbsp;UT
&nbsp;|&nbsp; 1 Min
<div class="adDesc">
Selling because I got it used then only used it once, works fantastically and comes with lots of games plus 4 controllers and a keyboard for the contr&nbsp;
<a class="listlink" href="?nid=218&ad=34170943&cat=&lpid=&search=&ad_cid=7">more...</a>
<div class="priceBox">
<a href="?nid=218&ad=34170943&cat=&lpid=&search=&ad_cid=7">
<span >$450<span class="priceCents">00
<!-- <div class="adDays">1 Min -->
<!--<div class="adTime">Dec 31, 1969-->
<div class="detailBox">
Great PS4 Bundle</a>
<span style="color: #555;">Spanish Fork,&nbsp;UT
&nbsp;|&nbsp; 1 Min
Have here a great PS4 bundle with 6Games 1controller and 3months of PS Network. Still has the box. Text 801-633-1659&nbsp;
<span >$400<span class="priceCents">00