PHP + cURL发布未返回预期结果

Question

I am trying to post to a site using PHP and cURL and get a search result as part of a scraping proof of concept. But the response is not the same as the one that I get by accessing the site directly. I have captured all the post parameters and the headers using FireBug and am sending all of them using cURL, but still the response is not the same. Could someone please help out in figuring out what could I be doing wrong?

The site I am trying to access is: http://book.goindigo.in/skylights/cgi-bin/skylights.cgi

In that site I am doing a oneway search which will return a bunch of flights (for example, Bombay-Delhi for September 25). I am not worried about this result. Once the flights are displayed, if a particular flight is picked, it displays the fare breakup on the left side under the Price Summary section. To get this price summary they are firing an ajax call, which is what I want to simulate using PHP and cURL.

When accessed through the site, the ajax call returns a bunch of javascript variables as response. But the same call returns the complete html of the page and NOT the expected javascript variables when accessed through cURL.

Any help would be appreciated.

Thanks

$url='https://book.goindigo.in/skylights/cgi-bin/skylights.cgi';

$header = array("Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",     "Accept-Language: en-us,en;q=0.5", "Connection: keep-alive", "Cache-Control: no-cache", "Accept-Encoding: gzip,deflate", "Content-Length: 342", "Content-Type: application/x-www-form-urlencoded; charset=UTF-8", "Cookie: __utma=1.649498667.1346160326.1346395277.1346415046.4; __utmz=1.1346160326.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmb=1.11.10.1346415046; __utmc=1", "Host: book.goindigo.in", "Pragma: no-cache", "Referer: https://book.goindigo.in/skylights/cgi-bin/skylights.cgi");

$user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0) Gecko/20100101 Firefox/15.0";

$referrer = 'https://book.goindigo.in/skylights/cgi-bin/skylights.cgi' ;

$post_params="page=INCLUSIVE_PRICING&module=SB&sid=&adults=1&children=0&language=EN&infants=0&numberMarkets=1&market1Data=201209036E%20146BLRDEL%20ININERE0BLRDEL201209031345201209031630Regular%20Fare%20ADULT%20ADT%2001&market2Data=&flt1_cnx1_date=&flt1_cnx2_date=&flt1_paxType1_fare=440000&flt1_paxType2_fare=&itin_type=Domestic";

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, $header);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
#curl_setopt($ch, CURLOPT_PROXY, $proxy);
#curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_REFERER, $referer);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);

$result = curl_exec($ch);
$error = curl_error($ch);

curl_close($ch);

    echo $result;

Answer 1

You're missing:

curl_setopt($ch, CURLOPT_POSTFIELDS, $post_params);

Here's what I did from the shell:

curl -d "page=INCLUSIVE_PRICING&module=SB&sid=&adults=1&children=0&language=EN&infants=0&numberMarkets=2&market1Data=201209116E   176BOMDEL                            ININERE0BOMDEL201209111325201209111530Regular Fare            ADULT ADT 01&market2Data=201209116E   175DELBOM                            ININNRN0BOMDEL201209111345201209111555RoundTrip Fare          ADULT ADT 01&flt1_cnx1_date=&flt1_cnx2_date=&flt2_cnx1_date=&flt2_cnx2_date=&flt1_paxType1_fare=443000&flt1_paxType2_fare=&flt2_paxType1_fare=402000&flt2_paxType2_fare=&itin_type=Domestic" http://book.goindigo.in/skylights/cgi-bin/skylights.cgi

The output was:

<!--
//////////////////////////////////////////////////////////////////////////////
//    Copyright(c) 2000 - 2006 Navitaire Inc.  All rights reserved.         //
//                                                                          //
//    This source code is protected by copyright law and international      //
//    treaties.  Unauthorized reproduction, distribution or alteration      //
//    of this source code, or any portion of it, may result in severe       //
//    civil and criminal penalties and will be prosecuted to the maximum    //
//    extent possible under the law.                                        //
//                                                                          //
//  Skylights 9.4b-6E                                    www.navitaire.com  //
//////////////////////////////////////////////////////////////////////////////
-->

numRoutes=2;
        r1destination=DEL;
        r1origin=BOM;

        r1fareLabel=Regular Fare            ;


            r1Pax1Type=ADT;

                                                r1Pax1DiscountBase= Rs 4,430.00 ;
                                                r1Pax1TotalFareCost= Rs 8,426.00 ;


                                                                                r1Pax1Tax1Amount= Rs 233.00 INR;


                            r1Pax1Tax1SummedAmount=233.00;


                                                 r1Pax1Tax1Desciption=Passenger Service Fee;







                                                                                r1Pax1Tax2Amount= Rs 2,800.00 INR;


                            r1Pax1Tax2SummedAmount=2800.00;


                                                 r1Pax1Tax2Desciption=Fuel Surcharge;







                                                                                r1Pax1Tax3Amount= Rs 50.00 INR;


                            r1Pax1Tax3SummedAmount=50.00;


                                                 r1Pax1Tax3Desciption=Transaction Charge;







                                                                                r1Pax1Tax4Amount= Rs 360.00 INR;


                            r1Pax1Tax4SummedAmount=360.00;


                                                 r1Pax1Tax4Desciption=Service Tax;








                                                                                r1Pax1Tax6Amount= Rs 553.00 INR;


                            r1Pax1Tax6SummedAmount=553.00;


                                                 r1Pax1Tax6Desciption=User Development Fee;






                                                r1Pax1TotalTaxAmount= Rs 3,996.00 ;



                           r1Pax1SummedFareAmount=4430.00;



                                                r1Pax1SummedFareAmount=4430.00;
                                                r1Pax1SummedTaxAmount= Rs 3,996.00 ;



        r1InfDescription=;
                                r1InfDiscountBase= Rs 0.00 ;
        r1InfSSRCode=;
                                r1InfTotalFareCost= Rs 0.00 ;





                                r1TotalInfTaxAmount= Rs 0.00 ;
                                r1RouteTotalTax= Rs 3,996.00 ;
                                r1RouteTotalAmount= Rs 8,426.00 ;
                                r1InfSummedFareAmount= Rs 0.00 ;
                                r1InfSummedTaxAmount= Rs 0.00;
                                r1SummedTaxAmount= Rs 3,996.00 ;
                                r1SummedAmount= Rs 8,426.00 ;

        r2destination=BOM;
        r2origin=DEL;

        r2fareLabel=RoundTrip Fare          ;


            r2Pax1Type=ADT;

                                                r2Pax1DiscountBase= Rs 4,020.00 ;
                                                r2Pax1TotalFareCost= Rs 8,101.00 ;


                                                                                r2Pax1Tax1Amount= Rs 146.00 INR;


                            r2Pax1Tax1SummedAmount=146.00;


                                                 r2Pax1Tax1Desciption=Passenger Service Fee;







                                                                                r2Pax1Tax2Amount= Rs 2,800.00 INR;


                            r2Pax1Tax2SummedAmount=2800.00;


                                                 r2Pax1Tax2Desciption=Fuel Surcharge;







                                                                                r2Pax1Tax3Amount= Rs 50.00 INR;


                            r2Pax1Tax3SummedAmount=50.00;


                                                 r2Pax1Tax3Desciption=Transaction Charge;







                                                                                r2Pax1Tax4Amount= Rs 340.00 INR;


                            r2Pax1Tax4SummedAmount=340.00;


                                                 r2Pax1Tax4Desciption=Service Tax;








                                                                                r2Pax1Tax6Amount= Rs 745.00 INR;


                            r2Pax1Tax6SummedAmount=745.00;


                                                 r2Pax1Tax6Desciption=User Development Fee;






                                                r2Pax1TotalTaxAmount= Rs 4,081.00 ;



                           r2Pax1SummedFareAmount=4020.00;



                                                r2Pax1SummedFareAmount=4020.00;
                                                r2Pax1SummedTaxAmount= Rs 4,081.00 ;



        r2InfDescription=;
                                r2InfDiscountBase= Rs 0.00 ;
        r2InfSSRCode=;
                                r2InfTotalFareCost= Rs 0.00 ;





                                r2TotalInfTaxAmount= Rs 0.00 ;
                                r2RouteTotalTax= Rs 4,081.00 ;
                                r2RouteTotalAmount= Rs 8,101.00 ;
                                r2InfSummedFareAmount= Rs 0.00 ;
                                r2InfSummedTaxAmount= Rs 0.00;
                                r2SummedTaxAmount= Rs 4,081.00 ;
                                r2SummedAmount= Rs 8,101.00 ;

                TransactionFeeAmount=0.00; 
                TotalPackageAmount=Rs 16,527.00 ;
                PrefixType=Rs ;