Error on cURL with POST data

Question

I want to extract data from this URL with Php Curl:

https://www.traveloka.com/en/flight/fullsearch?ap=JKTA.DPS&dt=14-10-2017.NA&ps=1.0.0&sc=ECONOMY

But I got the following error:

"{"errorType":"SERVER_ERROR","userErrorMessage":"Unexpected server error occurred. We apologize for the inconvenience. Please try again later.","errorMessage":"Exception occurred in server."}".

How I can extract the data from that link with php curl?

Here is my code

$url = 'https://api.traveloka.com/en/v2/flight/search/oneway';              

$params = [
    'context' => [
        'tvLifetime' => 'eTq6r7InDN+j0vrg5Bujah9yFLWfBGsNGWxzjTBUa/jvVfn8fy/IF40U7OQl0vjmoqMJwuSocopqxISYLLi6YlngzuFViHSWhNHdFgs+49yydWXm5gSjBRwDBFuO0UKHd+B69Ip0Tk1qnKH+oyzW43f2GdS7QOd10yBpqoCOyOk73cVe4oyqCjYUR7X72PoHr14UQNQEUjl1NP5Mcxp+1Gw6RzKF7uV7jMRzmsYbGfGKpYLfsYtxaSx1t35KGWOO605YN9Mj2n5kP5fOD7j2KA9adtfLBtEymWXf6tEt3ug8oBVyzj5c2/pp/hboYilQnDRCih+RwhV5WX7hPTw9IsKapSNtWZ1NX8biH7UyYuhNLgcLK03OS4WNpoO+NphjOPKh09oBpUgrEJ0UqeY+1rfj98lWMAdpMO5rp2E5pvmP7HRuW6CqBwSchPLtVPQAi7ceDGYgYneH+AfodZMd5A==',
        'tvSession' => '9tCFUug+5pqBk0WdAmwAbThaxD2lAm75JaxFJenJTB2MkEWW7bwVa5FW83NZnCLnlL2TAAijDDIDfD9YbC7NhRws3r5fKxPj62n1bJ+Nck309g3Rkogk+dtxsoMRpFHbkVkEJbYuNFbd9Ckp9iEBGg==',
        'nonce' => '5eebdd23-2574-4465-afa7-cecc94b8f909'
    ],
    'clientInterface' => 'desktop',
    'data' => [
        'currency' => 'IDR',
        'destinationAirportOrArea' => 'DPS',
        'flightDate' => [
            'day' => '14',
            'month' => '10',
            'year' => '2017'
        ],
        'isReschedule' => 'false',
        'locale' => 'en_ID',
        'newResult' => 'true',
        'numSeats' => [
            'numAdults' => '1',
            'numChildren' => '0',
            'numInfants' => '0'
        ],
        'seatPublishedClass' => 'ECONOMY',
        'seqNo' => 'null',
        'sortFilter' => [
            'filterAirlines' => [],
            'filterArrive' => [],
            'filterDepart' => [],
            'filterTransit' => [],
            'selectedDeparture' => '',
            'sort' => 'null'
        ],
        'sourceAirportOrArea' => 'JKTA',
        'searchId' => 'null',
        'usePromoFinder' => 'false',
        'useDateFlow' => 'false'                    
    ],
    'fields' => []          
];  

$ch = curl_init($url);                                                                      
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);        
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");    
curl_setopt($ch, CURLOPT_POST, 1);      
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params));                                                                                         
curl_setopt($ch, CURLOPT_REFERER, "https://www.traveloka.com/en/flight/fullsearch?ap=JKTA.DPS&dt=14-10-2017.NA&ps=1.0.0&sc=ECONOMY"); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); 
curl_setopt($ch, CURLOPT_VERBOSE, true); 
curl_setopt($ch, CURLOPT_HTTPHEADER, array(                                                                          
    "Host: api.traveloka.com",          
    "Accept: application/json, text/javascript, */*; q=0.01",
    "Accept-Language: en-us,en;q=0.5",
    "X-Requested-With: XMLHttpRequest",         
    "Connection: keep-alive",
    "Pragma: no-cache",
    "Cache-Control: no-cache")                                                                       
);                

$result = curl_exec($ch);

print_r($result);

curl_close($ch);    

Actually I'm not doing screen crawling, but I want to crawl its json data. I open the "Network" Tab in browser and see the XHR section, and then I want to grab the response from that XHR section. So how to do it and what's wrong with my code?

Answer 1

What you want to do is called Web scraping, there are plenty of tools that you can use to do that. For now, go for: https://github.com/FriendsOfPHP/Goutte or https://github.com/duzun/hQuery.php .

Answer 2

As per the URL you have given using the below code, you would get nothing rather the redirection link:

Replace with the link and parameters with the correct api:

General format of using CURL is given here:

            $ch = curl_init();

           $params = urldecode('{"ap":"' . 'JKTA.DPS' . '","dt":"' . '14-10-2017.NA' . '","ps":"' . '1.0.0'. '","sc":"' . 'ECONOMY'. '"}');



            $url = 'https://www.traveloka.com/en/flight/fullsearch?JsonData=' . $params;
            curl_setopt($ch, CURLOPT_URL, $url);
            curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
            // curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0");

            curl_setopt($ch, CURLOPT_HEADER, 0);

            // Should cURL return or print out the data? (true = return, false = print)
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

            // Timeout in seconds
            curl_setopt($ch, CURLOPT_TIMEOUT, 10);

            // Download the given URL, and return output
            $output = curl_exec($ch);

            // Close the cURL resource, and free system resources
            curl_close($ch);

            if (!empty($output)) {
                $x = json_decode($output, true);
                        var_dump($x);   
                   }else{
                      return FALSE;
                   }