How to load a full webpage using PHP

Question

There have been many questions like this asked however going through those suggestions have not helped. I'm using PHP's cURL to access the n-able database for our company. The n-able web interface is not an n-central server but a provided server from n-able.

I'm working on a project to integrate our web services such as Ubiquiti and N-able into a customized dashboard rather than surfing through seven different websites. Unifi has it's own API which works well. N-able has some attempts at API implementations which I have checked, tried, and failed.

Here are the projects:

github.com/N-able/API

https://github.com/Daandamhuis/N-Able-Dashboard

When trying to access the ncod51.n-able.com page using PHP and cURL, only the bottom of the page loads:

Test of website access

<?php

$url="https://ncod51.n-able.com"; 

//traverse array and prepare data for posting (key1=value1)
foreach ( $post_data as $key => $value) {
    $post_items[] = $key . '=' . $value;
}
//create the final string to be posted using implode()
$post_string = implode ('&', $post_items);
$ch = curl_init(); 
curl_setopt ($ch, CURLOPT_URL, $url.'/IndexAction.action'); 
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 
5.1; 
en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); 
curl_setopt ($ch, CURLOPT_TIMEOUT, 60); 
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1); 
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt ($ch, CURLOPT_REFERER, $url.'loginLoginAction.action'); 
    curl_setopt ($ch, CURLOPT_POSTFIELDS, $post_string); 
curl_setopt ($ch, CURLOPT_POST, 1); 
$result = curl_exec ($ch);
    if (!$result) { 
        $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE); 
        curl_close($ch); // make sure we closeany current curl sessions 
        die($http_code.' Unable to connect to server. Please come back later.'); 
} 
echo $result; 
?>

The actual application does not load. Now the goal is to pull the information off the n-able site and load it onto a php created interface we are implementing with what is currently a Wamp server.

Let me know if any additional information is needed. This is the only php file in use until I can gain access.

Any help is greatly appreciated!

NOTE: The n-able site is an https site.

Answer 1

when doing

foreach ( $post_data as $key => $value) {
    $post_items[] = $key . '=' . $value;
}

what happens when a key or a value contains the = character? let's say your $key is foo=bar , and your $value is baz=lel&what , you'll end up with foo=bar=baz=lel&what , which is an illegal urlencoding, don't expect the server to understand it. in fact, expect the server to respond with a HTTP 400 Bad Request response, because it is a bad request. the correct encoding would be foo%3Dbar=baz%26lel , and you have the same problem with spaces and < and > and & and ÆØÅ and a bunch of other characters. fix that, you can either do $post_items[] = urlencode($key) . '=' . urlencode($value); $post_items[] = urlencode($key) . '=' . urlencode($value); - or better yet, encode the whole string with http_build_query, that's why it exist. in fact, this encode code

foreach ( $post_data as $key => $value) {
    $post_items[] = urlencode($key) . '=' . urlencode($value);
}
$post_string = implode ('&', $post_items);

can be replaced with this single line:

$post_string=http_build_query($post_items);

do that. also, when debugging curl code, enable CURLOPT_VERBOSE, it prints lots of useful debugging info.

but you say that only the bottom of the page loads? post both the headers (as printed by CURLOPT_VERBOSE ) and response html of loading the page, both in curl and in firefox, there probably lies a clue in those (like a http error code in the headers? or perhaps the rest of the page is loaded by javascript in the html?)