PHP Curl request not working but works fine in POSTMAN

Question

I am trying to login to MCA portal ( POST URL : http://www.mca.gov.in/mcafoportal/loginValidateUser.do )

I tried logging in with POSTMAN app on Google Chrome which works fine. However, it doesnt work either in PHP/Python. I am not able to login through PHP/Python

Here is the PHP Code :

$url="http://www.mca.gov.in/mcafoportal/loginValidateUser.do"; 

$post_fields = array();

$post_fields['userNamedenc']='hGJfsdnk`1t';
$post_fields['passwordenc']='675894242fa9c66939d9fcf4d5c39d1830f4ddb9';
$post_fields['accessCode'] = ""

$str = call_post_mca($url, $post_fields);
$str = str_replace(" ","",$str);   

$dom = new DOMDocument();
$dom->loadHTML($str);
$xpath = new DOMXPath($dom);

$input_id =  '//input[@id="login_accessCode"]/@value';
$input_val = $xpath->query($input_id)->item(0);
$input_val1 = $input_val->nodeValue;

$url="http://www.mca.gov.in/mcafoportal/loginValidateUser.do"; 

$post_fields['userNamedenc']='hGJfsdnk`1t';
$post_fields['passwordenc']='675894242fa9c66939d9fcf4d5c39d1830f4ddb9';
$post_fields['accessCode'] = $input_val1;  //New Accesscode 

function  call_post_mca($url, $params)
{   
    #$user_agent = getRandomUserAgent();
    $user_agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36";
    $str = "";
    foreach($params as $key=>$value)
    {
        $str = $str . "$key=$value" . "&";
    }
    $postData = rtrim($str, "&");

    $ch = curl_init();  

    curl_setopt($ch,CURLOPT_URL,$url);
    curl_setopt($ch,CURLOPT_RETURNTRANSFER,true);
    curl_setopt($ch,CURLOPT_HEADER, false); 
    #curl_setopt($ch, CURLOPT_CAINFO, DOC_ROOT . '/includes/cacert.pem');

    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);   


    curl_setopt($ch,CURLOPT_USERAGENT, $user_agent);
    curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1);
    curl_setopt ($ch, CURLOPT_REFERER, $url); 

    $cookie= DOC_ROOT . "/cookie.txt";
    curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie); 
    curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie); 

    $output=curl_exec($ch);

    curl_close($ch);
    return $output;

}

Any idea what is missing ?

Answer 1

The site does a redirect, so you need to add

CURLOPT_FOLLOWLOCATION => 1

to your options array. When in doubt with cURL, try

$status = curl_getinfo($curl);
echo json_encode($status, JSON_PRETTY_PRINT);

giving :

{
"url": "http:\/\/www.mca.gov.in\/mcafoportal\/loginValidateUser.do?userNamedenc=hGJfsdnk%601t&passwordenc=675894242fa9c66939d9fcf4d5c39d1830f4ddb9&accessCode=-825374456",
"content_type": "text\/plain",
"http_code": 302,
"header_size": 1560,
"request_size": 245,
"filetime": -1,
"ssl_verify_result": 0,
"redirect_count": 0,
"total_time": 1.298891,
"namelookup_time": 0.526375,
"connect_time": 0.999786,
"pretransfer_time": 0.999844,
"size_upload": 0,
"size_download": 0,
"speed_download": 0,
"speed_upload": 0,
"download_content_length": 0,
"upload_content_length": -1,
"starttransfer_time": 1.298875,
"redirect_time": 0,
"redirect_url": "http:\/\/www.mca.gov.in\/mcafoportal\/login.do",
"primary_ip": "115.114.108.120",
"certinfo": [],
"primary_port": 80,
"local_ip": "192.168.1.54",
"local_port": 62524
}

As you can see, you got a 302 redirect status, but a redirect_count was 0 . After adding the option, i get:

{
"url": "http:\/\/www.mca.gov.in\/mcafoportal\/login.do",
"content_type": "text\/html;charset=ISO-8859-1",
"http_code": 200,
"header_size": 3131,
"request_size": 376,
"filetime": -1,
"ssl_verify_result": 0,
"redirect_count": 1,
"total_time": 2.383609,
"namelookup_time": 1.7e-5,
"connect_time": 1.7e-5,
"pretransfer_time": 4.4e-5,
"size_upload": 0,
"size_download": 42380,
"speed_download": 17779,
"speed_upload": 0,
"download_content_length": 42380,
"upload_content_length": -1,
"starttransfer_time": 0.30734,
"redirect_time": 0.915858,
"redirect_url": "",
"primary_ip": "14.140.191.120",
"certinfo": [],
"primary_port": 80,
"local_ip": "192.168.1.54",
"local_port": 62642
}

EDIT url encode the request parameters , and follow redirects

 $str = urlencode("userNamedenc=hGJfsdnk%601t&passwordenc=675894242fa9c66939d9fcf4d5c39d1830f4ddb9&accessCode=-825374456");
curl_setopt_array(
    $curl , array (
    CURLOPT_URL            => "http://www.mca.gov.in/mcafoportal/loginValidateUser.do" , // <- removed parameters here
    CURLOPT_RETURNTRANSFER => true ,
    CURLOPT_ENCODING       => "" ,
    CURLOPT_FOLLOWLOCATION => 1 ,
    CURLOPT_MAXREDIRS      => 10 ,
    CURLOPT_TIMEOUT        => 30 ,
    CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_1_1 ,
    CURLOPT_CUSTOMREQUEST  => "POST" ,
    CURLOPT_POSTFIELDS     => $str,       // <- added this here
    CURLOPT_HTTPHEADER     => array (
        "cache-control: no-cache"
    ) ,
)
);

Answer 2

This is honestly one of weird sites I have seen in a long time. First thing was to know how it works. So I decided to use chrome and see what happens when we login with wrong data

Observations:

• Blanks username and passwords fields
• Generates SHA1 hashes of username and password fields and sets then in userNamedenc and respectively
• We can override username and password directly in JavaScript and login to your account just by overriding the details from console.
• There are lot of different request which generates cookies but none of them look any useful

So the approach to solve the issue was to follow below steps

• Get the login url login.do
• Fetch the form details from the response for the access code
• Submit the form to loginValidateUser.do

The form sends below parameters

Now one interesting part of the same is below post data

displayCaptcha:true
userEnteredCaptcha:strrty

If we override the displayCaptcha to false then captcha is no more needed. So a wonderful bypass

displayCaptcha: false

Next was to code all of the above in PHP, but the site seemed so weird that many of the attempts failed. So finally I realized that we need to take it a bit closer to the browser login and also I felt delays between calls are needed

<?php

    require_once("curl.php");

    $curl = new CURL();
    $default_headers = Array(
        "Accept" => "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
        "Accept-Encoding" => "deflate",
        "Accept-Language" => "en-US,en;q=0.8",
        "Cache-Control" => "no-cache",
        "Connection" => "keep-alive",
        "DNT" => "1",
        "Pragma" => "no-cache",
        "Referer" => "http://www.mca.gov.in/mcafoportal/login.do",
        "Upgrade-Insecure-Requests" => "1"
    );

    // Get the login page 
    $curl
        ->followlocation(0)
        ->cookieejar("")
        ->verbose(1)
        ->get("http://www.mca.gov.in/mcafoportal/login.do")
        ->header($default_headers)
        ->useragent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36")
        ->execute();


    // Save the postfileds and access code as we would need them later for the POST field
    $post = $curl->loadInputFieldsFromResponse()
        ->updatePostParameter(array(
            "displayCaptcha" => "false",
            "userNamedenc" => "hGJfsdnk`1t",
            "passwordenc" => "675894242fa9c66939d9fcf4d5c39d1830f4ddb9",
            "userName" => "",
            "Cert" => ""))
        ->referrer("http://www.mca.gov.in/mcafoportal/login.do")
        ->removePostParameters(
            Array("dscBasedLoginFlag", "maxresults", "fe", "query", "SelectCert", "newUserRegistration")
        );

    $postfields = $curl->getPostFields();

    var_dump($postfields);


    // Access some dummy URLs to make it look like browser
    $curl
        ->get("http://www.mca.gov.in/mcafoportal/js/global.js")->header($default_headers)->execute()->sleep(2)
        ->get("http://www.mca.gov.in/mcafoportal/js/loginValidations.js")->header($default_headers)->execute()->sleep(2)
        ->get("http://www.mca.gov.in/mcafoportal/css/layout.css")->header($default_headers)->execute()->sleep(2)
        ->get("http://www.mca.gov.in/mcafoportal/img/bullet.png")->header($default_headers)->execute()->sleep(2)
        ->get("http://www.mca.gov.in/mcafoportal/getCapchaImage.do")->header($default_headers)->execute()->sleep(2);


    // POST to the login form the postfields saved earlier
    $curl
        ->sleep(20)
        ->header($default_headers)
        ->postfield($postfields)
        ->referrer("http://www.mca.gov.in/mcafoportal/login.do")
        ->post("http://www.mca.gov.in/mcafoportal/loginValidateUser.do")
        ->execute(false)
        ->sleep(3)
        ->get("http://www.mca.gov.in/mcafoportal/login.do")
        ->header($default_headers)
        ->execute(true);

    // Get the response from last GET of login.do
    $curl->getResponseText($output);

    //Check if user name is present in the output or not
    if (stripos($output, "Kiran") > 0) {
        echo "Hurray!!!! Login succeeded";
    } else {
        echo "Login failed please retry after sometime";
    }

After running the code it works few times and few times it doesn't. My observations

• Only one login is allowed at a time. So not sure if others were using the login when I was testing
• Without delays it would fail most of the time
• There is no obvious reason when it fails to login except the site doing something on server side to block the request

The reusable curl.php I created and used for chaining methods is below

<?php

class CURL
{
    protected $ch;
    protected $postfields;

    public function getPostFields() {
        return $this->postfields;
    }

    public function newpost()
    {
        $this->postfields = array();
        return $this;
    }

    public function addPostFields($key, $value)
    {
        $this->postfields[$key]=$value;
        return $this;
    }

    public function __construct()
    {
        $ch       = curl_init();
        $this->ch = $ch;
        $this->get()->followlocation()->retuntransfer(); //->connectiontimeout(20)->timeout(10);
    }

    function url($url)
    {
        curl_setopt($this->ch, CURLOPT_URL, $url);
        return $this;
    }

    function verbose($value = true)
    {
        curl_setopt($this->ch, CURLOPT_VERBOSE, $value);
        return $this;
    }

    function post($url='')
    {
        if ($url !== '')
            $this->url($url);
        curl_setopt($this->ch, CURLOPT_POST, count($this->postfields));
        curl_setopt($this->ch, CURLOPT_POSTFIELDS, http_build_query($this->postfields));
        return $this;
    }

    function postfield($fields)
    {
        if (is_array($fields)){
            $this->postfields = $fields;
        }
        return $this;
    }

    function close()
    {
        curl_close($this->ch);
        return $this;
    }

    function cookieejar($cjar)
    {
        curl_setopt($this->ch, CURLOPT_COOKIEJAR, $cjar);
        return $this;
    }

    function cookieefile($cfile)
    {
        curl_setopt($this->ch, CURLOPT_COOKIEFILE, $cfile);
        return $this;
    }

    function followlocation($follow = 1)
    {
        curl_setopt($this->ch, CURLOPT_FOLLOWLOCATION, $follow);
        return $this;
    }

    function loadInputFieldsFromResponse($response ='')
    {
        if ($response)
            $doc = $response;
        else
            $doc = $this->lastCurlRes;


        /* @var $doc DOMDocument */
        //simplexml_load_string($data)
        $this->getResponseDoc($doc);
        $this->postfields = array();

        foreach ($doc->getElementsByTagName('input') as $elem) {
            /* @var $elem DomNode */
            $name = $elem->getAttribute('name');
//            if (!$name)
//                $name = $elem->getAttribute('id');
            if ($name)
                $this->postfields[$name] = $elem->getAttribute("value");

        }

        return $this;
    }

    function retuntransfer($transfer=1)
    {
        curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, $transfer);
        return $this;
    }

    function connectiontimeout($connectiontimeout)
    {
        curl_setopt($this->ch, CURLOPT_CONNECTTIMEOUT, $connectiontimeout);
        return $this;
    }

    function timeout($timeout)
    {
        curl_setopt($this->ch, CURLOPT_TIMEOUT, $timeout);
        return $this;
    }
    function useragent($useragent)
    {
        curl_setopt($this->ch, CURLOPT_USERAGENT, $useragent);
        return $this;
    }

    function referrer($referrer)
    {
        curl_setopt($this->ch, CURLOPT_REFERER, $referrer);
        return $this;
    }

    function getCURL()
    {
        return $this->ch;
    }

    protected $lastCurlRes;
    protected $lastCurlResInfo;

    function get($url = '')
    {
        if ($url !== '')
            $this->url($url);
        curl_setopt($this->ch, CURLOPT_POST, 0);
        curl_setopt($this->ch, CURLOPT_HTTPGET, true);
        return $this;
    }

    function sleep($seconds){
        sleep($seconds);
        return $this;
    }

    function execute($output=false)
    {
        $this->lastCurlRes = curl_exec($this->ch);

        if ($output == true)
        {
            echo "Response is \n " . $this->lastCurlRes;
            file_put_contents("out.html", $this->lastCurlRes);
        }
        $this->lastCurlResInfo = curl_getinfo($this->ch);
        $this->postfields = array();
        return $this;
    }

    function header($headers)
    {
        //curl_setopt($this->ch, CURLOPT_HEADER, true);
        curl_setopt($this->ch, CURLOPT_HTTPHEADER, $headers);
        return $this;
    }
    function getResponseText(&$text){
        $text = $this->lastCurlRes;
        return $this;
    }


    /*
     *
    * @param DOMDocument $doc
    *
    *
    */
    function getResponseDoc(&$doc){
        $doc = new DOMDocument();
        libxml_use_internal_errors(false);
        libxml_disable_entity_loader();
        @$doc->loadHTML($this->lastCurlRes);
        return $this;
    }

    function removePostParameters($keys) {
        if (!is_array($keys))
            $keys = Array($keys);

        foreach ($keys as $key){
            if (array_key_exists($key, $this->postfields))
                unset($this->postfields[$key]);
        }

        return $this;
    }

    function keepPostParameters($keys) {
        $delete = Array();
        foreach ($this->postfields as $key=>$value){
            if (!in_array($key, $keys)){
                array_push($delete, $key);
            }
        }

        foreach ($delete as $key) {
            unset($this->postfields[$key]);
        }

        return $this;
    }

    function updatePostParameter($postarray, $encoded=false)
    {
        if (is_array($postarray))
        {
            foreach ($postarray as $key => $value) {
                if (is_null($value))
                    unset($this->postfields[$key]);
                else
                    $this->postfields[$key] = $value;
            }}
        elseif (is_string($postarray))
        {
            $parr = preg_split("/&/",$postarray);
            foreach ($parr as $postvalue) {
                if (($index = strpos($postvalue, "=")) != false)
                {
                    $key = substr($postvalue, 0,$index);
                    $value = substr($postvalue, $index + 1);
                    if ($encoded)
                        $this->postfields[$key]=urldecode($value);
                    else
                        $this->postfields[$key]=$value;
                }
                else
                    $this->postfields[$postvalue] = "";
            }


        }

        return $this;
    }

    function getResponseXml(){
        //SimpleXMLElement('<INPUT/>')->asXML();
    }

    function SSLVerifyPeer($verify=false)
    {
        curl_setopt($this->ch, CURLOPT_SSL_VERIFYPEER, $verify);
        return $this;
    }
}

?>

Answer 3

@yvesleborg and @tarun-lalwani gave the right hints. You need to take care of the cookies and the redirects. But nevertheless it was not working always for me. I guess the site operator requires some timeout between the two requests.

I rewrote your code a little bit to play around with it. mycurl.php:

function my_curl_init() {
    $url="http://www.mca.gov.in/mcafoportal/loginValidateUser.do"; 
    $user_agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36";

    $ch = curl_init();  

    curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

    return $ch;
}

/*
 * first call in order to get accessCode and sessionCookie
 */
$ch = my_curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, __DIR__ . "/cookie.txt"); // else cookielist is empty

$output = curl_exec($ch);

file_put_contents(__DIR__ . '/loginValidateUser.html', $output);

// save cookie info
$cookielist = curl_getinfo($ch, CURLINFO_COOKIELIST);
//print_r($cookielist);

curl_close($ch);

// parse accessCode from output
$re = '/\<input.*name="accessCode".*value="([-0-9]+)"/';
preg_match_all($re, $output, $matches, PREG_SET_ORDER, 0);
if ($matches) {
    $accessCode = $matches[0][1];

    // debug
    echo "accessCode: $accessCode" . PHP_EOL;   


    /*
     * second call in order to login
     */ 

    $post_fields = array(
        'userNamedenc' => 'hGJfsdnk`1t',
        'passwordenc'  => '675894242fa9c66939d9fcf4d5c39d1830f4ddb9',
        'accessCode'   => $accessCode
    );

    $cookiedata = preg_split('/\s+/', $cookielist[0]);
    $session_cookie = $cookiedata[5] . '=' . $cookiedata[6];

    // debug
    echo "sessionCookie: $session_cookie" . PHP_EOL;
    file_put_contents(__DIR__ . '/cookie2.txt', $session_cookie);

    /* 
     * !!! pause !!!
     */  
    sleep(20);

    // debug
    echo "curl -v -L -X POST -b '$session_cookie;' --data 'userNamedenc=hGJfsdnk`1t&passwordenc=675894242fa9c66939d9fcf4d5c39d1830f4ddb9&accessCode=$accessCode' http://www.mca.gov.in/mcafoportal/loginValidateUser.do > loginValidateUser2.html";

    $ch = my_curl_init();  

    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $post_fields);   

    curl_setopt($ch, CURLOPT_COOKIE, $session_cookie);

    $output = curl_exec($ch);

    file_put_contents(__DIR__ . '/loginValidateUser2.html', $output);

    curl_close($ch);
}

The script issues two request to the website. The output of the first one is used to read the accessCode and to store the session cookie. Then after a little break the second one is issued using the accessCode and session information together with the login credentials.

I tested it with PHP5.6 from a terminal ( php -f mycurl.php ). The script debugs all necessary information, outputs a curl command you could use in a terminal and logs the HTML and cookie information to some files in the same folder like the script.

Running the script too often doesn't work. The login won't work. So take your time and wait some minutes between your tries. Or change your IP ;)

Hope it helps.

Answer 4

Replication of the Problem

I did the same thing in Postman as you did your screenshot but was not able to log in:

The only difference I can see is that your request had cookies, which I suspect is why you were able to log in without all the other input fields. And it seems like there are quite a number of input fields:

Using Postman

So, I used postman intercept to capture all the fields used during a login, including the captcha and access code and I was able to login:

Update 1

I've found out that once you've solved a captcha to login, after you've logged out you may login again without displayCaptcha and userEnteredCaptcha in your form data, provided that you use the same cookie as the one you've used to login successfully. You just need to get a valid accessCode from the login page.

Answer 5

it doesnt work either in PHP/Python that is (as others have already pointed out) because you were using your browser's existing cookie session, which had already solved the captcha. clear your browser cookies, get a fresh cookie session, and DO NOT SOLVE THE CAPTCHA, and Postman won't be able to log in either. Any idea what is missing ? several things, among them, several post login parameters ( browserFlag , loginType , __checkbox_dscBasedLoginFlag , and many more), also your encoding loop here is bugged $str = $str . "$key=$value" . "&"; $str = $str . "$key=$value" . "&"; , it pretty much only works as long as both keys and values only contain [a-zA-Z0-9] characters, and since your userNamedenc contains a grave accent character, your encoding loop is insufficient. a fixed loop would be

foreach($params as $key=>$value){
    $str = $str . urlencode($key)."=".urlencode($value) . "&";
}
$str=substr($str,0,-1);

, but this is exactly why we have the http_build_query function, that entire loop and the following trim can be replaced with this single line:

$str=http_build_query($params);

, also, seems you're trying to login without a pre-existing cookie session, that's not going to work. when you do a GET request to the login page, you get a cookie, and a unique captcha, the captcha answer is tied to your cookie session, and needs to be solved before you attempt to login, you also provide no code to deal with the captcha. also, when parsing the "userName" input element, it will default to "Enter Username", which is emtied with javascript and replaced with userNamedenc, you must replicate this in PHP, also, it will have an input element named "dscBasedLoginFlag", which is removed with javascript, you must also do this part in php, also it has an input element named "Cert", which has a default value, but this value is cleared with javascript, do the same in php, and an input element named "newUserRegistration", which is removed with javascript, do that,

here's what you should do: make a GET request to the login page, save the cookie session and make sure to provide it for all further requests, and parse out all the login form's elements and add them to your login request (but be careful, there is 2x form inputs, 1 belong to the search bar, only parse the children of the login form, don't mix the 2 ), and remember to clear/remove the special input tags to emulate the javascript, as described above, then make a GET request to the captcha url, make sure to provide the session cookie, solve the captcha, then make the final login request, with the captcha answer, and userNamedenc and passwordenc and all the other elements parsed out from the login page... that should work. now, solving the captcha programmatically, the captha doesn't look too hard, cracking it probably can be automated, but until someone actually does that, you can use Deathbycaptcha to do it for you, however note that it isn't a free service.

here's a fully tested, working example implementation, using my hhb_curl library (from https://github.com/divinity76/hhb_.inc.php/blob/master/hhb_.inc.php ), and the Deathbycaptcha api:

<?php
declare(strict_types = 1);
header ( "content-type: text/plain;charset=utf8" );
require_once ('hhb_.inc.php');
const DEATHBYCATPCHA_USERNAME = '?';
const DEATHBYCAPTCHA_PASSWORD = '?';
$hc = new hhb_curl ( '', true );
$hc->setopt(CURLOPT_TIMEOUT,20);// im on a really slow net atm :(
$html = $hc->exec ( 'http://www.mca.gov.in/mcafoportal/login.do' )->getResponseBody (); // cookie session etc
$domd = @DOMDocument::loadHTML ( $html );
$inputs = getDOMDocumentFormInputs ( $domd, true ) ['login'];
$params = [ ];
foreach ( $inputs as $tmp ) {
    $params [$tmp->getAttribute ( "name" )] = $tmp->getAttribute ( "value" );
}
assert ( isset ( $params ['userNamedenc'] ), 'username input not found??' );
assert ( isset ( $params ['passwordenc'] ), 'passwordenc input not found??' );
$params ['userName'] = ''; // defaults to "Enter Username", cleared with javascript
unset ( $params ['dscBasedLoginFlag'] ); // removed with javascript
$params ['Cert'] = ''; // cleared to emptystring with javascript
unset ( $params ['newUserRegistration'] ); // removed with javascript
unset ( $params ['SelectCert'] ); // removed with javascript
$params ['userNamedenc'] = 'hGJfsdnk`1t';
$params ['passwordenc'] = '675894242fa9c66939d9fcf4d5c39d1830f4ddb9';
echo 'parsed login parameters: ';
var_dump ( $params );
$captchaRaw = $hc->exec ( 'http://www.mca.gov.in/mcafoportal/getCapchaImage.do' )->getResponseBody ();
$params ['userEnteredCaptcha'] = solve_captcha2 ( $captchaRaw );
// now actually logging in.
$html = $hc->setopt_array ( array (
        CURLOPT_POST => true,
        CURLOPT_POSTFIELDS => http_build_query ( $params ) 
) )->exec ( 'http://www.mca.gov.in/mcafoportal/loginValidateUser.do' )->getResponseBody ();
var_dump ( $hc->getStdErr (), $hc->getStdOut () ); // printing debug data
$domd = @DOMDocument::loadHTML ( $html );
$xp = new DOMXPath ( $domd );
$loginErrors = $xp->query ( '//ul[@class="errorMessage"]' );
if ($loginErrors->length > 0) {
    echo 'encountered following error(s) logging in: ';
    foreach ( $loginErrors as $err ) {
        echo $err->textContent, PHP_EOL;
    }
    die ();
}
echo "logged in successfully!";
/**
 * solves the captcha manually, by doing: echo ANSWER>captcha.txt
 *
 * @param string $raw_image
 *          raw image bytes
 * @return string answer
 */
function solve_captcha2(string $raw_image): string {
    $imagepath = getcwd () . DIRECTORY_SEPARATOR . 'captcha.png';
    $answerpath = getcwd () . DIRECTORY_SEPARATOR . 'captcha.txt';
    @unlink ( $imagepath );
    @unlink ( 'captcha.txt' );
    file_put_contents ( $imagepath, $raw_image );
    echo 'the captcha is saved in ' . $imagepath . PHP_EOL;
    echo ' waiting for you to solve it by doing: echo ANSWER>' . $answerpath, PHP_EOL;
    while ( true ) {
        sleep ( 1 );
        if (file_exists ( $answerpath )) {
            $answer = trim ( file_get_contents ( $answerpath ) );
            echo 'solved: ' . $answer, PHP_EOL;
            return $answer;
        }
    }
}
function solve_captcha(string $raw_image): string {
    echo 'solving captcha, hang on, with DEATBYCAPTCHA this usually takes between 10 and 20 seconds.';
    {
        // unfortunately, CURLFile requires a filename, it wont accept a string, so make a file of it
        $tmpfileh = tmpfile ();
        fwrite ( $tmpfileh, $raw_image ); // TODO: error checking (incomplete write or whatever)
        $tmpfile = stream_get_meta_data ( $tmpfileh ) ['uri'];
    }
    $hc = new hhb_curl ( '', true );
    $hc->setopt_array ( array (
            CURLOPT_URL => 'http://api.dbcapi.me/api/captcha',
            CURLOPT_POSTFIELDS => array (
                    'username' => DEATHBYCATPCHA_USERNAME,
                    'password' => DEATHBYCAPTCHA_PASSWORD,
                    'captchafile' => new CURLFile ( $tmpfile, 'image/png', 'captcha.png' ) 
            ) 
    ) )->exec ();
    fclose ( $tmpfileh ); // when tmpfile() is fclosed(), its also implicitly deleted.
    $statusurl = $hc->getinfo ( CURLINFO_EFFECTIVE_URL ); // status url is given in a http 300x redirect, which hhb_curl auto-follows
    while ( true ) {
        // wait for captcha to be solved.
        sleep ( 10 );
        echo '.';
        $json = $hc->setopt_array ( array (
                CURLOPT_HTTPHEADER => array (
                        'Accept: application/json' 
                ),
                CURLOPT_HTTPGET => true 
        ) )->exec ()->getResponseBody ();
        $parsed = json_decode ( $json, false );
        if (! empty ( $parsed->captcha )) {
            echo 'captcha solved!: ' . $parsed->captcha, PHP_EOL;
            return $parsed->captcha;
        }
    }
}
function getDOMDocumentFormInputs(\DOMDocument $domd, bool $getOnlyFirstMatches = false): array {
    // :DOMNodeList?
    $forms = $domd->getElementsByTagName ( 'form' );
    $parsedForms = array ();
    $isDescendantOf = function (\DOMNode $decendant, \DOMNode $ele): bool {
        $parent = $decendant;
        while ( NULL !== ($parent = $parent->parentNode) ) {
            if ($parent === $ele) {
                return true;
            }
        }
        return false;
    };
    // i can't use array_merge on DOMNodeLists :(
    $merged = function () use (&$domd): array {
        $ret = array ();
        foreach ( $domd->getElementsByTagName ( "input" ) as $input ) {
            $ret [] = $input;
        }
        foreach ( $domd->getElementsByTagName ( "textarea" ) as $textarea ) {
            $ret [] = $textarea;
        }
        foreach ( $domd->getElementsByTagName ( "button" ) as $button ) {
            $ret [] = $button;
        }
        return $ret;
    };
    $merged = $merged ();
    foreach ( $forms as $form ) {
        $inputs = function () use (&$domd, &$form, &$isDescendantOf, &$merged): array {
            $ret = array ();
            foreach ( $merged as $input ) {
                // hhb_var_dump ( $input->getAttribute ( "name" ), $input->getAttribute ( "id" ) );
                if ($input->hasAttribute ( "disabled" )) {
                    // ignore disabled elements?
                    continue;
                }
                $name = $input->getAttribute ( "name" );
                if ($name === '') {
                    // echo "inputs with no name are ignored when submitted by mainstream browsers (presumably because of specs)... follow suite?", PHP_EOL;
                    continue;
                }
                if (! $isDescendantOf ( $input, $form ) && $form->getAttribute ( "id" ) !== '' && $input->getAttribute ( "form" ) !== $form->getAttribute ( "id" )) {
                    // echo "this input does not belong to this form.", PHP_EOL;
                    continue;
                }
                if (! array_key_exists ( $name, $ret )) {
                    $ret [$name] = array (
                            $input 
                    );
                } else {
                    $ret [$name] [] = $input;
                }
            }
            return $ret;
        };
        $inputs = $inputs (); // sorry about that, Eclipse gets unstable on IIFE syntax.
        $hasName = true;
        $name = $form->getAttribute ( "id" );
        if ($name === '') {
            $name = $form->getAttribute ( "name" );
            if ($name === '') {
                $hasName = false;
            }
        }
        if (! $hasName) {
            $parsedForms [] = array (
                    $inputs 
            );
        } else {
            if (! array_key_exists ( $name, $parsedForms )) {
                $parsedForms [$name] = array (
                        $inputs 
                );
            } else {
                $parsedForms [$name] [] = $tmp;
            }
        }
    }
    unset ( $form, $tmp, $hasName, $name, $i, $input );
    if ($getOnlyFirstMatches) {
        foreach ( $parsedForms as $key => $val ) {
            $parsedForms [$key] = $val [0];
        }
        unset ( $key, $val );
        foreach ( $parsedForms as $key1 => $val1 ) {
            foreach ( $val1 as $key2 => $val2 ) {
                $parsedForms [$key1] [$key2] = $val2 [0];
            }
        }
    }
    return $parsedForms;
}

example usage: in terminal, write php foo.php | tee test.html php foo.php | tee test.html , after a few seconds it will say something like:

the captcha is saved in /home/captcha.png
 waiting for you to solve it by doing: echo ANSWER>/home/captcha.txt

then look at the captcha in /home/captcha.png , solve it, and write in another terminal: echo ANSWER>/home/captcha.txt , now the script will log in, and dump the logged in html in test.html, which you can open in your browser, to confirm that it actually logged in, screenshot when i run it: https://image.prntscr.com/image/_AsB_0J6TLOFSZuvQdjyNg.png

also note that i made 2 captcha solver functions, 1 use the deathbycaptcha api, and wont work until you provide a valid and credited deathbycaptcha account on line 5 and 6, which is not free, the other 1, solve_captcha2, asks you to solve the captcha yourself, and tells you where the captcha image is saved (so you can go have a look at it), and what command line argument to write, to provide it with the answer. just replace solve_captcha with solve_captcha2 on line 28, to solve it manually, and vise-versa. the script is fully tested with solve_captcha2, but the deathbycaptcha solver is untested, as my deathbycatpcha account is empty (if you would like to make a donation so i can actually test it, send 7 dollars to paypal account divinity76@gmail.com with a link to this thread, and i will buy the cheapest deathbycaptcha credit pack and actually test it)

• disclaimer: i am not associated with deathbycaptcha in any way (except that i was a customer of theirs a couple of years back), and this post was not sponsored.

Answer 6

The simplest thing you can do, since you already have it working in POSTMAN, is to render out the PHP code in POSTMAN. Here is link about to get PHP code from POSTMAN. Then you can compare the POSTMAN example to your code.

<?php

$curl = curl_init();

curl_setopt_array($curl, array(
  CURLOPT_URL => "http://www.mca.gov.in/mcafoportal/loginValidateUser.do?userNamedenc=hGJfsdnk%601t&passwordenc=675894242fa9c66939d9fcf4d5c39d1830f4ddb9&accessCode=",
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_ENCODING => "",
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 30,
  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
  CURLOPT_CUSTOMREQUEST => "POST",
  CURLOPT_HTTPHEADER => array(
    "cache-control: no-cache",
    "postman-token: b54abdc0-17be-f38f-9aba-dbf8f007de99"
  ),
));

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
  echo "cURL Error #:" . $err;
} else {
  echo $response;
}

What is immediately popping out to me is this 'hGJfsdnk`1t'. The backward quote can be an escape character '`'. This could very well be throwing an error where error handling redirects back to the login page. POSTMAN likely has something built in to render out the escape character to 'hGJfsdnk%601t'. Thus, this works in POSTMAN, but not in your code.

Here is the status of this request:

{
"url": "http:\/\/www.mca.gov.in\/mcafoportal\/login.do",
"content_type": "text\/html;charset=ISO-8859-1",
"http_code": 200,
"header_size": 3020,
"request_size": 821,
"filetime": -1,
"ssl_verify_result": 0,
"redirect_count": 1,
"total_time": 2.920125,
"namelookup_time": 8.2e-5,
"connect_time": 8.7e-5,
"pretransfer_time": 0.000181,
"size_upload": 0,
"size_download": 42381,
"speed_download": 14513,
"speed_upload": 0,
"download_content_length": -1,
"upload_content_length": -1,
"starttransfer_time": 0.320995,
"redirect_time": 2.084554,
"redirect_url": "",
"primary_ip": "115.114.108.120",
"certinfo": [],
"primary_port": 80,
"local_ip": "192.168.1.3",
"local_port": 45086
}

Here is shows the successful login.