Bit.ly API call hitting original URL before/after creating short url

Question

I'm generating Bit.ly short url with v3 api. In original URL I have some DB action after hitting it. But when I generates the bit.ly url, it automatically hit the original URL.

Check my below code of bit.ly api call.

$url = 'http://api.bit.ly/v3/shorten?login='.BITLYAPICALLLOGIN.'&apiKey='.BITLYAPICALLAPIKEY.'&uri='.urlencode($longurl).'&format=json';

$s = curl_init();  
curl_setopt($s,CURLOPT_URL, $url);  
curl_setopt($s,CURLOPT_HEADER,false);  
curl_setopt($s,CURLOPT_RETURNTRANSFER,1); 
curl_setopt($s,CURLOPT_CONNECTTIMEOUT,2); 
$result = curl_exec($s);  
curl_close( $s );

Can we avoid automatic URL call from bit.ly?

Answer 1

Bitly does fetch the long URL of a Bitlink to retrieve page title and related information. Our requests respect the robots.txt standard ( http://www.robotstxt.org ).

If you wish to request our systems stop making these requests, you may do so by updating your robots.txt file on the appropriate domain. Note that we do cache a site's robots.txt for 24 hours, so it will take a day before the change takes effect.

The robots.txt stanza to disable this behavior is:

User-agent: bitlybot
Disallow: /

Also I'd recommend updating to V4 of our API as it is the latest and greatest: https://dev.bitly.com

Answer 2

I did not have the privilege to add/edit the robots.txt, rather I came up with the solution inside the code(written in PHP). Added the below code snippet at the top of the file just to stop bitlybot to do any operation.

if (strpos($_SERVER['HTTP_USER_AGENT'], 'bitlybot') !== false) {
    header('Location: https://bit.ly/', true, 301);
    exit();
}