获取System.Net.WebException：远程服务器返回错误：（403）Forbidden。在Google URL Shortener API Call中

Question

I am using below code to shorten long urls

public static string UrlShorten(string url)
{
    string post = "{\"longUrl\": \"" + url + "\"}";
    string shortUrl = url;
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.googleapis.com/urlshortener/v1/url?key=" + ReadConfig("GoogleUrlShortnerApiKey"));
    try
    {
        request.ServicePoint.Expect100Continue = false;
        request.Method = "POST";
        request.ContentLength = post.Length;
        request.ContentType = "application/json";
        request.Headers.Add("Cache-Control", "no-cache");

        using (Stream requestStream = request.GetRequestStream())
        {
            byte[] postBuffer = Encoding.ASCII.GetBytes(post);
            requestStream.Write(postBuffer, 0, postBuffer.Length);
        }

        using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
        {
            using (Stream responseStream = response.GetResponseStream())
            {
                using (StreamReader responseReader = new StreamReader(responseStream))
                {
                    string json = responseReader.ReadToEnd();
                    shortUrl = Regex.Match(json, @"""id"": ?""(?<id>.+)""").Groups["id"].Value;
                }
            }
        }
    }
    catch (Exception ex)
    {
        // if Google's URL Shortner is down...
        Utility.LogSave("UrlShorten", "Google's URL Shortner is down", url, ex.ToString());
        //System.Diagnostics.Debug.WriteLine(ex.Message);
        //System.Diagnostics.Debug.WriteLine(ex.StackTrace);
    }
    return shortUrl;
}

I have created a scheduler to shorten large number of urls. And most of the time of got below exception

System.Net.WebException: The remote server returned an error: (403) Forbidden. at System.Net.HttpWebRequest.GetResponse()

I was thinking that due to Courtesy Limit i was getting this exception so increased Per-User Limit by 100,000.0 requests/second/user but still I'm getting the same exception.

I don't understand why its happening even i am making hardly 2000 request to the server at a time.

Please advise.

Answer 1

Looking at your question I presume it's a rate-limit exceeded error. You can retrieve the error response if you modify your code like this:

try
{
  ........
}
catch (WebException exception)
{
   string responseText;

   using(var reader = new StreamReader(exception.Response.GetResponseStream()))
   {
     responseText = reader.ReadToEnd();
   }
}
catch (Exception ex)
{
  ......
}

If it's a rate limit exceeded error you will find something like this in responseText :

{
 "error": {
  "errors": [
   {
    "domain": "usageLimits",
    "reason": "rateLimitExceeded",
    "message": "Rate Limit Exceeded"
   }
  ],
  "code": 403,
  "message": "Rate Limit Exceeded"
 }
}

Answer 2

This error occuring because your posting the data to webservice very fastly so google detecting it as robot.(if you post data manually this frequently it will ask to enter captcha code ).

so to resolve this problem you need to increase time interval between each post request.