http get returns 403

Question

I'm trying to get some instant answer from the DuckDuckGo API, but I keep getting error 403. here is my code so far:

    static JSONObject getData(String term){
    JSONParser parser = new JSONParser();
    JSONObject json = null;
    term = term.replaceAll(" ", "%20");

    try {
        URL ob=new URL("http://api.duckduckgo.com/?q=" + term + "&format=json&pretty=0");
        HttpURLConnection con=(HttpURLConnection) ob.openConnection();
        con.setRequestMethod("GET");
        con.setRequestProperty("USER_AGENT", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.95 Safari/537.11");

        BufferedReader br = new BufferedReader((new InputStreamReader(con.getInputStream())));
        json = (JSONObject) parser.parse(br.readLine());
    } catch (Exception ex) {
        ex.printStackTrace();
    }
    return json;
}

I can't seem to get it to work. I've read I need to identofy as a normal browser, but I don't see what I'm doing wrong. Any help with this?

Answer 1

I got the 403 (same as you) with HttpURLConnection. But, it worked when I used URLConnection instead of HttpURLConnection. Because of quick availability, I just used org.json.simple parser. The json object printed the parsed data at the end.

import java.io.*;
import java.net.*;
import java.util.*;

import org.json.simple.*;
import org.json.simple.parser.*;

public class HelloWorld{

  public static void main(String []args){

     try {
         JSONParser parser = new JSONParser();
         JSONObject json = null;
         URL url = new URL("http://api.duckduckgo.com/?q=science&format=json&pretty=0");
         URLConnection conn = url.openConnection();
         conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB;     rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)");

         Scanner scn = new Scanner(conn.getInputStream()).useDelimiter("\\A");
         String buffer = scn.hasNext() ? scn.next() : "";
         json = (JSONObject) parser.parse(buffer);
         System.out.println(json.toString());

     } catch (Exception e) {
         e.printStackTrace();
     }
  }
}