如何使用Apache HttpClient使用代理在Java中加载URL？

Question

Hi I am trying to test the response time by using URL of a server in JAVA using Apache HttpClient

here is my code

    HttpClient client = new HttpClient();
    HttpMethod method = new HeadMethod("http://www.google.com/");
    StopWatch sWatch = new StopWatch();

    try {

        sWatch.start();
    client.executeMethod(method);

    System.out.println(sWatch.toString());

    }catch(Exception e)
    {
        e.printStackTrace();
    }

MY PROBLEM is I can access the www.google.com from my browser but When I try to execute that logic is giving This exception

    java.net.ConnectException: Connection timed out: connect
at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:157)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:280)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at com.sampl.DAP.CheckResponse.checkStatus(CheckResponse.java:31)
at com.sampl.DAP.CheckResponse.main(CheckResponse.java:44)

Answer 1

Try the following

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.InetSocketAddress;
import java.net.MalformedURLException;
import java.net.Proxy;
import java.net.URL;
import java.net.URLConnection;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * @author djc39_000
 */
public class URLBrowser {

    public static void main(String[] args) {
        try {            
            URLConnection uc =  new URL("https://www.google.com").openConnection(
                    new Proxy(Proxy.Type.HTTP, new InetSocketAddress("my.proxy.example.com", 3128))
            );
            BufferedReader in = new BufferedReader(new InputStreamReader(uc.getInputStream()));
            String line;            
            while ((line = in.readLine()) != null) {
                System.out.println(line);
            }
        } catch (MalformedURLException ex) {
            Logger.getLogger(URLBrowser.class.getName()).log(Level.SEVERE, null, ex);
        } catch (IOException ex) {
            Logger.getLogger(URLBrowser.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
}

I formated the program this way so that if you don't have a proxy you can comment out that line and everything else will still work. If you do it with making a var Proxy proxy = new Proxy(...) then you have to comment multi lines.

Answer 2

Since you are using a library. I would recommend using jsoup . The below code has been tested.

long startTime = System.nanoTime();
Response response= Jsoup.connect(location)
       .ignoreContentType(true)
       .userAgent("Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0")  
       .referrer("http://www.google.com")   
       .timeout(12000) 
       .followRedirects(true)
       .execute();
long endTime = System.nanoTime();