Java URL库，用于在网站上抓行

Question

I want to be able to grab N lines (HTML text content that start on new lines) on a specific URL eg www.sitename.com and store them as strings in an array.

something like

public void grabLines(){

//create instance of class from imported library

//pass sitename into it

//from the instance, call a method for grabbing the lines on the site and pass in "N" as a parameter

//the  method returns an array/list of N Strings that I can access later

}

Is there a native Java library I can import to do this? Does it allow me do what I want easily?

Thanks

Answer 1

Are you trying to make a screen scraper? you will be pulling html as opposed to just what you see. also if the website is dynamic you won't be able to pull everything that you can see. If you want just html and stuff you can try something like this. I tried to build a bloomberg screen scraper and then parse out the random html tags.

 try {
        URL bbg = new URL("http://www.bloomberg.com/markets/economic-calendar/");
        BufferedReader r =  new BufferedReader(new InputStreamReader( bbg.openStream()));
        while( (temp = r.readLine())!= null){
            System.out.println(temp);
        }

    } catch (Exception e){
        e.printStackTrace();
    }

Answer 2

Apache HttpClient是上述URL / Reader技术之上的抽象，但是类似： Apache HTTP Client