I have been using following function for fetching and parsing URLs.
public static void getPage(String url_string, String page)
{
try
{
URL url = new URL(url_string);
System.out.println(url.getPort() + " " + url.getDefaultPort());
URLConnection conn = url.openConnection();
BufferedReader br =
new BufferedReader(new InputStreamReader(conn.getInputStream()));
BufferedWriter bw = new BufferedWriter(new FileWriter(page));
String line = "";
while((line = br.readLine()) != null)
{
bw.write(line + "\n");
}
bw.close();
br.close();
System.out.println("Page fetched in "+page);
}
catch(Exception e)
{
System.out.println("\nError while fetching the page - ");
e.printStackTrace();
}
}
I call it as -
getPage("http://google.com", "tmp.html");
I am able to fetch any type of URLs but not able to fetch this particular URL.
http://www.toysrus.com/storefrontsearch/stores.jsp?skuId=13112916&quantity=1&postalCode=79414&productId=13066123&searchRadius=10000
Though we can see this page on Firefox and Chrome. It is not showing up here also:
http://www.rexswain.com/httpview.html
http://google.com
sends a HTTP 302 Statuscode wich means that the location is not available (for now). But you receive a new location in the header data.
You can parse the header of the answer an get the new URL from the field location
. Try to open this new URL.
You should always check the header data of HTTP requests.
Similar to drkunibar, I would say to examine the headers. You can do this in both Chrome and Firefox. In Chrome, open tools>developer_tools and switch to the network tab. Then load the webpage. By clicking on the request, you can examine everything about it. If you don't see anything strange, then I would try examining it in a debugger. Good Luck!
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.