确定URL是否存在？

Question

I have a loop which iterates through a lot of URLs. My problem is that the program is writing out the content of each url in the terminal, and I just want to ignore the broken urls. How do I determine whether a URL is referring to something or not?

Am I forced to make use of the exception FileNotFoundException that is being thrown? Because of that it affects other parts of the program too, I want to ensure that the main while loop directly jumps to the next iteration if the url is broken. The exception is thrown by a method that I am using (within a class that I cannot change), how do I handle that?

Here is my loop (simplified):

while(!queue.isEmpty()) {
    URL thisURL = (URL)queue.poll();
    String page = Customurlclass.openURL(thisURL); // return a string containing the page that the url is refering to.
    System.out.println(page);
    // Some other things is also happening here, an I don't want them to happen if the url is broken.
}

So the openURL() is catching the FileNotFoundException and there is a lot of things printed in the terminal, I just want to ignore them, how do I do that?

Answer 1

To verify if your String is valid URL or not, you can use Apache commons-validator URLValidator class as below:

String[] schemes = {"http","https"}; // DEFAULT schemes = "http", "https", "ftp"
UrlValidator urlValidator = new UrlValidator(schemes);
if (urlValidator.isValid("ftp://foo.bar.com/")) {
   System.out.println("url is valid");
} else {
   System.out.println("url is invalid");
}

---

Or even if you like to do it without using the Apache common-validator you can have something like below:

try {
    URL url = new URL("http://www.yoursite.com/");
    URLConnection conn = url.openConnection();
    conn.connect();
} catch (MalformedURLException e) {
    // the URL is not in a valid form
} catch (IOException e) {
    // the connection couldn't be established
}