404 error when parsing URL using jsoup

Question

I am getting a 404 error when using Jsoup. The call is Document doc = Jsoup.parse(url, 30000) and the URL string is http://www.myland.co.il/%D7%9E%D7%97%D7%A9%D7%91-%D7%94%D7%A9%D7%A7%D7%99%D7%94 and the URL displays fine in Chrome. The error I am getting is java.io.IOException: 404 error loading URL http://www.myland.co.il/vmchk/××ש×-×שק××

Any ideas?

Answer 1

Don't use parse() -method for websites, use connect() instead. So you can set more connection settings.

final String url = "http://www.myland.co.il/%D7%9E%D7%97%D7%A9%D7%91-%D7%94%D7%A9%D7%A7%D7%99%D7%94";

Document doc = Jsoup.connect(url).get();

However the problem is the url-encoding:

Exception in thread "main" org.jsoup.HttpStatusException: HTTP error fetching URL. Status=404, URL=http://www.myland.co.il/vmchk/××ש×-×שק××

Even decoding the url back to utf-8 doesn't solve this.

Do you have an "alternative" url?

Answer 2

try decodeURL()

String url = "http://www.myland.co.il/%D7%9E%D7%97%D7%A9%D7%91-%D7%94%D7%A9%D7%A7%D7%99%D7%94";
Document doc = Jsoup.connect(url.decodeURL()).get();

Answer 3

Remember to add to connection of Jsoup this:

 Jsoup.connect(url)
    .ignoreHttpErrors(true)
    .get();