Unexpected URL request result in Java

Question

There is a site http://l2on.net/ with dropdown list:

<select size="1" onchange="window.location.href = '/?c=market&amp;a=search&amp;q=&amp;type=0&amp;setworld=' + this.value;">
<option selected value="43">Cadmus</option>
<option value="44">Athebaldt</option>
<option value="45">Blackbird</option>
<option value="46">Ramsheart</option>
<option value="47">Esthus</option>
<option value="49">Lancer</option>
<option value="13">Airin</option>
<option value="5">Erica</option>
<option value="27">Elcardia</option>
</select>

My guess is that it is possible to select the server I need with "GET" query, just like it is in the html code.

<select size="1" onchange="window.location.href = '/?c=market&amp;a=search&amp;q=&amp;type=0&amp;setworld=' + this.value;">

After suggestions here code looks like:

    l2on = new URL("http://l2on.net/?setworld=BlackBird");
    l2onConn = l2on.openConnection();
    l2onConn.setRequestProperty("Accept-Charset", "UTF-8");
    l2onConn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0");
    l2onConn.connect();
    br = new BufferedReader(new InputStreamReader(l2onConn.getInputStream()));
    String line = null;
    while ((line = br.readLine()) != null) {
        content += line + System.lineSeparator();
    }
    System.out.println(content.contains("<option selected value=\"43\">Cadmus</option>"));

Also I tried UrlEncoder.encode() to encode query properly. And passing as setworld parameter id of server (which I have found in html page source code), like http://l2on.net/?setworld=45 .

All this things work perfectly from browser and don't work from Java( I know that because in accepted page after all I can find

<option selected value="43">Cadmus</option>

which shows that the server is not changed in any way, and info for default one is shown, and want to retrieve somehow information for all possible servers.

Sorry, I don't know much about html, so probably I'm doing something wrong,

Help please.

Answer 1

This may be a wild guess. Could it be possible that the reason it works in your browser is that, in your browser, you're already authenticated to the website?

My browser isn't authenticated to this website. When I try this URL:

http://l2on.net/?setworld=Atlant

I'm getting redirected to the home page: http://l2on.net .

Which is exactly what happens to your Java program.

Answer 2

Perhaps another wild guess, but this helped me recently when connecting to certain sites.

Some sites block certain user agents, and by default, java is using a user-agent that makes you look like a bot. It will be something like "Java/1.6.0_14" with the latter part depending on the java version.

Have you tried setting another user agent? You can do it like this:

l2onConn.setRequestProperty("User-Agent", "Agent_you_want");

I'd suggest trying what your browser is using to start. You can get that here .

If that works, then using something to identify yourself would probably be the right thing to do.