Having trouble reading HTML in Java

Question

I'm a student who wants to have some fun during his summer vacation. There is a website called www.saltybet.com , where a fighting game is played by two AI's. Players can bet fake money on the outcome.

I want to make a piece of software in Java that stores every encountered matchup of the game in a MySQL database, then places a bet based on previous results. To do this, a first step would be reading the names of the two AI players.

Now, when I highlight one of the character names (bottom right and bottom left) in Firefox, and check the source of the website, it comes up as this:

<div class="left">
    <span id="p1name" class="redtext">Lobo</span><br><br />
    <span class="field" id="player1wager">$2484343</span><br />
</div>

However, when I choose to view the entire page source, it comes up as this:

<div class="left">
    <span id="p1name" class="redtext">Player 1</span><br/><br />
    <span class="field" id="player1wager">$0</span><br />
</div>

Note that "Lobo" changed to "Player 1". Now, I know how to read a website's HTML code into Java. However, I keep getting "Player 1" and "Player 2" instead of the character names. Does anyone have an idea as to how to get the correct player names out of the page source?

Answer 1

You don't need to read the HTML of this page in Java. By looking at the source, this website gets its data by AJAX from the file:

http://www.saltybet.com/betdata.json

It is updated quite often. An example content:

{"p1name":"The atom","p2name":"Sponge bob","p1total":"0","p2total":"0","status":"open","alert":""}

Just download this file and use a JSON java library to convert the content to an object:

• http://code.google.com/p/google-gson/
• http://jackson.codehaus.org/
• ...

Answer 2

The fields are being updated by code Javascript when the page opens. This makes things difficult if you want to keep programming in Java, but here are a couple of suggestions of what you could do:

• Have a look at something like Selenium - I'm not that experienced with it myself but it allows for webpage automation and can run as a Firefox plugin. This might help because if you open the page in a web browser it would allow the Javascript to execute.
• Figure out how the Javascript on the page works. This might be tough, but perhaps you can find out how they query to server to fetch the player names, which would make your project much easier.