Can't figure out the correct Parameter to mimic a website form in Java

Question

I'm a beginner with website source code and Java, I'm trying to figure out a way to access a certain site programmatically. I figured out the login/authentication part, but now I'm stuck at something that should be fairly easy. I need to tell Java to select from a list of options, but I can't figure out what parameters to send for the following source code:

<FORM ACTION="/services/reinforce" METHOD="POST" onSubmit="return checkSubmit()">
<INPUT TYPE="hidden" NAME="call1" VALUE="search">
<TABLE  CLASS="Data_table" summary="Select an option"width="100%"><CAPTION class="ctext">Search:    </CAPTION>
<TR>
<TD CLASS="default"><LABEL for=input_id><SPAN class=fvisible>Options</SPAN></LABEL>
<SELECT NAME="val_num" SIZE="1"  ID="optionId">
<OPTION VALUE="">None
<OPTION VALUE="1">Option 1
<OPTION VALUE="2">Option 2
<OPTION VALUE="3">Option 3
<OPTION VALUE="4">Option 4
<OPTION VALUE="5">Option 5
<OPTION VALUE="6">Option 6
<OPTION VALUE="7">Option 7
...

For instance if I want to select option 1, what I usually do is

String data = URLEncoder.encode("val_num", "UTF-8") + "=" + URLEncoder.encode("1", "UTF-8"); 

Somehow it is working, its NOT returning a FileNotFoundException, but it's redirecting me to another link. When I do it manually in any browser it works fine and it redirects me to Option1. What might be wrong?

Answer 1

Are you also sending in the parameter corresponding to hidden input element.

String data = "call1=search&val_num=1";

(Note there is no need to URL-encode when you can see clearly that the string only contains URL-safe characters.)

Also I assuming you are passing this data string to POST request (not a GET request). Is that correct?

Answer 2

Here is a check list:

1. Are you making a POST request to /services/reinforce ?
2. Does the checkSubmit() JavaScript function make any changes to the form's input/select/textarea elements before the form is submitted? If so, you need to simulate those changes in the data that you send with the POST request. (It may help to edit your question and add the source of the checkSubmit() function.)
3. Are you setting the request content type to application/x-www-form-urlencoded ?
4. Are you sending the appropriate authorization? Usually web sites send a cookie upon successful authentication. Are you sending the correct Cookie header?
5. Is data the content body of the POST request?
6. Have you tried sending call1=search in the body as well? Perhaps the hidden field is also necessary.
7. What User-Agent header are you sending?