Axios POST request receiving ERROR on Desktop App But Same URL Works In Browser
Question
I am building an application that scrapes and aggregates a lot of real estate data from various sites.
I am having an issue with one of the sites. I send an Axios POST request with the required form data.
When I take that exact url string and enter it in the browser, I am brought to the desired page. However, using the exact same data from my own server returns this:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd" >
<html>
<head>
<title>Search Results</title>
<meta http-equiv="Content-type" content="text/html;charset=utf-8">
<meta name="robots" content="noindex">
</head>
<form method=post>
<body bgcolor=WHITE text=BLACK link=GOLDENROD vlink=RED
alink=RED>
<input type=hidden name=POSTDATA value='{"ms_user":"monm","passwd":"data","srch_type":"1","select_cc":"0701","district":"0714","adv":"1","out_type":"2","ms_ln":"50","p_loc":"143-145+Chadwick+Ave","owner":"","block":"","lot":"","qual":""}' >
<input type=hidden name=pageno value='1' >
ERROR
I have tried sending the POST request with the entire query string as it would be in the browser. This is what that would be: https://tax1.co.monmouth.nj.us/cgi-bin/inf.cgi?ms_user=monm&passwd=data&srch_type=1&select_cc=0701&district=0714&adv=1&out_type=2&ms_ln=50&p_loc=141-143+Chadwick+Ave&owner=&block=&lot=&qual=
I have also tried sending the POST with the data object like so:
let asessmentURL = `https://tax1.co.monmouth.nj.us/cgi-bin/inf.cgi?`
//First we grab the body of the html with axios
const response = await axios.post(asessmentURL, {
'ms_user': 'monm',
'passwd': 'data',
'srch_type': '1',
'select_cc': '0701',
'district': '0714',
'adv': '1',
'out_type': '2',
'ms_ln': '50',
'p_loc': '143-145+Chadwick+Ave',
'owner': '',
'block': '',
'lot': '',
'qual': ''
}, { headers: { 'User-Agent': 'Mozilla/5.0' }
})
.then(response => console.log("RESPONSE: ", response.data))
I have also tried using POSTMAN to send the request and it returns the same HTML and ERROR as the application on my server.
Any ideas? Please and thank you in advance.
1 answers
solution1
0 2020-07-18 00:47:00
I fixed it. There were two pitfalls that I would like to mention in the event someone goes down this path one day.
Although the DEV Tools on Chrome AND the form method="post" said the request was a POST request, I produced the desired results using a GET request. After doing some research I saw that CGI form submissions need to be GET requests.
After incorporating that, I was no longer receiving the same error. Now I was getting a "property not found" error. I noticed that the address was being passed with + instead of spaces. I added the %20 for proper spacing and voila!!!
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.