邮差得到状态代码200没有标题GET METHOD但python请求得到500错误

Question

[enter image description here][1]As a title, I can't get the content of website with python requests , but works with postman

I had tried to connect website with postman and get content of the web page, but I couldn't achieve it when I executed the code that was produced by postman like below, Instead, I got error 500 status code with python 2 and 3.

import requests

url = "https://www.screener.reuters.wallst.com/Stock/US/ResultsTable"

querystring = {"returnChoice":"","page":"2","sortBy":"RCCMultexCompanyName","sortDir":"A","quickscreen":"gaarp","criteria":"%5B%22StockUniverse%22%2C%22PriceEarnings%22%2C%22EPSGrowthRate%22%2C%22Region%22%2C%22SectorAndIndustry%22%2C%22PEGRatio%22%5D","Currency":"USD","PEGRatio":"%7B%22view%22%3A%22button%22%2C%22button_inputs%22%3A%5B%5D%2C%22range_inputs%22%3A%22LSS%7C1%22%7D","SectorAndIndustry":"%7B%22industries%22%3A%5B%2257111%22%2C%2257112%22%2C%2257121%22%2C%2257131%22%2C%2257132%22%2C%2257211%22%2C%2257212%22%5D%7D","Region":"%7B%22countries%22%3A%5B%22TW%22%5D%7D","EPSGrowthRate":"%7B%22view%22%3A%22button%22%2C%22button_inputs%22%3A%5B%5D%2C%22range_inputs%22%3A%22GTR%7C15%22%7D","PriceEarnings":"%7B%22view%22%3A%22button%22%2C%22button_inputs%22%3A%5B%5D%2C%22range_inputs%22%3A%22GEQ%7C0%7CLEQ%7C15%22%7D","StockUniverse":"%7B%22button_inputs%22%3A%5B%22LIKE%7CUnited%2BStates%22%2C%22NOTLIKE%7CUnited%2BStates%22%5D%7D","OriginalCurrency":"USD%0A"}

headers = {
    'cache-control': "no-cache",
    }

response = requests.request("GET", url, headers=headers, params=querystring)

print(response.text)

I excepted get 200 status code，but got 500 actually. It is strange because postman can get right result and python cannot,even I had filled in the headers.

headers = {
    'Accept-Encoding': "gzip, deflate, br",
    'Accept-Language': "zh-TW,zh;q=0.9,en-US;q=0.8,en;q=0.7",
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.90 Safari/537.36',
    'Content-Type': "application/x-www-form-urlencoded",
    'Accept': "application/json",
    'X-Requested-With': "XMLHttpRequest",
    'Connection': "keep-alive",
    'cache-control': "no-cache",
    }



The below url can get content if you connect it with browser or postman but  can't  get if you connect it with python reqeusts or urllib. 

https://www.screener.reuters.wallst.com/Stock/US/ResultsTable?returnChoice=&page=2&sortBy=RCCMultexCompanyName&sortDir=A&quickscreen=gaarp&criteria=%5B%22StockUniverse%22%2C%22PriceEarnings%22%2C%22EPSGrowthRate%22%2C%22Region%22%2C%22SectorAndIndustry%22%2C%22PEGRatio%22%5D&Currency=USD&PEGRatio=%7B%22view%22%3A%22button%22%2C%22button_inputs%22%3A%5B%5D%2C%22range_inputs%22%3A%22LSS%7C1%22%7D&SectorAndIndustry=%7B%22industries%22%3A%5B%2257111%22%2C%2257112%22%2C%2257121%22%2C%2257131%22%2C%2257132%22%2C%2257211%22%2C%2257212%22%5D%7D&Region=%7B%22countries%22%3A%5B%22TW%22%5D%7D&EPSGrowthRate=%7B%22view%22%3A%22button%22%2C%22button_inputs%22%3A%5B%5D%2C%22range_inputs%22%3A%22GTR%7C15%22%7D&PriceEarnings=%7B%22view%22%3A%22button%22%2C%22button_inputs%22%3A%5B%5D%2C%22range_inputs%22%3A%22GEQ%7C0%7CLEQ%7C15%22%7D&StockUniverse=%7B%22button_inputs%22%3A%5B%22LIKE%7CUnited%2BStates%22%2C%22NOTLIKE%7CUnited%2BStates%22%5D%7D&OriginalCurrency=USD

And below is postman screenshot
  [1]: https://i.stack.imgur.com/DO8ev.png

Answer 1

I found postman will store cookie secretly even if cookies doesn't present in the python code which is postman generated with new desktop postman version.

The way to get right content is to requests post below url with corrent params https://www.screener.reuters.wallst.com/Stock/US/Results It will get 500 error,but with this session I can get the content of my question.