从urllib2获取数据或在Python中请求

Question

我必须从python获取搜索结果。

假设以下是我在搜索heart关键字的目标url。

http://journals.aps.org/search/results?clauses=[{%22operator%22:%22AND%22,%22field%22:%22all%22,%22value%22:%22heart%22}]&sort=relevance

但是urllib2搜索结果和request没有到来。

在以下代码上工作：

根据request

>>> import requests
>>> url = "http://journals.aps.org/search/results"
>>> payload = {"clauses": [{"operator":"AND","field":"all","value":"heart"}], "sort": "relevance"}
>>> r = requests.get(url, params=payload)
>>> b = r.text

由urllib2

>>> import urllib2
>>> search_url ="http://journals.aps.org/search/results?clauses=%5B%7B%22operator%22%3A%22AND%22%2C%22field%22%3A%22all%22%2C%22value%22%3A%22heart%22%7D%5D&sort=relevance"
>>> 
>>> req = urllib2.Request(search_url, headers={'User-Agent' : "Mozilla/5.0"})
>>> f1 = urllib2.urlopen(req)

上述脚本无法获得正确的结果。

Answer 1

从您的问题尚不清楚您被困在哪里，两个示例都对我有用。 您期望的“正确结果”是什么？

>>> import requests
>>> url = "http://journals.aps.org/search/results"
>>> payload = {"clauses": [{"operator":"AND","field":"all","value":"heart"}], "sort": "relevance"}
>>> r = requests.get(url, params=payload)
>>> print r.text[:40]
<!DOCTYPE html><!--[if IE 8]><html class
>>>>

和：

>>> import urllib2
>>> search_url ="http://journals.aps.org/search/results?
>>> clauses=%5B%7B%22operator%22%3A%22AND%22%2C%22field%22%3A%22all%22%2C%22value%22%3A%22heart%22%7D%5D&sort=relevance"
>>> req = urllib2.Request(search_url, headers={'User-Agent' : "Mozilla/5.0"})
>>> f1 = urllib2.urlopen(req)
>>> print f1.read()[:40]
<!DOCTYPE html><!--[if IE 8]><html class
>>>