So there's a web page that I want to retreive with python.Requests
https://ororo.tv/api/v2/episodes/9
which requires basic authentication. If I do it with curl like this
curl -u test@example.com:password https://ororo.tv/api/v2/episodes/9
I get the response I want, however, when trying to do the same in python with Requests
library, like this
>>> r = requests.get('https://ororo.tv/api/v2/episodes/9', auth=('test@example.com', 'password'))
>>> r
<Response [520]>
I always get 520 response. Could someone tell me, what i could be doing wrong ?
Yes, there are subtle differences. There are slight differences in the headers being sent, and those apparently matter to this API.
If you change the URL queried to use http://httpbin.org/get
(an end-point of the online HTTP test service HTTPBin.org , you can see the differences in what curl
and requests
send:
$ curl -u test@example.com:password http://httpbin.org/get
{
"args": {},
"headers": {
"Accept": "*/*",
"Authorization": "Basic dGVzdEBleGFtcGxlLmNvbTpwYXNzd29yZA==",
"Host": "httpbin.org",
"User-Agent": "curl/7.51.0"
},
"origin": "84.92.98.170",
"url": "http://httpbin.org/get"
}
$ python -c "import requests; print(requests.get('http://httpbin.org/get', auth=('test@example.com', 'password')).text)"
{
"args": {},
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Authorization": "Basic dGVzdEBleGFtcGxlLmNvbTpwYXNzd29yZA==",
"Host": "httpbin.org",
"User-Agent": "python-requests/2.11.1"
},
"origin": "84.92.98.170",
"url": "http://httpbin.org/get"
}
To highlight the differences:
requests
sends one extra header, Accept-Encoding
, set to gzip, deflate
User-Agent
header differs; both reflect the current agent. You'd have to see which one of these headers causes the issue on the https://ororo.tv/api/v2
site. When I correct the URL to use v2
and https
, like the curl
command, and set the User-Agent
header then I get a valid response:
>>> headers = {'User-Agent': 'curl/7.51.0'}
>>> r = requests.get('https://ororo.tv/api/v1/episodes/9',
auth=('test@example.com', 'password'), headers=headers)
>>> r
<Response [200]>
>>> from pprint import pprint
>>> pprint(r.json())
{'airdate': '2005-10-13',
'download_url': 'https://static-uk2.ororo.tv/uploads/video/file/9/Everybody.Hates.Chris.S01E04.DVDRip.Everybody.Hates.Sausage_1480525209.mp4?attachment=true&wmsAuthSign=aWQ9ODAzNDI3Kyt2aWRlbys5JnNlcnZlcl90aW1lPTIvOC8yMDE3IDI6Mjc6MDQgUE0maGFzaF92YWx1ZT1kbEpGM3c1bldSOXBOMUg5V2N1S0NnPT0mdmFsaWRtaW51dGVzPTk2MCZzdHJtX2xlbj05NQ%3D%3D',
'id': 9,
'name': 'Everybody Hates Sausage',
'number': '4',
'plot': 'When Julius buys a big crate of sausage, he makes everyone eat it '
'with every meal. But Tonya refuses to, causing friction between the '
'her and Rochelle. While at school, Chris is sentenced to 3 days of '
'detention after a rumor goes round about him beating up the school '
'bully, Joey.',
'resolution': 'SD',
'season': 1,
'show_name': 'Everybody hates Chris',
'subtitles': [{'lang': 'en',
'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/4867/Everybody.Hates.Chris.S01E04.DVDRip.Everybody.Hates.Sausage.eng.vtt'},
{'lang': 'ru',
'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/28629/Everybody.Hates.Chris.S01E04.DVDRip.Everybody.Hates.Sausage.vtt'},
{'lang': 'es',
'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/55744/1x04_EH_Sausage.vtt'},
{'lang': 'pt',
'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/124429/Everybody_Hates_Chris_104_-_Everybody_Hates_Sausage.vtt'},
{'lang': 'cs',
'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/217213/Everybody_Hates_Chris_104_-_Everybody_Hates_Sausages.vtt'},
{'lang': 'tr',
'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/192405/Everybody_Hates_Chris_S01E04_-_Everybody_Hates_Sausages-tur.vtt'}],
'updated_at': 1480640069,
'url': 'https://static-gra.ororo.tv/uploads/video/file/9/Everybody.Hates.Chris.S01E04.DVDRip.Everybody.Hates.Sausage_1480525209.smil/playlist.m3u8?wmsAuthSign=aWQ9ODAzNDI3Kyt2aWRlbys5JnNlcnZlcl90aW1lPTIvOC8yMDE3IDI6Mjc6MDQgUE0maGFzaF92YWx1ZT1FajlGK2JPMEd3aU1Lc3lnN1M4NlpBPT0mdmFsaWRtaW51dGVzPTk2MCZzdHJtX2xlbj05Ng%3D%3D'}
It's possible that the api you're trying to use requires you to format your request in a specific way, possibly requiring headers and base64 encoded authentication, when using something like Python's requests.
Check out this example, which will show you how to both send a base64 encoded authentication header, along with some data:
import requests
import base64
username = "some_username"
password = "some_password"
request_url = "https://ororo.tv/api/v2/episodes/9"
# In this example, I'm passing some data along with the request.
# these are generally what you would expect to pass along in an encoded url:
# /api?some_url_param_key=some_url_param_value
data = {}
data["some_url_param_key"] = "some_url_param_value"
# This is an example header, not necessarily what you need,
# but it should serve as a good starting point.
headers = {}
headers["Authorization"] = "Basic " + base64.encodestring(username + ":" + password).replace('\n', '')
headers["Accept"] = "*/*"
headers["Content-Type"] = "application/x-www-form-urlencoded"
headers["User-Agent"] = "runscope/0.1"
# You can use post() in some cases where you would expect to use get().
# Every API is its own unique snowflake and expects different inputs.
# Try opening up the Chrome console and run the request in the
# browser, where you know it works. Examine the headers and response
# in cases where the API you're accessing doesn't provide you
# with the necessary inputs.
result = requests.post(request_url, headers=headers, data=data)
print result
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.