Python请求无效的URL标签错误

Question

I'm trying to access Shopify's API which uses a URL format of - https://apikey:password@hostname/admin/resource.xml

eg http://7ea7a2ff231f9f7:95c5e8091839609c864@iliketurtles.myshopify.com/admin/orders.xml

doing $curl api_url downloads the correct XML however when I do

 import requests
 api_url = 'http://7ea7a2ff231f9f7d:95c5e8091839609c864@iliketurtles.myshopify.com/admin/orders.xml'
 r = requests.get(api_url) # Invalid url label error

Any idea why I'm getting this? Curl / opening the link directly in the browser is working fine. Is it because the length of the URL is too long?

Thanks!

Answer 1

It's not the length of the URL. If I do:

import requests
test_url = 'http://www.google.com/?somereallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallyreallylongurl=true'
r = requests.get(test_url)

returns <Response [200]>

Have you tried making the request with the requests Authentication parameters detailed here

>>> requests.get('http://iliketurtles.myshopify.com/admin/orders.xml', auth=('ea7a2ff231f9f7', '95c5e8091839609c864'))
<Response [403]>

Answer 2

The error ( 'URL has an invalid label.' ) is probably a bug in requests library: it applies idna encoding (for internationalized domain names) on hostname with userinfo attached, source :

netloc = netloc.encode('idna').decode('utf-8')

that might raise 'label empty or too long' error for the long username:password. You can try to report it on the requests' issue tracker .

a:b@example.com form is deprecated otherwise requests.get('https://a:b@example.com') should be equivalent to requests.get('https://example.com', auth=('a', 'b')) if all characters in username:password are from [-A-Za-z0-9._~!$&'()*+,;=] set.

curl and requests also differ then there are percent-encoded characters in userinfo eg, https://a:%C3%80@example.com leads to curl generating the following http header:

Authorization: Basic YTrDgA==

but requests produces:

Authorization: Basic YTolQzMlODA=

ie:

>>> import base64
>>> base64.b64decode('YTrDgA==')
'a:\xc3\x80'
>>> print _
a:À
>>> base64.b64decode('YTolQzMlODA=')
'a:%C3%80'