python requests and urllib https errors

Question

I want to read the data from the nasa earth api , opening the url in the browser displays the data. When I try to make a GET request with python and urllib it throws an error.

request.urlopen("https://api.nasa.gov/planetary/earth/imagery?lon=100.75&lat=1.5&date=2014-02-01&api_key=DEMO_KEY").read()

urllib.error.HTTPError: HTTP Error 400: Bad Request

When I try it with Requests. It returns an error.

r = requests.get("https://api.nasa.gov/planetary/earth/imagery?lon=100.75&lat=1.5&date=2014-02-01&api_key=DEMO_KEY")

r.content is:

{"error": {"code": "HTTPS_REQUIRED", "message": "Requests must be made over HTTPS. Try accessing the API at: https://api.nasa.gov/planetary/earth/imagery/?lon=100.75&lat=1.5&date=2014-02-01&api_key=DEMO_KEY "}}

If i print out r.url it is http and not https:

http://api.nasa.gov/planetary/earth/imagery/?lon=100.75&lat=1.5&date=2014-02-01&api_key=DEMO_KEY

I dunno why it happens i am using python 3.7. Any help is appreciated

Answer 1

I was able to reproduce your mistake. However, when I copied the link from the Nasa website, it worked:

r = requests.get("https://api.nasa.gov/planetary/earth/imagery/?lon=100.75&lat=1.5&date=2014-02-01&api_key=DEMO_KEY")
r.json()

Answer 2

Are you sure your URL is correct? When i run this in request with debug logging i see that the first request gets HTTP 301 redirected

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.nasa.gov:443
send: b'GET /planetary/earth/imagery?lon=100.75&lat=1.5&date=2014-02-01&api_key=DEMO_KEY HTTP/1.1\r\nHost: api.nasa.gov\r\nUser-Agent: python-requests/2.22.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'
reply: 'HTTP/1.1 301 MOVED PERMANENTLY\r\n'
header: Server: openresty
header: Date: Tue, 25 Feb 2020 10:56:09 GMT
header: Content-Type: text/html; charset=utf-8
header: Content-Length: 399
header: Connection: keep-alive
header: X-RateLimit-Limit: 40
header: X-RateLimit-Remaining: 36
header: Location: http://api.nasa.gov/planetary/earth/imagery/?lon=100.75&lat=1.5&date=2014-02-01&api_key=DEMO_KEY

The URL returned here is http which then results in a request to that which returns a HTTP 400 bad request

send: b'GET /planetary/earth/imagery/?lon=100.75&lat=1.5&date=2014-02-01&api_key=DEMO_KEY HTTP/1.1\r\nHost: api.nasa.gov\r\nUser-Agent: python-requests/2.22.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'
DEBUG:urllib3.connectionpool:http://api.nasa.gov:80 "GET /planetary/earth/imagery/?lon=100.75&lat=1.5&date=2014-02-01&api_key=DEMO_KEY HTTP/1.1" 400 None
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Server: openresty
header: Date: Tue, 25 Feb 2020 10:56:09 GMT

Looking at your URL vs the one that it tells you to use they are differnt.

Your URL : https://api.nasa.gov/planetary/earth/imagery?lon=100.75&lat=1.5&date=2014-02-01&api_key=DEMO_KEY

Their URL: https://api.nasa.gov/planetary/earth/imagery/?lon=100.75&lat=1.5&date=2014-02-01&api_key=DEMO_KEY

You look like you are missing a / after the word imagery. When i use the URL they suggest i get data back like

b'{\n  "date": "2014-02-04T03:30:01", \n  "id": "LC8_L1T_TOA/LC81270592014035LGN00", \n  "resource": {\n    "dataset": "LC8_L1T_TOA", \n    "planet": "earth"\n  }, \n  "service_version": "v1", \n  "url": "https://earthengine.googleapis.com/api/thumb?thumbid=1e37797ab6e6638b5a0d02392acb479f&token=dc7d50c412dd5dcd7b014d52f0a1f91c"\n}'