Request times out, cURL does not

Question

I'm trying to pull the play by play from NBA STATS api using the python requests package. I kept running in to timeout errors, so I tried it in cURL.

Here's the url for the request of the single game I'm trying to get:

http://stats.nba.com/stats/playbyplayv2?EndPeriod=10&GameID=0021500492&StartPeriod=1

You can go to this link, check it out, everything on that end works.

I tried doing a cURL request to just that link, and it timed out. Eventually, I inserted the Request Headers as cURL parameters one by one until it worked.

curl -X GET -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" -H "User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0" -H "Accept-Language: en-US,en;q=0.5" -H "Accept-Encoding: gzip, deflate" "http://stats.nba.com/stats/playbyplayv2?EndPeriod=10&GameID=0021500492&StartPeriod=1"

Here's my query in requests:

pars = {'Accept-Encoding': 'gzip, deflate'\\ , 'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0'\\ , 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'\\ , 'Accept-Language':'en-US,en;q=0.5'} response = requests.get('http://stats.nba.com/stats/playbyplayv2?EndPeriod=10&GameID=0021500492&StartPeriod=1'\\ , params=pars)

I either need to get this working in python through requests, or figure out a way to do this in cURL in such a way that I can (a) fill in the GameID part based on text from a file/filename in a loop and (b) save it as a json in the correct directory (not working directory).

Any ideas? Is there a urllib based solution?

Answer 1

You're passing header information as parameters. You need to pass it through headers.

import requests

headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) 
Gecko/20100101 Firefox/53.0'}

params = {
    'EndPeriod': 10,
    'GameID': '0021500492',
    'StartPeriod': 1
}
r = requests.get("http://stats.nba.com/stats/playbyplayv2", params=params, 
headers=headers)
print(r.json())

The params in requests is used to pass in parameter information to the API in order for the API to process your request on their end. Whereas Headers are used to authenticate or interpret your request.