Python: How to obtain headers and payload information for requests.Session()?

Question

In Python, how can I obtain headers and payload information for a particular website to make requests via requests.Session() ?

eg:

headers = {
            'Host': 'www.testsite.com',
            'Accept': 'application/json',
            'Proxy-Connection': 'keep-alive',
            'X-Requested-With': 'XMLHttpRequest',
            'Accept-Encoding': 'gzip, deflate',
            'Accept-Language': 'en-us',
            'Content-Type': 'application/x-www-form-urlencoded',
            'Origin': 'http://www.testsite.com',
            'Connection': 'keep-alive',
            'User-Agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Mobile/11D257',
            'Referer': 'http://www.testsite.com/mobile'
}

Thank you in advance and will be sure to upvote and accept answer

Answer 1

Most of those headers are automatically supplied by the requests module. Here is an example:

import requests
from pprint import pprint

with requests.Session() as s:
    s.get('http://httpbin.org/cookies/set?name=joe')
    r = s.get('http://httpbin.org/cookies')
    pprint(dict(r.request.headers))

assert r.json()['cookies']['name'] == 'joe'

The output of the pprint() call is this:

{'Accept': '*/*',
 'Accept-Encoding': 'gzip, deflate',
 'Connection': 'keep-alive',
 'Cookie': 'name=joe',
 'User-Agent': 'python-requests/2.9.1'}

As you can see, s.get() fills in several headers.

Answer 2

A response object has a headers attribute:

import requests

with requests.Session() as s:
    r = s.get("http://google.es")
    print(r.headers)

Output:

>> {
    'Date': 'Tue, 22 Aug 2017 00:37:13 GMT', 
    'Expires': '-1', 
    'Cache-Control': 'private, 
     max-age=0', 
     'Content-Type': 'text/html; charset=ISO-8859-1',
     ...
    }