get json data from host that requires headers={'user-agent', 'cookie', x-xsrf-token'}

Question

• There is a server that contains a json dataset that I need
• I can manually use chrome to login to the url and use chrome developer tool to read the request header for said json data
• I determined that the minimum required headers that should be sent to the json endpoint are ['cookie', 'x-xsrf-token', 'user-agent']
• I don't know how I can get these values so that I can automate fetching this data. I would like to use request module to get the data
• I tried using selenium, to navigate to the webpage that exposes these header values, but cannot get said headers values (not sure if selenium supports this)
• Is there a way for me to use request module to inch towards getting these header values...by following the request header "bread crumbs" so to speak?
• Is there an alternative module that excels at this?
• To note, I have used selenium to get the required datapoints successfully, but selenium is resource heavy and prone to crash;
• By using the request module with header values greatly simplifies the workflow and makes my script reliable

Answer 1

Based on pguardiario's comment

Sessions cookies and csrf-token are provided by the host when a request is made against the Origin url. These values are needed to make subsequent requests against the endpoint with the JSON payload. By using request.session() against the Origin url, and then updating the header when using request.get(url, header ). I was able to access the json data