How to get status code 200 instead of 401 from api calls json response

Question

Before including user-agent as header I got 403 and after injecting user-agent I got 401. I'm not understanding How can I get 200 response.

Script:

import requests
import json
#url:https://www.barchart.com/stocks/quotes/MSFT/competitors?quoteSectors=-INO&page=1&orderBy=weightedAlpha&orderDir=desc
headers = {
    "x-xsrf-token": "eyJpdiI6IlhXSVFja0FhdzV1L3ZtRXBmbDFieXc9PSIsInZhbHVlIjoicUllR0hFL2NWbysxL2pjSVoyR2ZYYk03Zk42Qmh3MGJqckhSRUE3Vi9PREp1ME9jMVJlK2djSHNWRmVQWmcvRkxpM2VIUlFLTFZma2J2c2VLNDd5cGdJV0JnVEM2T0ZzUS9KTzJJSUFIWVlud0w0eGtyUzRRdHdlT2RKMVJPcFEiLCJtYWMiOiIzNjgxZTA1NTAyMjZiZGE2NGU1NmYwYjMxMTVjNjA3ZmU1Mjg0NTRjODA1OTdlNTZmYjU4YTFlNGFlMjkwZDQzIn0=",
    "User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36"
    }

params = {
    "symbol": "MSFT",
    "lists": "stocks.inSector.all(-INO)",
    "fields": "symbol,symbolName,weightedAlpha,lastPrice,priceChange,percentChange,highPrice1y,lowPrice1y,percentChange1y,tradeTime,symbolCode,symbolType,hasOptions",
    "orderBy": "weightedAlpha",
    "orderDir": "desc",
    "meta": "field.shortName,field.type,field.description,lists.lastUpdate",
    "hasOptions": "true",
    "page": "1",
    "limit": "100",
    "raw": "1"
    }

api_url = 'https://www.barchart.com/proxies/core-api/v1/quotes/get?'
s=requests.Session()
req = s.get(api_url, headers=headers, params=params)
print(req)

Answer 1

You were very close to getting it right, the cookies are set by the landning page (which you commented out) so you need to go there and they will be added to the s variable. Then you correctly added the xsrf token to your headers but that needs to be the one that is in the response header after you land on the landing page. Also, it is URL encoded and needs to be decoded, this can be done by replacing the final %3D with an equals sign =

See below example:

import requests

s = requests.Session()

headers = {
    "User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36"
    }

url = 'https://www.barchart.com/stocks/quotes/MSFT/competitors?quoteSectors=-INO&page=1&orderBy=weightedAlpha&orderDir=desc'

landing_page = s.get(url,headers=headers)
print(landing_page)

cookies = s.cookies.get_dict()

params = {
    "symbol": "MSFT",
    "lists": "stocks.inSector.all(-INO)",
    "fields": "symbol,symbolName,weightedAlpha,lastPrice,priceChange,percentChange,highPrice1y,lowPrice1y,percentChange1y,tradeTime,symbolCode,symbolType,hasOptions",
    "orderBy": "weightedAlpha",
    "orderDir": "desc",
    "meta": "field.shortName,field.type,field.description,lists.lastUpdate",
    "hasOptions": "true",
    "page": "1",
    "limit": "100",
    "raw": "1"
    }

new_headers = {
    'accept':'application/json',
    'user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36',
    'x-xsrf-token':cookies['XSRF-TOKEN'].replace('%3D','=')}

api_url = 'https://www.barchart.com/proxies/core-api/v1/quotes/get?'
req = s.get(api_url, headers=new_headers, params=params).json()
print(req)