Get CSRF token using python requests

Question

I am currently using Python Requests, and need a CSRF token for logging in to a site. from my understanding requests.Session() gets the cookie, but obviously I need the token. And Also I would like to know where to place it in my code. import requests

user_name = input('Username:')
payload = {
'username': 'user_name',
'password': 'randompass123'
}


with requests.Session() as s:
p = s.post('https://examplenotarealpage.com', data=payload)

Answer 1

See the following code example. You can use it directly to login into a website that only uses cookies to store login information.

import requests

LOGIN_URL = 'https://examplenotarealpage.com'
headers = {
    'accept': 'text/html,application/xhtml+xml,application/xml',
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'
}

response = requests.get(LOGIN_URL, headers=headers, verify=False)

headers['cookie'] = '; '.join([x.name + '=' + x.value for x in response.cookies])
headers['content-type'] = 'application/x-www-form-urlencoded'
payload = {
    'username': 'user_name',
    'password': 'randompass123'
}

response = requests.post(LOGIN_URL, data=payload, headers=headers, verify=False)
headers['cookie'] = '; '.join([x.name + '=' + x.value for x in response.cookies])

There are a few possible locations of the CSRF token. Different websites use different ways to pass it to browser. Here are some of them:

• It can come with response headers, in that case getting it is easy.

• Sometimes page meta holds the CSRF token. You have to parse the html content of the page to get it. Find the proper CSS selector for it. See an example:

   from bs4 import BeautifulSoup soup = BeautifulSoup(response.text, 'lxml') csrf_token = soup.select_one('meta[name="csrf-token"]')['content']

• It can be inside of a script tag with JavaScript code. Getting it will be tricky. But, you can always use regex to isolate it.

Answer 2

import requests
from bs4 import BeautifulSoup
headers = {'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 
           (KHTML, like Gecko) Chromium/80.0.3987.160 Chrome/80.0.3987.163 
           Safari/537.36'
 }
login_data = {
             'name' : 'USERNAME',
             'pass' : 'PASSWORD',
             'form_id':'new_login_form',
             'op':'login'
  }

with requests.Session() as s:
    url = 'https://www.codechef.com/'
    r = s.get(url,headers=headers,verify=False)
    #print(r.content) # to find name of csrftoken and form_build_id
    soup = BeautifulSoup(r.text, 'lxml')

    csrfToken = soup.find('input',attrs = {'name':'csrfToken'})['value']
    form_build_id = soup.find('input',attrs = {'name':'form_build_id'}) 
    ['value']

    login_data['csrfToken'] = csrfToken
    login_data['form_build_id'] = form_build_id

    r = s.post(url,data=login_data,headers = headers)
    print(r.content)

You can directly use this but their are few things to change:
1.check your user-agent in your browser network option
2.check your name attribute for csrf-token and form_build_id by print(r.content) and find csrftoken and form-build-id and check their name attribute.

final step :

search logout in your r.content if it is their then you are login.

Answer 3

I put it out here because it took me a lot of time and analysis of the network interaction to find this answer...

I had to login to a swagger/openAPI with python/requests. I could login to the site with a browser, but to login with requests I would need the x_csrf_token/sails.sid combo...

After trying and failing all answers here and otherwhere, checked the browser communication. It turns out the only way was to first get the 'sails.sid', and then do a GET to the undocumented(?) /csrfToken...

base_host = '...'
base_path= '/api/v2'
base_url = base_host + base_path
data = {
  "email": "...",
  "password": "..."
}
resp = requests.post(f"{base_url}/login", data=data)
session_cookie = resp.cookies
session_dict=session_cookie.get_dict()

sails_sid = session_dict.get('sails.sid','could not get valid [sails.sid]')
print(f'sails.sid:{[sails_sid]}')

Then:

cookies = {
    'sails.sid': sails_sid,
}
headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:100.0) Gecko/20100101 Firefox/100.0',
}
r2 = requests.get(f"{base_url}/csrfToken",  cookies=cookies, headers=headers)
print(r2.json())

Notice that in my case it was emai/password... I found all this from analysing the browser via Firefox inspect, so that might be your last option too...