Logging in to this website using requests module in python3

Question

Here's what I have done so far. I had to extract the CSFRToken manually (I don't know regex, so that part is messy). Is CSFR part of cookies? Because my cookies only shows two other ID type params, so I dropped the cookie part and did it this way.

import requests


URL = r'http://login.cheezburger.com/'
client = requests.session()
login_page = client.get(URL)
index = login_page.text.find("CSRFToken")
token = login_page.text[index:index+90].split('"')[-2]   # This works, I guarantee :)
#print(token)         I checked it manually


login_data = {'rlm': 'Shopper',
              'for': r'http://login.cheezburger.com/',
              'username': 'myusername', 
              'password': 'mypassword',
              'CSRFToken': token}
req = client.post(URL, data=login_data)

Now, there is no error per say, but I am not logging in to this site either. The text of this request shows that I am still stuck in the login page!

The parameters send are (as shown in the dev tools of firefox):

rlm: 'Shopper'
for: 'http://login.cheezburger.com/'
username: 'myusername', 
password: 'mypassword',
CSRFToken: '8uhhbf67-1233-fff3-123g1-123123fsdfs22'

The websites source is as follows (the part that contains the form data):

<div class="contents-msl">
    <h2>Client Login</h2>
    <p>Enter username and password</p>
    <div class="form-all-msl">
        <form action="/login.action" id="loginForm" method="post"
              enctype="application/x-www-form-urlencoded"><input type=hidden name=rlm
                                                                 value="Shopper"><input
                type=hidden
                name=for
                value="http%3a%2f%2flogin%cheezburger%2ecom%2f">
            <ul class="form-section-msl">
                <label class="form-label-left-msl" for="loginUserName">
                    Username<span class="form-required">*</span>
                </label>

                <div class="form-input-msl">
                    <input type="text" class="form-textbox-msl" id="loginUserName" name="username"
                           size="20">
                </div>
                <label class="form-label-left-msl" for="loginPwd">
                    Password<span class="form-required-msl">*</span>
                </label>

                <div class="form-input-msl">
                    <input type="password" class="form-textbox-msl" id="loginPwd" name="password"
                           size="20">
                </div>
                <div class="form-input-msl">
                    <div class="form-single-column-msl">

                    </div>
                </div>
            </ul>
    </div>


            <button class="members-btn-msl" type="submit">Login</button>
<input type="hidden" name="CSRFToken" class="CSRFToken" value="8uhhbf67-1233-fff3-123g1-123123fsdfs22" />                    </form>
        </div>
    </div>

Answer 1

You maybe want to add some headers

headers = {
'Accept':
    'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'Accept-Encoding':
    'gzip, deflate, sdch',
'Accept-Language':
    'en-US,en;q=0.8,vi;q=0.6',
'Cache-Control':
    'max-age=0',
'Connection':
    'keep-alive',
'User-Agent':
    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36',

}

Then add headers to your code:

req = client.post(URL, data=login_data)

Good luck !