Logging in to a site using Python Requests

Question

I am trying to programmatically log into this site using the Python Requests library:

login_url='https://login.drexel.edu/cas/login?service=https://one.drexel.edu/c/portal/login'

login_payload = {                                                         
    'username':'lk12',                                                    
    'password':'1q2w3e4r5t6y'                                                 
}

s = requests.post(login_url, data=login_payload)    

print s.text

Note, the username and password key was retrieved from the page source's username and password id field.

Upon running the script and looking at the output of s.text , it appears I was never logged in despite my login credentials being valid. The output of s.text is simply the login page.

Using Firefox, I checked what the cURL request looks like:

curl 'https://login.drexel.edu/cas/login?service=https%3A%2F%2Fone.drexel.edu%2Fc%2Fportal%2Flogin' -H 'Host: login.drexel.edu' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:35.0) Gecko/20100101 Firefox/35.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Referer: https://login.drexel.edu/cas/login?service=https%3A%2F%2Fone.drexel.edu%2Fc%2Fportal%2Flogin' -H 'Cookie: JSESSIONID=B3DD2F8E144D98090AC63B995D9030AB; IDMSESSID=lk12' -H 'Connection: keep-alive' --data 'username=lk12&password=1q2w3e4r5t6y&lt=LT-745860-yMF7kyKD3SSVfeUmXPiDLWQobbczCq&execution=e2s1&_eventId=submit&submit=Connect'

That request looks exactly like my Python request. Not quite sure what I am doing wrong

Answer 1

Someone can probably help you more, but here are a few things you can look at meanwhile:

1. Set your user agent to an actual browser. If you don't, it's set to "Python", and some(many?) websites can have that user-agent blocked by default. Test that by sending your script to: http://whatsmyuseragent.com/

How to set user-agent: Sending "User-agent" using Requests library in Python

2. If you plan on keeping the login session to make other requests, you should use session()

Python Requests and persistent sessions

3. Check the html for hiddin fields in the form. Looks like there are a few. And you may have to send those too. Looks like LT parameter may be an authentication token.

If that's true, you need to GET the loging page first, read the content, parse the page to extract the token string, and add that to your payload. Notice those are being sent over through your browser request. If you match all values sent, it should work. (except for taken value which is likely dynamically generated on each type you request the login page).

HIDDEN FIELDS

<input type="hidden" name="lt" value="LT-392592-vZcTMdl2wvqcC5WEBcW9foIIJiLoWz" />
<input type="hidden" name="execution" value="e2s1" />
<input type="hidden" name="_eventId" value="submit" />