Issues with logging on to a website with Python 2.7 requests

Question

I am attempting to automatically log on to the William Hill Website ( http://sports.williamhill.com/bet/en-gb ) using the requests module for Python 2.7

import requests

with requests.Session() as c:
    url = "https://sports.williamhill.com/bet/"
    Username = "blarxxx"
    PASSWORD = "blarxxx"
    r = c.get(url)
    Rcookies = r.cookies
    login_data = dict(username=Username, password=PASSWORD, action="DoLogin", target_page="http://sports.williamhill.com/bet/en-gb", tmp_username="Username")
    page1 = c.post(url,data=login_data, headers={"Referer" : "http://sports.williamhill.com/bet/en-gb/betting/t/295/English-Premier-League.html"})

print page1.content

However, this does not log me on.

I have reviewed the post data from Chrome's inspect element and there are two other variable posted, loginuid and ioBlackBoxCopy. I assumed that these would be provided by the site in a cookie however I have not been able to locate them.

So you are aware the cookie information from https://sports.williamhill.com/bet/ is as follows:

<<class 'requests.cookies.RequestsCookieJar'>[<Cookie CSRF_COOKIE=c2962a27a04d24250b13 for .williamhill.com/>, <Cookie TS01b0a0b6=0148840b445dac7f6388fc9588136faf4ebdb641d2b4ba2c7e1c3eda9a7079782232d97eaad7893999b63092cda29c0e07ad6757dfa994c354e332e3abda1d2bbfd684a7edc8beb96d001cad8478ffc5e53ab684ffaa39ed82c91fb9b77f5c70f63f174ad9421f9daa69f1d8a3170251041bca95512cbfcf8eaa3b483e3659120e2fc4690c for .williamhill.com/>, <Cookie cust_lang=en-gb for .williamhill.com/>, <Cookie cust_login= for .williamhill.com/>, <Cookie cust_prefs=en|ODDS|form|TYPE|PRICE|||0|SB|0|0||0|en|0|TIME|TYPE|0|1||0||0|1|0||TYPE| for .williamhill.com/>, <Cookie TS017d04d1=0148840b4403f8c68f0800245a6bf453cfe0f084759d52c21d738abcb353a043964350398d for sports.williamhill.com/>]>

I'm relatively new to the Requests module and to logging on to a website through python. I would be grateful if you could let me know if I am approaching this correctly and how I can obtain the additional inputs in my post request, if indeed they are needed.

Many thanks

J

Answer 1

Did you try to see the source of WH in browser? There are two hidden fields - login_uid and action. May be it will be usefull to you?

But I don't know if you will be able to download html at all because it is built by big js script which is loaded when you send request to WH.

PS I'm trying to do the same and only WebDriver works for me.