How can I log in this specific website using Python Requests?

Question

I am trying to log in this website using the following request but it doesn't work

The cookie never contains 'userid'.

What should I change? Do I need to add headers in my post request?

import requests

payload = {
    'ctl00$MasterMainContent$LoginCtrl$Username': 'myemail@email.com',
    'ctl00$MasterMainContent$LoginCtrl$Password': 'mypassword',
    'ctl00$MasterMainContent$LoginCtrl$cbxRememberMe' : 'on',
}

with requests.Session() as s:
    login_page = s.get('http://www.bentekenergy.com/')
    response = s.post('http://benport.bentekenergy.com/Login.aspx', data=payload)
    if 'userid' in response.cookies:
       print("connected")
    else:
       print("not connected")

Edit 1 (following comments): I am not sure about what to put in the request headers, below is what I tried, but unsuccessfully.

request_headers = {
    'Accept':'image/webp,image/*,*/*;q=0.8',
    'Accept-Encoding':'gzip, deflate, sdch, br',
    'Accept-Language':'en-US,en;q=0.8',
    'Connection':'keep-alive',
    'Cookie':'ACOOKIE=C8ctADJmMTc1YTRhLTBiMTEtNGViOC1iZjE0LTM5NTNkZDVmMDc1YwAAAAABAAAASGYBALlflFnvWZRZAQAAAABLAAC5X5RZ71mUWQAAAAA-',
    'Host':'statse.webtrendslive.com',
    'Referer':'https://benport.bentekenergy.com/Login.aspx',
    'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'
}

Edit 2 (following stovfl answer):

I use now the following payload, fill each attributes with the value in the form and complete it with username, password and rememberMe. I also tried with the following headers in the request. Still not connected

payload = {
    '__VIEWSTATE' : '',
    '__VIEWSTATEGENERATOR' : '',
    '__PREVIOUSPAGE' : '',
    '__EVENTVALIDATION' : '',
    'isAuthenticated' : 'False',
    'ctl00$hfAccessKey' : '',
    'ctl00$hfVisibility' : '',
    'ctl00$hfDateTime' : '',
    'ctl00$hfHash' : '',
    'ctl00$hfAnnouncementsUrl' : '',
    'ctl00$MasterMainContent$LoginCtrl$Username' : '',
    'ctl00$MasterMainContent$LoginCtrl$Password' : '',
    'ctl00$MasterMainContent$LoginCtrl$cbxRememberMe' : '',
}

request_headers = {
        'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
        'Accept-Encoding':'gzip, deflate, br',
        'Accept-Language':'en-US,en;q=0.8',
        'Cache-Control':'max-age=0',
        'Connection':'keep-alive',
        'Content-Length':'7522',
        'Content-Type':'application/x-www-form-urlencoded',
        'Cookie':'',
        'Host':'benport.bentekenergy.com',
        'Origin':'https://benport.bentekenergy.com',
        'Referer':'https://benport.bentekenergy.com/Login.aspx',
        'Upgrade-Insecure-Requests':'1',
        'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'
}

with requests.Session() as s:
response = s.get('http://benport.bentekenergy.com/Login.aspx')
soup = BeautifulSoup(response.text, "html.parser")
if soup.find("input", {"name" : "ctl00$MasterMainContent$LoginCtrl$Username"}):
    print("not connected")
    soup = BeautifulSoup(response.text, "lxml")
    for element in soup.select("input"): 
       if element.get("name") in payload:
           payload[element.get("name")] = element.get("value")

    payload['ctl00$MasterMainContent$LoginCtrl$Username'] = 'myemail@email.com'
    payload['ctl00$MasterMainContent$LoginCtrl$Password'] = 'mypassword'
    payload['ctl00$MasterMainContent$LoginCtrl$cbxRememberMe'] = 'on'

    response = s.post('http://benport.bentekenergy.com/Login.aspx', data=payload, headers=request_headers)

    print (s.cookies)
    soup = BeautifulSoup(response.text, "html.parser")
    if soup.find("input", {"name" : "ctl00$MasterMainContent$LoginCtrl$Username"}):
            print("not connected")
    else:
            print("connected")

s.cookies contains:

<RequestsCookieJar[<Cookie BenportState=q1k2r2eqftltjm55igy5mg55 for .bentekenergy.com/>, <Cookie RememberMe=True for .bentekenergy.com/>]>

Edit 3 (answer!):

I added

'__EVENTTARGET' : ''

in the payload and filled it with the value 'ctl00$MasterMainContent$LoginCtrl$btnSignIn'

Now I am connected! NB: the headers were not necessary, just the payload

Answer 1

Comment : ... found that there is a parameter '__EVENTTARGET' that was not in the payload. It needed to contain 'ctl00$MasterMainContent$LoginCtrl$btnSignIn'. Now I am connected!

Yes, overlooked the Submit Button , there is a Javascript :

href="javascript:WebForm_DoPostBackWithOptions(new WebForm_PostBackOptions("ctl00$headerLoginCtrl$btnSignIn",

Relevant: SO Answer How To see POST Data

---

Comment : ... based on your answer (Edit 2). Still not connected

1. You are using http instead of https
   Will be Auto-Redirected to https .

2. The <RequestsCookieJar has changed, so some progress.
   I'm still unsure about your Authenticated Check: if soup.find("input", {"name"... .
   Have you Check the Page Content?
   Any Error Message?

3. Don't use BeautifulSoup(... your following Requests should be using Session s to reuse the assigned Cookie .
   Eg response = s.get('<url to some resticted page>

4. Try request_headers with only 'User-Agent'

---

Analysis <form> :
Login URL: https://benport.bentekenergy.com/Login.aspx
Form: action: /Login.aspx , method: post

If value not empty means: Pre-Set-Values from Login Page.

 1:input type:hidden   value:/wEPDwUKLT... id:__VIEWSTATE 
 2:input type:hidden   value:0BA31D5D      id:__VIEWSTATEGENERATOR 
 3:input type:hidden   value:2gILTn0H1S... id:__PREVIOUSPAGE 
 4:input type:hidden   value:/wEWDAKIr6... id:__EVENTVALIDATION 
 5:input type:hidden   value:False         id:isAuthenticated 
 6:input type:hidden   value:nu66O9eqvE    id:ctl00_hfAccessKey 
 7:input type:hidden   value:public        id:ctl00_hfVisibility 
 8:input type:hidden   value:08%2F16%2F... id:ctl00_hfDateTime 
 9:input type:hidden   value:3AB353573D... id:ctl00_hfHash 
10:input type:hidden   value://announce... id:ctl00_hfAnnouncementsUrl 
11:input type:text     value:empty         id:ctl00_MasterMainContent_LoginCtrl_Username 
12:input type:password value:empty         id:ctl00_MasterMainContent_LoginCtrl_Password 
13:input type:checkbox value:empty         id:ctl00_MasterMainContent_LoginCtrl_cbxRememberMe