Request.post returns nothing

Question

I have very simple script, which should perform login process to webpage:

import requests
from bs4 import BeautifulSoup

headers = {'User-Agent': 'Mozilla/5.0'}
credentials = {'user': 'user',
               'pass': 'pass'}
session = requests.Session()
req = session.get('https://myurl.com', verify=False)
html = BeautifulSoup(req.text)

login_form = html.find('form', {'name': 'loginForm'})
login_url = login_form['action']

# at this moment login_url has this form: https://myurl.com/webclient;jsessionid=AC7F87C5D38C0B2EABBF6D76379BB75B?pageid=816
req2 = session.post(login_url, data=credentials, headers=headers, verify=False)
print 'req2:', req2.text    # prints nothing

Why req2.text (although req2.status_code returns 200) is empty? When I'm checking action atribute of form in html source I see URL without jsessionid . Why? And to which address should I send my credentials in request.post?

PS. pageid is generated dynamically, so this is the reason why I use BeautifulSoup to get this url from html form.

Answer 1

I found the problem and it was really trivial - submit button value is mandatory in this case. So all I need to do is just add button value to credentials dictionary.

Answer 2

Try encoding the values:

credentials = {
      'user'.encode("utf-8"): 'user'.encode("utf-8"),
      'pass'.encode("utf-8"): 'pass'.encode("utf-8")
}

I trust you to write it better. I had the same problem and that was the solution.