How to stay logged in to a website with Python while running a program and requesting data?

Question

I found a way to stay logged in to a website with Python. The problem is that every once in a while, I get disconnected and logged out. I'm guessing that the session is timing out but I don't know how to fix it.

I used the Live HTTP Headers add-on for Firefox and copied headers from my login request into my program.

import urllib
import urllib2
import cookielib

data = urllib.urlencode({"inUserName":"MY EMAIL", "inUserPass":"MY PASSWORD"})
jar = cookielib.FileCookieJar("cookies")
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))
opener.addheaders.append(('User-agent', 'Mozilla/5.0'))
opener.addheaders.append(('Referer', 'http://www.locationary.com/'))
opener.addheaders.append(('Cookie','site_version=REGULAR; __utma=47547066.601656247.1344371114.1344602507.1344606239.16; __utmz=47547066.1344371114.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; jforumUserId=1; locaCountry=1227; locaState=null; locaCity=Atlanta; PSESSIONID=533e2fb9fda008d5d16bfbdc9b9a6afed0e5ac54; Locacookie=enable; sortOrder=1; JSESSIONID=DE58AC8BC78D1DF20BF338E195336E58; __utmc=47547066; __utmb=47547066.6.10.1344606239'))
request = urllib2.Request("https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction", data)
response = opener.open(request)
page = opener.open(url).read()
soup = BeautifulSoup(page)

I used cookielib and urrlib / urllib2 . The cookie that really makes it work is the really long one, But I don't really know what it all means and I just copied it in from the add-on. If my connection goes out, I go to my browser and log in again and get a new cookie from the add-on. Like I said before, I'm guessing it has to do with the session or the sessionid or something, but I don't know how I can make it so that I am always logged in.

Thanks.

EDIT

Can someone tell me why this is a bad question or am I just stupid? -1 for what?

EDIT 2

Okay! If I don't stay logged in, then is there a way to keep my connection/cookie from not working?

EDIT 3

I don't know how to get a new cookie other than going to the Firefox add-on myself...haha

EDIT 4

Okay. I made a new test program:

import urllib
import urllib2
import cookielib
import re

url = 'http://www.locationary.com/home/index2.jsp'

data = urllib.urlencode({"inUserName":"email", "inUserPass":"password"})
jar = cookielib.FileCookieJar("cookies")
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))
opener.addheaders.append(('User-agent', 'Mozilla/5.0 (Windows NT 6.1; rv:13.0) Gecko/20100101 Firefox/13.0.1'))
opener.addheaders.append(('Referer', 'http://www.locationary.com/'))
opener.addheaders.append(('Cookie','site_version=REGULAR; __utma=47547066.601656247.1344371114.1344612897.1344615635.18; __utmz=47547066.1344371114.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; jforumUserId=1; locaCountry=1227; locaState=null; locaCity=Atlanta; PSESSIONID=533e2fb9fda008d5d16bfbdc9b9a6afed0e5ac54; Locacookie=enable; sortOrder=1; JSESSIONID=781FD0C497FB596954BB78B1323215F6; __utmc=47547066; __utmb=47547066.9.10.1344615635'))
request = urllib2.Request("https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction", data)
response = opener.open(request)
page = opener.open(url).read()
print re.findall(r'<title>(.*)</title>', page)
h = response.info().headers
print h

Output:

['Home Page']
['Server: nginx/1.0.8\r\n', 'Date: Fri, 10 Aug 2012 16:50:47 GMT\r\n', 'Content-Type: text/html;charset=UTF-8\r\n', 'Transfer-Encoding: chunked\r\n', 'Connection: close\r\n', 'P3P: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT"\r\n']

Answer 1

Why don't you try using the requests module and seeing if that makes a difference?

Take a look at the examples here ; it's very easy to use, and the built-in cookie jar may help if only to prevent you from accidentally making mistakes in Python's otherwise miserable HTTP libraries.

Answer 2

You can use selenium, to use firefox to stay logged in into this webpage. selenium

from selenium import selenium

localport = 12345

## you create a socket proxy here listening on localport
## connecting to www.locationary.com:80

selenium = selenium("localhost", 4444, 
                    "*firefox", "http://localhost:%i" % localport)

while 1:
    # open the selenium side every 5 minutes
    selenium.open("/home/index2.jsp")
    selenium.wait_for_page_to_load("30000")
    time.sleep(5 * 60) # seconds

to install selenium you need the python package and the selenium server .jar file to start it.

Because firefox logs in through you program, you can parse the traffic for valid session ids.

Questions?