通过urllib2 python发送/设置cookie

Question

I am quite new to python and I am rather struck for a couple of days now trying to send a cookie with urllib2. So, basically, on the page I want to get, I see from firebug that there is a "sent cookie" which looks like:

 list_type=height

.. which basically arranges the list on the page in a certain order.

I would like to send this above cookie info via urllib2, so that the rendered page taked this above setting into effect - and here is the code I am trying to write to make it work:

class Networksx(object):
    def __init__(self):
        self.cj = cookielib.CookieJar()
        self.opener = urllib2.build_opener\
                #socks handler
        self.opener.addheaders = [
        ('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13'),
        ('Accept-Charset', 'ISO-8859-1,utf-8;q=0.7,*;q=0.7'),
        ('Keep-Alive', '115'),
        ('Connection', 'keep-alive'),
        ('Cache-Control', 'max-age=0'),
        ('Referer', 'http://www.google.com'),
        ("Cookie", {"list_type":"height"}),
    ]
    urllib2.install_opener(self.opener)
    self.params = { 'Set-Cookie': "list_type":"height"}
    self.encoded_params = urllib.urlencode( self.params )

    def fullinfo(self,url):
        return self.opener.open(url,self.encoded_params).read()

..as you can see, I have tried a couple of things:

• setting the parameter via a header
• setting a cookie

however, these do not seem to render the page in the certain list_order (height) as I would like. I was wondering if someone could point me in the right direction as to how to send the cookie information with urllib2

Thanks.

Answer 1

An easy way to generate a cookie.txt is this chrome extension: https://chrome.google.com/webstore/detail/cookietxt-export/lopabhfecdfhgogdbojmaicoicjekelh

import urllib2, cookielib

url = 'https://example.com/path/default.aspx'
txheaders =  {'User-agent' : 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'}

cj = cookielib.LWPCookieJar()
# cj.load signature: filename=None, ignore_discard=False, ignore_expires=False
cj.load('/path/to/my/cookies.txt') 

opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
urllib2.install_opener(opener)

req = urllib2.Request(url, None, txheaders)
handle = urllib2.urlopen(req)

[update]

Sorry, I was pasting from an old code snippet long forgotten. From the LWPCookieJar docstring:

The LWPCookieJar saves a sequence of "Set-Cookie3" lines. "Set-Cookie3" is the format used by the libwww-perl libary, not known to be compatible with any browser, but which is easy to read and doesn't lose information about RFC 2965 cookies.

So it is not compatible with the cookie.txt generated by modern browsers. If you try to load it with you will get: LoadError: 'cookies.txt' does not look like a Set-Cookie3 (LWP) format file .

You can do as the OP and convert the file:

there is something wrong with the format of the output from chrome extension. I just googled the lwp problem and found: code.activestate.com/recipes/302930-cookielib-example the code spits out the cookie in lwp format and then I follow your steps as it is. - James W

You can also use this Firefox addon , and then "Tools->Export cookies". Make sure the first line in the cookies.txt file is "# Netscape HTTP Cookie File" and use:

cj = cookielib.MozillaCookieJar('/path/to/my/cookies.txt')
cj.load() 

Answer 2

You would better look into the 'request' module for Python making HTTP much easier approachable than through the low-level urllib modules.

See

http://docs.python-requests.org/en/latest/user/quickstart/#cookies