Get Request Using PyCurl after logging into website

Question

After doing a post to log into my website, I try to do a get on my the site and I get a bunch of garbage " 0 `&) 붋...." instead of the data from my site. Why is that? How do I fix that?

Answer 1

Obviously a dead thread, but if anyone else stumbles across this, funky data like that is most likely compressed with zlib or gzip. If you are using pycurl, this should do the trick:

import pycurl

ch = pycurl.Curl()
ch.setopt(pycurl.URL, 'http://example.com')
ch.setopt(pycurl.ENCODING, '')
ch.perform()

Setting the ENCODING option to an empty string sets the 'Accept-Encoding' headers to all encodings supported by libcurl and tells libcurl to decode the response data. OP was probably setting the headers manually and libcurl wasn't expecting encoded data.