Decompress Python requests response with zlib

Question

I'm trying to decompress the response from a web request using Python requests and zlib but I'm not able to decompress the response content properly. Here's my code:

import requests
import zlib

URL = "http://" #omitted real url
r = requests.get(URL)
print r.content
data = zlib.decompress(r.content, lib.MAX_WBITS)
print data

However, I keep getting various errors when changing the wbits parameter.

zlib.error: Error -3 while decompressing data: incorrect header check
zlib.error: Error -3 while decompressing data: invalid stored block lengths

I tried the wbits parameters for deflate, zlip and gzip as noted here zlib.error: Error -3 while decompressing: incorrect header check

But still can't get pass these errors. I'm trying to this in Python, I was given this piece of code that did it with Objective-C but I don't know Objective-C

#import "GTMNSData+zlib.h"
+ (NSData*) uncompress: (NSData*) data
{
    Byte *bytes= (Byte*)[data bytes];
    NSInteger length=[data length];
    NSMutableData* retdata=[[NSMutableData alloc]   initWithCapacity:length*3.5];

    NSInteger bSize=0;
    NSInteger offSet=0;
    while (true) {
        offSet+=bSize;
        if (offSet>=length) {
            break;
        }
        bSize=bytes[offSet];
        bSize+=(bytes[offSet+1]<<8);
        bSize+=(bytes[offSet+2]<<16);
        bSize+=(bytes[offSet+3]<<24);
        offSet+=4;
        if ((bSize==0)||(bSize+offSet>length)) {
            LogError(@"Invalid");
            return data;
        }
        [retdata appendData:[NSData gtm_dataByInflatingBytes: bytes+offSet length:bSize]];
    }
    return retdata; 
}

Answer 1

According to Python requests documentation at:

• http://docs.python-requests.org/en/master/user/quickstart/#binary-response-content

it says:

You can also access the response body as bytes, for non-text requests:

>>> r.content
b'[{"repository":{"open_issues":0,"url":"https://github.com/...

The gzip and deflate transfer-encodings are automatically decoded for you.

If requests understand the encoding, it should therefore already be uncompressed.

Use r.raw if you need to get access to the original data to handle a different decompression mechanism.

• http://docs.python-requests.org/en/master/user/quickstart/#raw-response-content

Answer 2

The following is an untested translation of the Objective-C code:

import zlib
import struct

def uncompress(data):
    length = len(data)
    ret = []
    bSize = 0
    offSet = 0
    while True:
        offSet += bSize
        if offSet >= length:
            break

        bSize = struct.unpack("<i", data[offSet:offSet+4])
        offSet += 4
        if bSize == 0 or bSize + offSet > length:
           print "Invalid"
           return ''.join(ret)
        ret.append(zlib.decompress(data[offSet:offSet+bSize]))

   return ''.join(ret)