JavaScript Zlib解压缩

Question

I'm trying to decompress zlib'ed XML such as the following: https://drive.google.com/file/d/0B52P0MZLTdw8ZzQwQzVpZGZVZWc

Uploading to online decompress services works, such as: http://i-tools.org/gzip

In PHP, I'm using this code and works just fine, I get the XML string:

$raw = file_get_contents("file_here");
$uncompressed = zlib_decode($raw);

However, I want to do this in JavaScript.

• The app is a client-side Chrome extension which uses chrome.devtools.network that reads from the network logs
• Reads binary responses. Example at Google Drive link at the top
• JS needs to decompress that response to its original XML and parsed afterwards into object

The only problem I have is the zlib decompress part.

As of Latest Update, the Decompression Libraries work but unpacking doesn't. Please skip to the Update Sept 16 at the bottom.

---

I have already tried several JavaScript libraries and still cannot make it work:

Pako : https://github.com/nodeca/pako

unpack() code: https://codereview.stackexchange.com/questions/3569/pack-and-unpack-bytes-to-strings

function unpack(str) {
    var bytes = [];
    for(var i = 0, n = str.length; i < n; i++) {
        var char = str.charCodeAt(i);
        bytes.push(char >>> 8, char & 0xFF);
    }
    return bytes;
}

$.get("file_here", function(response){
    var charData    = unpack(response);
    var binData     = new Uint8Array(charData);
    var data        = pako.inflate(binData);
    var strData     = String.fromCharCode.apply(null, new Uint16Array(data));
    console.log(strData);
});

Error: Uncaught incorrect header check

It's the same even placing the response elsewhere:

• new Uint8Array(response);
• pako.inflate(response);

---

Imaya's zlib : https://github.com/imaya/zlib.js

$.get("file_here", function(response){
    var inflate = new Zlib.Inflate(response);
    var output = inflate.decompress();
    console.log(output);
});

Error: Uncaught Error: unsupported compression method inflate.js:60

Still using Imaya's zlib, combining with this Stack Overflow question: Decompress gzip and zlib string in javascript

$.get("file_here", function(response){
    var response = response.split('').map(function(e) {
        return e.charCodeAt(0);
    });
    var inflate = new Zlib.Inflate(response);
    var output = inflate.decompress();
    console.log(output);
});

Error: Uncaught Error: invalid fcheck flag:29 inflate.js:65

---

dankogai's js-deflate : https://github.com/dankogai/js-deflate

console.log(RawDeflate.inflate(response));

Output: empty

---

augustl's js-inflate : https://github.com/augustl/js-inflate

console.log(JSInflate.inflate(response));

Output: empty

---

zlib-browserify : https://github.com/brianloveswords/zlib-browserify

Error: ReferenceError: exports is not defined

This is just a wrapper for Imaya's zlib. I think this is requireJS ? I'm not even sure how to use it. Can it even be used without installing anything and just jQuery/JS? The app as mentioned is downloadable Chrome extension with just HTML importing JS files.

---

UPDATE Sept 16, 2014

It seems the problem is with the JavaScript unpack( ) function. When I use the ByteArray generated by PHP: http://pastebin.com/uDWvK94B , the JavaScript decompression functions work.

PHP unpacking that works:

$unpacked = unpack("C*", $raw);

For the JavaScript unpack( ) code that I use, which doesn't work, see top of the post under Pako section.

So the new question is, why does JavaScript generate a different ByteArray values than the one generated by PHP.

• Is it really a problem with the unpack( ) function?
• or is it something when the JS fetches the file, the encoding or whatsoever changes thus bytes get messed up?
• and lastly, what is your suggested fix?

UPDATE Sept 20, 2014

With more research and some of the answers here giving leads

• Sebastian S opening the idea that the problem was in the manner of retrieving data and it had something to do with text encodings
• user3995789 providing an example that it will work even without the unpack( ) function, though outside the context of Chrome extensions
• Isaac providing examples in the context of Chrome extensions, but still does not work

With that I researched further combining all leads which lead me to a theory that the reason behind all this is that Chrome is unable to get "raw" data through its request.getContent function. See here for the Chrome documentation for the said function.

As of now, I have taken the issue to Chrome, see here .

UPDATE March 24, 2015

Although the problem was not fully resolved, the answer which I think was the most useful to me was from @Sebastian S, who proposed that "the way" I was taking or receiving the data was at fault and a bad conversion was the cause, which is as near as the problem was.

Answer 1

Jquery reads in utf8 format, you have to read the raw file, this function will work.

function readTextFile(file)
{
    var rawFile = new XMLHttpRequest();
    rawFile.open('GET', file, true);  
    rawFile.responseType = 'arraybuffer';
    rawFile.onload = function (response)
    {
      var words = new Uint8Array(rawFile.response);
       console.log(words[1]);
      console.log(pako.ungzip(words));

    };
    rawFile.send();
}

For more information see this answer .

Answer 2

I understood that you wanna use the zlib decompression inside a chrome extension while reading reponses bodies from the network log.

You need first to retrieve the base64 who will be decompressed. You can achieve this while using the getContent method.

function zlibDecompress(base64Content){
    // var base64Content        = base64Content.split(',')[1]; // Not sure if need to keep it

    // Decode base64 (convert ascii to binary)
    var strData     = atob(base64Content);

    // Convert binary string to character-number array
    var charData    = strData.split('').map(function(x){return x.charCodeAt(0);});

    // Turn number array into byte-array
    var binData     = new Uint8Array(charData);

    // Pako inflate
    var data        = pako.inflate(binData, { to: 'string' });

    return data;

}

chrome.devtools.network.onRequestFinished.addListener(
    function(request) {
        request.getContent(
            function(content, encoding){
                if(encoding == 'base64'){
                    var output = zlibDecompress(content);
                }
            }
        );
    }
);

https://developer.chrome.com/extensions/devtools_network#type-Request

Using XMLHttpRequest :

<script type="text/javascript" src="pako.js"></script>
<script type="text/javascript">

function zlibDecompress(url){
    var xhr = new XMLHttpRequest();
    xhr.open('GET', url, true);
    xhr.responseType = 'blob';

    xhr.onload = function(oEvent) {
        // Base64 encode
        var reader = new window.FileReader();
        reader.readAsDataURL(xhr.response); 
        reader.onloadend = function() {
            base64data      = reader.result;  
            var base64      = base64data.split(',')[1];

            // Decode base64 (convert ascii to binary)
            var strData     = atob(base64);

            // Convert binary string to character-number array
            var charData    = strData.split('').map(function(x){return x.charCodeAt(0);});

            // Turn number array into byte-array
            var binData     = new Uint8Array(charData);

            // Pako inflate
            var data        = pako.inflate(binData, { to: 'string' });

            console.log(data);
        }
    };

    xhr.send();
}
zlibDecompress('fileurl');
</script>

If you wanna use XMLHttpRequest with chrome extension

{
  "name": "My extension",
  ...
  "permissions": [
    "http://www.domain.com/", // The domain that hold the file
    "http://*/" // Or every domain
  ],
  ...
}

https://developer.chrome.com/extensions/xhr

Feel free to ask if you have any questions ;)

Answer 3

In my opinion the question you should really be asking is: How do you retrieve the compressed data? As soon as it becomes an UTF-16 string, the trouble begins. I'm not even sure, if the conversion from raw byte data to javascript strings is lossless.

As you wrote something about php, I assume you're communicating to some sort of backend. If this is true, there are options to handle binary data with native means. Maybe this can help you: https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/Sending_and_Receiving_Binary_Data