binary encoding for JSON?

Question

My Javascript application is downloading quite a bit of data from the server and I was thinking that in addition to normal gzip that's done by the server, I can encode the data in some binary format rather than textual JSON.
Is there a standard way of doing this?
Ideally it should be some small tool that can take a JSON text file and convert it to a generic binary format and a small Javascript library which decodes it.

Also, is there something special that needs to be done in XHR to pass binary data?

Answer 1

Check out BSON

BSON, short for Binary JSON, is a binary-encoded serialization of JSON-like documents. Like JSON, BSON supports the embedding of documents and arrays within other documents and arrays. BSON also contains extensions that allow representation of data types that are not part of the JSON spec. For example, BSON has a Date type and a BinData type.

Find a good explanation here http://kaijaeger.com/articles/introducing-bison-binary-interchange-standard.html

Answer 2

If gzip doesn't compress well enough, chances are your binary format won't either, especially if you wan't to be able to decode it via javascript within a reasonable amount of time.

Remember that the unzipping when using gzip is done natively by the browser and is orders of magnitude faster than anything you can do in javascript.

If you feel that the JSON deserialization is too slow, because you are supporting older browsers like ie7 which doesn't decode JSON natively but depends on eval for the job, consider going away from JSON to a custom encoding based on string splitting, which is much much faster to deserialize.

For inspiration try to read this article:

http://code.flickr.com/blog/2009/03/18/building-fast-client-side-searches/

Answer 3

MongoDB is using something like that for their document-oriented storage. You can get more details directly on BSON website .
Unfortunately, BSON does not work with Javascript (as you can see from the implementation list ), so I think it is not a good answer to your question.

You could think about using Protocol Buffers ; it has a JS encoder/decoder , but it is still quite experimental.
You may try it - lots of times, experimental open source project are already good enough for usage in specific scenarios.

Note also that there is some questioning about BSON being more compact than JSON; the same could be true also for other protocols like protbuf - I would strongly suggest you doing some math and check it out if there are actual gains.