HTTP data streaming

Question

I've got a backend to be implemented in Python that should stream data to a web browser where the JavaScript is creating the representation (eg continuously updating a variable or drawing to a <canvas> ).

That data will update at a rate of up to 100 Hz (and might as a worst case scenario even be at 1000 Hz...) with perhaps 10 - 20 Bytes each.

So my first thought of using the COMET pattern would produce far too much overhead, I guess.

My next guess were WebSockets. They would be a perfect fit - but being disabled in Firefox makes them unusable for me.

So what is your recommendation to use in this case?

(Requirement: running in a few modern browsers on pure JavaScript, no Flash or Java allowed. Back end in Python. Already used lib is jQuery. Implementation should be easy, preferably using lightweight libs)

Answer 1

The solution I took now is to use the COMET pattern and transport all data that queued up in the backend since the last request. So I'm not polling during times of slow data generation (-> COMET) and I'll only have that amount of connections that the frontend (ie the browser) can handle as it's creating them.

And the overhead is reduced as each request contains a few data points. (You could even say that the overhead is scaled dynamically depending on the data rate. As the data rate gets higher, the overhead sinks...)

Answer 2

As an update to this question, nowadays, you should be able to use Server-sent events. I didn't use XHR due to it keeping the entire response in memory, and didn't use websockets, since I didn't need duplex comms. I had pretty much the same question, answered it here:

How to process streaming HTTP GET data?