Javascript array unshift and pop operation for big dataset

Question

I have a big array used to store numbers (size of 1000 or more), this array values will be used to render the y-axis line graph in real time.

I will constantly unshift this array to add new data and pop to remove the last data in the array. This operation allows my graph to move from left to right.

However, this method of unshift and pop is easy, but it will cause memory fragmentation and time performance when my array size grows and i could have more than 1 graph, (n-th graph and each contain one of the big array). May I know is there any better way to do this operation?

Answer 1

I was curious enough to measure shift/push against the traditional same-memory copying and the results are pretty clear that there's no optimization necessary.

 var p = [], q = []; for (var i = 0; i < 10000; i++) p[i] = i; for (var i = 0; i < 10000; i++) q[i] = i; function rotate_1(a, x) { var len = a.length; for (var i = 1; i < len; i++) a[i - 1] = a[i]; a[len - 1] = x; } function rotate_2(a, x) { a.shift(); a.push(x); } t = new Date() for (var i = 0; i < 100000; i++) rotate_1(p, i); document.write("copy="); document.write(new Date() - t); document.write("<br>"); t = new Date() for (var i = 0; i < 100000; i++) rotate_2(q, i); document.write("shift/push="); document.write(new Date() - t); document.write("<br>"); 

Answer 2

There are two sides of this problem:

Is there any better way to do this operation?

It strongly depends on how do you draw the graph. Do you use some kind of library for that? If so, it probably expects a specific data (eg. Array) so there isn't much you can do about it. On the other hand, if you're implementing the drawing part by yourself then you can use any data structure you want (including lists which could be suitable here).

Memory fragmentation and Array performance

First of all ask yourself a question: Do you actually have any performance problems with Array operations or is this just theoretical?

Drawing the graph is far more time-consuming than any array operations. I think you shouldn't worry about Array methods' performance unless you actually have any problems with them. Mind that in JS engines Array s are not necessarily implemented as arrays . They might be binary search trees or even something else, depending on how you use them and therefore it's hard to predict which optimisations will work for your case and which will make things worse.

Is there any way to increase performance?

Yes. But it depends on how do you actually use the array. I can imagine if you draw a live graph and unshift / pop data eg. every 1 second, you know exactly the length of the array, is that correct? It should remain constant. Therefore one of the things you could do is, at the beginning, allocate memory for an array of that length. Your memory shouldn't get fragmented if you do unshift + pop in the future because that's an easy case to optimise by the engine. But then again, it's just guessing, because JavaScript engines are smarter than I am . More hints and tricks are described in the article Let's get those Javascript Arrays to work fast .

Don't try to fix something that's not broken. Measure before you start refactoring and don't overthink. This is called premature optimisation and I'm positive that Array operations won't be any priority to optimise when you actually have the application running.