处理大量数字时功能会变慢

Question

I've wrote a script that instead of giving the real average of a set of data returns a windows that contains most data points. Let me show some code:

time.tic()
var selectedAverage = 0;
var highestPointCount = 0;
for (var i = 1; (i*step) <= maxValue; i++) {
    var dataPointCount = 0;
    for (var j = 0; j < myArray.length; j++) {
        if (myArray[j] >= minValue+(i-1)*step && myArray[j] <= minValue+i*step) {
            dataPointCount++;
        }
    }
    if (dataPointCount > highestPointCount) {
        highestPointCount = dataPointCount;
        selectedAverage = (minValue+(i-1)*step)+Math.round(0.5*step);
    }
}
console.log(time.toct().ms) 
return selectedAverage;

First the step value is calculated by subtracting the minimum value from the maximum value and then deciding by 10. So there are 10 'horizontal' windows. Then the script counts the amount of datapoint within each window and returns a appropriate average.

It appears however that script slows down extremely (sometimes more than 200 times) when an array of larger numbers is passed in (1.000.000 for example). Array lengths are roughly 200 but always the same length so it must be associated with the actual values. Any idea where it is going wrong?

EDIT: The code to get the step value:

var minValue = myArray.min();
var maxValue = myArray.max();
var step = Math.round((maxValue-minValue)/10);
if (step === 0) {
    step = 1
}

The .min() and .max() are prototypes attached to Array. But this all goes very fast. I've measured every step and it is the for loop that slows down.

Answer 1

There are 2 different things I think of your issue:

1. Removed unnecessary / repeated calculation

Inside your nested code you have minValue+(i-1)*step and minValue+i*step calculated everytime for the same value of minValue , i and step . You should pull it up before the 2nd for-loop where it becomes:

var dataPointCount = 0;
var lowerLimit = minValue+(i-1)*step;
var higherLimit = minValue+1*step;
for (var j = 0; j < myArray.length; j++) {
    if (myArray[j] >= lowerLimit && myArray[j] <= higherLimit) {
        dataPointCount++;
    }
}

2. You got severe performance hit when you are handling big data array are likely caused by memory swapping. From your point of view you are dealing with a single array instance, however when you have such a big array it is unlikely the JavaScript VM has access to consecutive memory space to hold all those values. It is very likely JavaScript VM has to juggle multiple memory blocks that it gets from the operating system and have to spend extra effort to translate which value is where during reading/writing.

Answer 2

If I understood your algorithm correctly, this should remove all unnecesary calculations and be much faster:

var arr = [];
var maxQty=0;
var wantedAverage = 0;
for (var j = 0; j < 11; j++) {
    arr[j]=0;
}
for (var j = 0; j < myArray.length; j++) {
    var stepIndex = Math.floor((myArray[j]-minValue)/step)
    arr[stepIndex]+=1;

    if(arr[stepIndex] > maxQty ){
        wantedAverage = minValue + stepIndex*step +Math.round(0.5*step);
        maxQty = arr[stepIndex]
    }
}
console.log(maxQty, wantedAverage)

We just iterate over each element of the array only once, and calculate the index of the window it belongs to, adding one to the quantity array. Then we update the wantedAverage if we have a bigger amount of points in window found