在 Javascript 中处理大型 arrays 的最佳方法是什么？

Question

I have a React JS app that uses an API to pull a JSON object that contains an array of 10,000+ objects. This array is then used to populate a table, with certain filters and options from checkboxes, dropdowns, that can manipulate the data. When tapping a checkbox, filter, sort, reduce functions are used on the array to return a specific subset of the array that can populate the table again.

There are 10-15 options to choose from, so 10-15 filter/map/reduce functions running on the data each time a box is checked.

These filtering options now cause a noticeable lag between clicking on the checkbox and changing the table. The app freezes while it calculates the new array. Is there a more efficient flow to filter my data?

Some example functions below below:

//gameData is an array of 10k+ objects
let inData = gameData

const options = {
    dateToTime: new Date('2020-03-01'),
    servers:[1,2,3],
    maps:['A','B','C']
}

function groupByArray(array, key) {
    return array.reduce(function (rv, x) {
        let v = key instanceof Function ? key(x) : x[key];
        let el = rv.find((r) => r && r.key === v);
        if (el) {
            el.values.push(x);
        } else {
            rv.push({ key: v, values: [x] });
        } return rv;
    }, []);
}

const gamesGrouped = groupByArray(inData, 'gameid')
inData = gamesGrouped.filter(a => a.playername != "new")
inData = inData.filter(game => {
    const thisTime = new Date(game.creationtime)
    return (thisTime < options.dateToTime)
})
inData = inData.filter(game => options.servers.includes(game.serverip))
inData.filter(game => options.maps.includes(game.map))

Thanks in advance!

Answer 1

I would say it is impossible to give general answer how to process array data, but I can give some pointers.

• be careful when nesting loops (to avoid repeating same iterations)
• avoid overheads, eg. find() can be replaced with for loop which is quite faster (I know it is easier to write find(), but you are looking at roughly 30% performance increase by switching it to for loop)
• paginate - you can process array in chunks using generators (eg. if you need to show only first 10 results, that would be faster then processing all of them)

Also code that you provided is bit cryptic, you might want to use better naming.

Here is performance comparison for groupByArray() function: https://jsbench.me/t7kltjm1sy/1

Worth noting, whenever I deal with performance sensitive situations I keep code to the VanillaJS as close as possible, because with large data sets even slight function overhead can be noticeable.