当您有 10 万到一百万个对象/索引时，维护脏对象或索引列表的有效方法是什么？

Question

I have to iterate over +100k objects in real-time (for a video game).

To make things faster, I implemented a HashSet<int> DirtyIndexes and whenever an object is modified, I do DirtyIndexes.Add(index) and iterate over those indexes only.

Unfortunately HashSet.Add() 's duplicate check is kinda slow and, at 10k+ objects per frame, the process of adding that many indexes kills any advantage of using this system.

I was hoping to find a collection that allowed multiple objects to share a key and would return an array of everything that matches the search key, but couldn't find anything.

Another idea is to use an int array of the same size as the number of objects and set 0 for clean, 1 for dirty, but then I have to add several Ifs to my code to filter out indexes, which would also slow things down.

I could also add the dirty objects to a DirtyObject list, but then I once again have to prevent duplicates and end up with the same problem.

Does anyone have a solution to this?

Answer 1

For efficiency its better to use an array than a list. If you want to use the features of the List class but to use the array when needed, you can use a class that has an array and a list as class variables, and update the array only when the list is changed, like this:

using System;
using System.Collections.Generic;

public class CustomList<T>
{
    //private
    bool _isDirty;
    List<T> _list;
    T[] _array;

    //public
    T[] getArray() {
        if (_isDirty)
            _array = _list.ToArray();
        return _array;
    }

    //handler methods
    public CustomList() {
        _list = new List<T>();
    }

    public void Add(T item) {
        _list.Add(item);
        _isDirty = true;
    }

    public void Remove(T item) {
        _list.Remove(item);
        _isDirty = true;
    }

    public void RemoveAt(int i) {
        _list.RemoveAt(i);
        _isDirty = true;
    }

    public bool Contains(T item) {
        return _list.Contains(item);
    }

    public int Count() {
        return _list.Count;
    }

    public T this[int index] {
        get { return _list[index]; }
        set { _list[index] = value; }
    }

    public void Clear() {
        _list.Clear();
        _isDirty = true;
    }

    public void RemoveAll(Predicate<T> predicate) {
        _list.RemoveAll(predicate);
        _isDirty = true;
    }

    public T Find(Predicate<T> predicate) {
        return _list.Find(predicate);
    }

    public void Insert(int index, T item) {
        _list.Insert(index, item);
        _isDirty = true;
    }

    public void ForEach(Action<T> action) {
        _list.ForEach(action);
    }
}

Answer 2

You might as well use a List<int> for DirtyObjects , since List<T> is essentially just an array with quality of life features on top of it, and you can iterate over it very quickly.

As for preventing duplicates, you could maintain a bool isDirty flag in each object and only add an object to the list if its flag isn't already set:

private void OnChanged() {
    if (!this.isDirty) {
        this.isDirty = true;
        DirtyObjects.Add(this.index);
    }
}

This would prevent any object from being added to DirtyObjects more than once. (And it doesn't have to do any kind of lookup in the process!) This logic also doesn't need to be particularly fast, since it's only executed whenever a clean object becomes dirty.

Then whatever code eventually "cleans up" the dirty objects can set their isDirty flags back to false , and the cycle continues.

Answer 3

You guys made excellent suggestions, but running with this many entities, I had to come up with something else.

The secret is to use 2 arrays. One for the value and the second one for its dirty flag (0 or 1). This prevents duplicates (doesn't matter if the value is set dirty multiple times) and is much faster. Both arrays are initialized ahead of time to my pool size.

The idea is to use a set of array like this for every value I want to iterate through. Doing it this way allows processing the data at different rates while maintaining compatibility with Unity's Job system.

Burst is amazing, Vectorization, you are next!