当大量文档从 Firestore 快照到达时，如何不让 ReactJS 应用程序冻结？

Question

I've a ReactJS application made up of 3 simple elements: 2 numbers and an input text. This is a test page of course.
The 2 numbers refers to the number of elements that are present in 2 separate Firestore collections, and the input text doesn't do anything.
The goal of my experiment is to understand why if I write a huge amount of docs in a short time in the collections (I'm using k6 in order to) and just print those two numbers by listening to Firestore snapshots, my page completely freezes until all the messages have arrived.

Here is my code

const App = () => {
  const [col1, setCol1] = useState(0);
  const [col2, setCol2] = useState(0);
  const [value, setValue] = useState('');
  useEffect(() => {
    console.debug('*** EFFECT');

    db
      .collection('abcd/efgh/collection1')
      .orderBy('timestamp')
      .startAt(new Date())
      .limitToLast(1)
      .onSnapshot(({ docs }) => {
        setCol1((e) => e + 1);
      });

    db
      .collection('qwer/tyui/collection2')
      .orderBy('timestamp')
      .onSnapshot(({ docs }) => {
        console.debug('***', docs.length);
        setCol2(docs.length);
      });
  }, []);

  return (
    <>
      {exist}

      <br />

      {messages}

      <br />

      <Input onChange={({ target: { value } }) => setValue(value)} value={value} />
    </>
  );
};

Of course the final target is to understand how to not cause the freeze.
From this I see that spawning a separate thread will help, so the equivalent in JS desktop are WebWorkers.
Could that be a right idea? Do you have any other suggestion?

Answer 1

As per my understanding, I don't think snapshot is the right use here if you are just trying to calculate the document counts. The onSnapshot() method is mainly for real time updates use cases. Each time the contents change/add/delete, another call updates the snapshot. So with a huge amount of documents, the snapshots are frequently updating until everything is caught up. You can review this page for some general onSnapshot use cases. you can limit the write rate, and exceeding the limit can cause contention and latency issues .

If you are just trying to do a bulk write update, and after that you would like to know how many documents are in the collection, then you can do the updates first, and use get() to retrieve the count of documents instead of onSnapshot to constantly get the update. There are many ways to retrieve the count of documents, for example get() method or using a counter to keep track of the count. third-party post .

Answer 2

I had basically the exact same issue, so I spent a bunch of time testing this myself, and here's what I found.

Debouncing State Updates: Helps a little .

Basically, if you're constantly re-rendering large amounts of data, you can use a lot of memory and/or drop frames, so I assumed that debouncing the function would help.

It did help a bit but the app would still freeze or crash if more than a few hundred rows were updated.

Here's how I implemented:

const debounce = (func, wait) => {
    let timeout;

    return function executedFunction(...args) {
        const later = () => {
            clearTimeout(timeout);
            func(...args);
        };

        clearTimeout(timeout);
        timeout = setTimeout(later, wait);
    };
};

...

    const debouncedSetItems = useMemo(() =>
        debounce((newItems) => {
            setCol1(newItems);
        }, 5000), []
    );

...

    useEffect(() => {
        const unsubscribe = firestore.collection("myCollection").onSnapshot((querySnapshot) => {
            const newItems = [];
            querySnapshot.forEach((doc) => {
                newItems.push({
                    id: doc.id,
                    ...doc.data()
                });
            });
    
            debouncedSetItems(newItems);

        });
 }, [debouncedSetItems]);

Slowing the Write Speed : Helps a lot; has limitations.

Essentially, if you can pause every once in a while when writing large amounts of data, the app won't hang.

I was doing large numbers of writes via a cloud function, so what I did was pause for a few seconds every 50 writes:

async function sleep(milliseconds) {
    return new Promise((resolve) => setTimeout(resolve, milliseconds));
}

...

if (i > 0 && i % 50 === 0) {
    console.log("Sleeping for 5 seconds");
    await sleep(5000);
}

This totally fixed the issue, but it has the obvious limitation of slowing your writes down, which may or may not be an issue. For me, it wasn't a huge deal because I still had plenty of buffer leftover in the 9-minute cloud function runtime limit, but your mileage may vary.

Curious to see how others have solved this.