How to implement infinite scrolling with React Hooks?

Question

I'm setting up a React web app which requests URLs of images and then displays them in a gallery (like Google Photos). I'm concerned about performance (front-end and back-end), which is why I'm trying to implement infinite scrolling. The app should support switching between galleries of different users as well.

I was able to implement the selection of a user's gallery. On mount, it requests the URLs of the initially selected user and displays the images. When selecting another user, it deletes the current state regarding the now old user sets the new user which triggers the first useEffect hook again.

My issue becomes clear when looking at the code:

const Gallery = () => {
  const [user, setUser] = useState('user1');
  const [stories, setStories] = useState([]);
  const [loadedAll, setLoadedAll] = useState(false);
  const [loading, setLoading] = useState(true);
  const galleryRef = useRef(null);
  const loadedAmount = useRef(0);

  useEffect(() => {
    let didCancel = false;

    async function fetchData() {
      setLoading(true);

      const query = `/api/v1/stories/part/${user}?limit=${20}&skip=${loadedAmount.current}&sortBy=uploadedAt_desc`;
      const result = await axios.get(query);
      const count = result.data.length;

      const transformedStories = result.data.map((story) => ({
        ...story,
        width: story.width || 1080,
        height: story.height || 1920,
      }));

      if (!didCancel) {
        setStories((currentStories) => [
          ...currentStories,
          ...transformedStories,
        ]);
        loadedAmount.current += count;

        if (count < 20) {
          setLoadedAll(true);
        }

        setLoading(false);
      }
    }

    fetchData();

    return () => {
      didCancel = true;
      setStories([]);
      loadedAmount.current = 0;
      setLoadedAll(false);
    };
  }, [user]);

  useEffect(() => {
    const handleScroll = (evt) => {
      const scroller = galleryRef.current;
      const loadTreshold = 500;
      const shouldLoad =
        window.innerHeight + window.scrollY >=
        scroller.clientHeight + scroller.offsetTop - loadTreshold;

      if (shouldLoad) {
        //
        // Load next batch.
        //
      }
    };

    window.addEventListener('scroll', handleScroll);
    return () => window.removeEventListener('scroll', handleScroll);
  }, []);


   return (
      ...
   );
}

How can I refactor my code, so that the next batch of images gets loaded when handleScroll succeeds? handleScroll checks whether the user scrolled far enough for us to request and display the next batch of images.

Thanks for any help, I'm really stuck right now.