Handling Media Playback State in React

Question

Curently using expo + react native and encountering challenges handling media playback state efficiently. With expo's AV libraries we're given an imperative interface around a playback resource, with a parameter to hook into playback state updates:

// this runs every 50ms (or faster) during active playback
const handlePlaybackStatusUpdate = (nextStatus: AVPlaybackStatus) => {
    // do something in here
}

const loadedSound = (await Audio.Sound.createAsync({ uri }, playbackConfig, handlePlaybackStatusUpdate)).sound

// ...

await loadedSound.playAsync()

Understanding that 'high performance' is caveated by limitations around thread communication in RN apps ,

My implementation goals are:

1. Components relying on playback status/functionality should be high performance (not be constrained by playback based rerenders/updates)
2. Playback functionality should be consistent, ideally shareable between consumers at various points within the hierarchy (eg. a progress bar and a play pause overlay)
3. Components which need a higher rate of playback updates (eg. playback progress bar) should be able to opt-in to more frequent updates without impacting other consumers

I've been struggling to find a way to efficiently process these playback status updates. For further context. Playback status exposes many dimensions of playback state - isPlaying, isBuffering, didJustFinish, durationMillis (length of playback in ms), positionMillis (current position within playback) etc. Based on experimentation and observation, positionMillis is the only value that changes in the vast majority of updates (barring those related to commands like playAsync, pauseAsync etc.) Hence the below attempts to expose a stream of updates based on measuring position, independent from the rest of the context's state.

A few things I've tried:

• Coupling these updates with component state and sharing via context. This understandably yields tons of rerenders to the point where touch events seemingly aren't being broadcasted or handled.

const PlaybackProvider = ({ children }: { children: ReactNode }) => {
    const [playbackStatus, setPlaybackStatus] = useState<PlaybackStatus>()
    // ...
    await Audio.Sound.createAsync({ uri }, playbackConfig, setPlaybackStatus)

• Attaching a ref and polling it according to consumer's preference. This approach smells. Maybe workable for a minimal, deep child component that only consumes, but seems untenable for achieving an 'updating ref' that's not powered by a DOM node + callback ref.

const usePlaybackPosition = (
    onUpdate: (position: number, duration: number) => Promise<void> | void,
    interval = 200
) => {
    const { playbackStatusRef } = usePlayback()
    const [timer, setTimer] = useState(0)
    useInterval(() => {
        const { positionMillis, durationMillis } = playbackStatusRef?.current ?? {}
        onUpdate(positionMillis, durationMillis)
        setTimer(timer + 1)
    }, interval)
}

• Attempting to set context state intelligently within the onPlaybackStatusUpdate hook. This seems to queue up TONS of stale setPlaybackStatus invocations/generally not behaving as expected. Having a hard time reasoning about why this is happening.

const handlePlaybackStatusUpdate = (newStatus?: AVPlaybackStatus): void => {
    // save a ref for components that want to listen to position
    playbackStatusRef.current = newStatus
    
    if (newStatus === undefined) {
        setPlaybackStatus(PlaybackStatus.None)
        return
    }

    if (!newStatus.isLoaded) {
        if (newStatus.error) {
            console.log(`Encountered a fatal error during playback: ${newStatus.error}`)
            setPlaybackStatus(PlaybackStatus.Error)
            return
        }
    } else {
        const { isPlaying, isBuffering, positionMillis, didJustFinish, isLooping } = newStatus

        let nextStatus: PlaybackStatus = PlaybackStatus.Idle

        if (isPlaying) {
            nextStatus = PlaybackStatus.Playing
        } else if (positionMillis > 0) {
            nextStatus = PlaybackStatus.Paused
        } else {
            // not playing and position is 0
            nextStatus = PlaybackStatus.Idle
        }

      if (didJustFinish && !isLooping) {
          nextStatus = PlaybackStatus.Finished
      }

      if (nextStatus !== playbackStatus) {
          // this conditional ends up running TONS of times before playbackStatus actually changes
          setPlaybackStatus(nextStatus)
      }
    }
  }

I am hoping to keep a solution simple and orthodox re: react best practices. Although the React docssuggest that media playback is a valid use of refs, so maybe just need to commit to and harden a ref-based interface. My hope is that smarter composition/refactoring of the component hierarchy + hooks/context to replicate/share behavior will be sufficient.

There are likely some incorrect assumptions above. I would be grateful to have these pointed out. Likewise, any and all insights regarding known limitations/issues with media playback in expo/react-native are greatly appreciated. Thank you for the help!

Answer 1

One optimization we did was to debounce the onPlaybackStatusUpdate hook. Only process updates every n milliseconds. Sharing piece of the code for reference (not the full implementation):

const debouncedPlaybackStatusUpdate = useMemo(
        () => debounce(onPlaybackStatusUpdate, 100),
        [],
    );

async function playSound(status: boolean) {
        if (!status) {
            await soundObj.playAsync();
            soundObj.setOnPlaybackStatusUpdate(debouncedPlaybackStatusUpdate);
        } else {
            await soundObj.pauseAsync();
        }
    }

This should reduce majority of re-renders.