conditionally active rxjs observable rate limiter

Question

I want to create a rate limiter for an observable that conditionally changes the way that it limits a value. The use case is for a downloader that is continually receiving new urls to download. I want to be able to queue up incoming urls and switch between two methods of queueing. The two methods are by limiting a rate (eg no more than 10 requests every 2 seconds) and by number of concurrent requests (eg no more than 10 requests can be fired at a time).

A simple rate limiter can be implemented like below (borrowed from here ):

const rateLimit = (limit, rate, scheduler = asyncScheduler) => {
  let tokens = limit
  const tokenChanged = new BehaviorSubject(tokens)
  const consumeToken = () => tokenChanged.next(--tokens)
  const renewToken = () => tokenChanged.next(++tokens)
  const availableTokens = tokenChanged.pipe(filter(() => tokens > 0))
  return source =>
    source.pipe(
      mergeMap(val =>
        availableTokens.pipe(
          take(1),
          map(() => {
            consumeToken()
            timer(rate, scheduler).subscribe(renewToken)
            return val
          })
        )
      )
    )
}
const o = urlSource.pipe(
  rateLimit(10, 2000),
  mergeMap(downloadUrl)
)
o.toPromise()

And this is a simple concurrent limiter:

const o = urlSource.pipe(
  mergeMap(downloadUrl, maxConcurrent)
)
o.toPromise()

Finally, I can create this combined switcher to choose which type of limiter to use:

const toggleableLimiter = (func, limit, rate, concurrent, toggleObservable) => {
  let useRateLimiter = true
  toggleObservable.subscribe(() => (useRateLimiter = !useRateLimiter))
  const rateLimiter = rateLimit(limit, rate)
  return source => {
    const operators = useRateLimiter
      ? [rateLimiter, mergeMap(func)]
      : [mergeMap(func, concurrent)]
    return source.pipe(...operators)
  }
}

const e = new EventEmitter()
const toggler = fromEvent(e, 'toggle')
const o = urlSource.pipe(
    toggleableLimiter(downloadUrl, 2, 1000, 2, toggler)
)
o.toPromise()
// using rate limiter
e.emit('toggle')
// incoming values now use concurrent limiter

This all works as a decent solution to my problem. I can toggle between two methods using an event emitter. The issue however, is that whatever has been passed to the toggleableLimiter before the event is emitted, will have to adhere to that limiter operator. What I want to know, is if I can conditionally keep values in a queue and choose how to limit the queued values on a whim.

Answer 1

Ok! I have a solution, unfortunately it involves handling the queue myself. I believe this is a necessity due to the nature of observables and backpressure, I found a lot of discussion on the topic in this issue . In the past this could have been handled more simply with the controlled operator, but it is deprecated . Instead I simply used a timer observable and wrapped mergeMap 's concurrency control myself (though mergeMap still manages concurrency internally as a safety measure.

const rateLimitToggle = (func, limit, rate, maxConcurrent, toggler) => {
  const rateTimer = Rx.timer(0, rate).pipe(ops.mapTo(true))
  return source =>
    new Rx.Observable(subscriber => {
      const concurrentLimiter = new Rx.Subject()
      // stateful vars
      const queue = []
      let inProgress = 0
      let closed = false

      const enqueue = val => {
        queue.push(val)
        concurrentLimiter.next()
      }
      const dequeue = useRateLimit => {
        const availableSlots = useRateLimit ? limit : maxConcurrent - inProgress
        const numberToDequeue = Math.min(availableSlots, queue.length)
        const nextVals = queue.splice(0, numberToDequeue)
        inProgress += availableSlots
        return nextVals
      }

      Rx.merge(Rx.of(true), toggler)
        .pipe(
          ops.switchMap(useRateLimiter => (useRateLimiter ? rateTimer : concurrentLimiter)),
          ops.takeWhile(() => !closed || queue.length),
          ops.mergeMap(dequeue),
          ops.mergeMap(val => func(val), maxConcurrent)
        )
        .subscribe(val => {
          inProgress--
          concurrentLimiter.next()
          subscriber.next(val)
        })

      source.subscribe({
        next(val) {
          enqueue(val)
        },
        complete() {
          closed = true
        }
      })
    })
}

example usage:

const timeout = n => val => {
  console.log('started', val)
  return new Promise(resolve => setTimeout(() => resolve(val), n))
}

const emitter = new EventEmitter()
const toggler = Rx.fromEvent(emitter, 'useRateLimiter')
const downloader = Rx.range(0, 10).pipe(
  rateLimitToggle(timeout(1000), 2, 1000, 10, toggler)
)

downloader.subscribe(val => console.log('finished', val))
setTimeout(() => {
  console.log('now use concurrentLimiter')
  emitter.emit('useRateLimiter', false)
}, 2000)
/* outputs:

  started 0
  started 1
  started 2
  started 3 // 0 - 3 all executed under rateLimiter
  finished 0
  finished 1
  now use concurrentLimiter
  started 4 // 4 - 9 executed under concurrentLimiter
  started 5
  started 6
  started 7
  started 8
  started 9
  finished 2
  finished 3
  finished 4
  finished 5
  finished 6
  finished 7
  finished 8
  finished 9
*/