Hardware Throttling / Rate limit

Question

My service uses 1 emphasized text hardware resource per request. At any point of time, at-most X (lets say 100) number of such resources are available. What is the best technique I should use?

Point to note:

1. I have a distributed system.
2. I don't have to drop any request.

Answer 1

You could configure a first-in-first-out queue per hardware resource. Every incoming request is assigned to the resource with the shortest queue.

If possible, you could also use a central queue and keep the reqests in this queue until a resource becomes available.

A mixed approach would be to partition the resources in groups and use one queue per group.