Time step in reinforcement learning

Question

For my first project in reinforcement learning I'm trying to train an agent to play a real time game. This means that the environment constantly moves and makes changes, so the agent needs to be precise about its timing. In order to have a correct sequence, I figured the agent will have to work in certain frequency. By that I mean if the agent has 10Hz frequency, it will have to take inputs every 0.1 secs and make a decision. However, I couldn't find any sources on this problem/matter, but it's probably due to not using correct terminology on my searches. Is this a valid way to approach this matter? If so, what can I use? I'm working with python3 in windows (the game is only ran in windows), are there any libraries that could be used? I'm guessing time.sleep() is not a viable way out, since it isn't very precise (when using high frequencies) and since it just freezes the agent.

EDIT: So my main questions are:

a) Should I use a certain frequency, is this a normal way to operate a reinforcement learning agent?

b) If so what libraries do you suggest?

Answer 1

There isn't a clear answer to this question, as it is influenced by a variety of factors, such as inference time for your model, maximum accepted control rate by the environment and required control rate to solve the environment.

As you are trying to play a game, I am assuming that your eventual goal might be to compare the performance of the agent with the performance of a human. If so, a good approach would be to select a control rate that is similar to what humans might use in the same game, which is most likely lower than 10 Hertz.

You could try to measure how many actions you use when playing to get a good estimate,

However, any reasonable frequency, such as the 10Hz you suggested, should be a good starting point to begin working on your agent.