多智能体强化学习环境 公共交通问题

Question

For my Msc thesis I want to apply multi-agent RL to a bus control problem. The idea is that the busses operate on a given line, but without a timetable. The busses should have bus stops where passengers accumulate over time and pick them up, the longer the interval between busses, the more passengers will be waiting at the stop (on average, it's a stochastic process). I also want to implement some intersections where busses will have to wait for a green light.

I'm not sure yet what my reward function will look like, but it will be something along the lines of keeping the intervals between busses as regular as possible or minimising total travel time of the passengers.

The agents in the problem will be the busses, but also the traffic lights. The traffic lights can choose when to show green light for which road: apart from the busses they will have other demand as well that has to be processed. The busses can choose to speed up, slow down, to wait longer at a stop or to continue on normal speed.

To be able to put this problem in a RL framework I will need an enviroment and suitable RL algorithms. Ideally I would have a flexible simulation environment to re-create my case study bus line and connect this to of-the-shelf RL algorithms. However, so far I haven't found this. This means I may have to connect a simulation environment to something like an OpenAI gym myself.

Does anyone have advice for which simulation environment may be suitable? And if it's possible to connect this to of-the-shelf RL algorithms?

I feel most comfortable with programming in Python, but other languages are an option as well (but this would mean considerable extra effort from my side).

So far I have found the following simulation environments that may be suitable:

NetLogo
SimPy
Mesa
MATSim ( https://www.matsim.org )
Matlab
CityFlow ( https://cityflow-project.github.io/#about )
Flatland ( https://www.aicrowd.com/challenges/neurips-2020-flatland-challenge/ )

For the RL algorithms the options seem to be:

1. Code them myself
2. Create the environment according to the OpenAI gym API guidelines and use the OpenAI baselines algorithms.

I would love to hear some suggestions and advice on which environments may be most suitable for my problem!

Answer 1

您还可以检查SUMO作为交通模拟器和多智能体强化学习的RLLib 库。