如何在每回合接收 3 个缩放器和一个矩阵的自定义健身房环境中定义动作空间？

Question

for a personal project, I need to define a custom gym environment that runs a certain board game. each turn of the game, the environment takes the state of the board as a matrix of ones and zeros, and an action - described as a tuple:

(integer, integer, small matrix)

from reading online, I know that a gym env should take the shape:

 class CustomEnv(gym.Env):
  """Custom Environment that follows gym interface"""
  metadata = {'render.modes': ['human']}

  def __init__(self, arg1, arg2, ...):
    super(CustomEnv, self).__init__()

    self.action_space = 
    self.observation_space = 

  def step(self, action):
    ...
  def reset(self):
    ...
  def render(self, mode='human', close=False):

now, I feel like the action input here does not exactly fall into "discrete" or "continuous" - how should I implement the action part of the init function and the step function?

Answer 1

Defining your action space in the init function is fairly straight forward using gym's Tuple space:

from gym import spaces
space = spaces.Tuple((
  spaces.Discrete(5),
  spaces.Discrete(4),
  spaces.Box(low=0, high=1, shape=(2, 2))))

The Discrete space represents a range of integers and the Box space to represents a n-dimensional array. You can print a sample of your space to get an idea of what it looks like:

print(space.sample())
>>> (3, 1, array([[0.20318432, 0.26787955], [0.5323673 , 0.6564413 ]], dtype=float32))

For the step function, you just need to interact with your environment based on the input action, which will be formatted just like the sample.