为什么我会得到：IndexError：元组索引超出范围？

Question

i've been trying to create a tictactoe agent and when i ran the training loop i got an error pointing towards the (.0) in the function that checks whether someone won(check_game_status). if you need the whole environment let me know.

here is the way the board was created:

board = [0] * 9

which means the 0 means empty spot, 1 for O and 2 for x.

here is after_action_state function:

def after_action_state(state, action):
    """Execute an action and returns resulted state.

    Args:
        state (tuple): Board status + mark
        action (int): Action to run

    Returns:
        tuple: New state
    """

    board, mark = state
    nboard = list(board[:])
    nboard[action] = tocode(mark)
    nboard = tuple(nboard)
    return nboard, next_mark(mark)

tocode function basically converts 'X' to 2 and 'O' to 1

this is the function that the error pointed at:

def check_game_status(board):
"""Return game status by current board status.

Args:
    board (list): Current board state

Returns:
    int:
        -1: game in progress
        0: draw game,
        1 or 2 for finished game(winner mark code).
"""
for t in [1, 2]:
    for j in range(0, 9, 3):
        if [t] * 3 == [board[i] for i in range(j, j+3)]:
            return t
    for j in range(0, 3):
        if board[j] == t and board[j+3] == t and board[j+6] == t:
            return t
    if board[0] == t and board[4] == t and board[8] == t:
        return t
    if board[2] == t and board[4] == t and board[6] == t:
        return t

for i in range(9):
    if board[i] == 0:
        # still playing
        return -1

# draw game
return 0

this is the play_one_step function that takes experience and puts it in a replay buffer using greedy epsilon policy.

 def play_one_step(self, env ,state, available_actions, agents_model ,epsilon= 0.2):
    action = self.greedy_policy(state, available_actions, agents_model ,epsilon= epsilon)
    next_state, reward, done, info = env.step(action)
    self.replay_buffer.append((state, action,reward, next_state, done))
    return next_state, reward, done, info

and this is the epsilon greedy policy:

def greedy_policy(self, state ,available_actions, agents_model ,epsilon = 0.2):
        if np.random.rand() < epsilon:
            return np.random.choice(available_actions)
        
        else:
            
            for next_action in available_actions: # checking if the next possible action wins the game and if it does then it returns it
                next_state = after_action_state(state, next_action)
                game_status = check_game_status(next_state)
                if game_status > 0 & tomark(game_status)==self.mark:
                    return next_action
                
            Q_values = agents_model.predict(state[np.newaxis])
            return np.argmax(Q_values[0])

this is the training loop:

agent_1 = Agent('X', model_1)
agent_2 = Agent('O', model_2)
agent_1_rewards = []
agent_2_rewards = []
agents = [agent_1, agent_2]


n_episodes = 600
start_mark = 'O'
batch_size = 15



for episode in range(n_episodes):
    env = TicTacToeEnv()
    env.set_start_mark(start_mark)  
    state = env.reset()
    while not env.done:
        _ , mark = state
        available_actions = env.available_actions()
        epsilon = max( 1 - episode/500 , 0.01)
        agent = agent_by_mark(agents, mark)
        agents_model = agent.agents_model
        
        
        next_state, reward, done, _ = agent.play_one_step(env, state, available_actions, agents_model  ,epsilon)
        state = next_state
        if agent is agent_1:
            agent_1_rewards.append(reward)
        else:
            agent_2_rewards.append(reward)
            
            
        env.render()
        
    
    if episode > 60:
        agent.training_step(15)

this is the error trace:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_17792/1466320926.py in <module>
     26 
     27 
---> 28         next_state, reward, done, _ = agent.play_one_step(env, state, available_actions, agents_model  ,epsilon)
     29         state = next_state
     30         if agent is agent_1:

~\AppData\Local\Temp/ipykernel_17792/3137506088.py in play_one_step(self, env, state, available_actions, agents_model, epsilon)
     60 
     61     def play_one_step(self, env ,state, available_actions, agents_model ,epsilon= 0.2):
---> 62         action = self.greedy_policy(state, available_actions, agents_model ,epsilon= epsilon)
     63         next_state, reward, done, info = env.step(action)
     64         self.replay_buffer.append((state, action,reward, next_state, done))

~\AppData\Local\Temp/ipykernel_17792/3137506088.py in greedy_policy(self, state, available_actions, agents_model, epsilon)
     35                 for next_action in available_actions: # checking if the next possible action wins the game and if it does then it returns it
     36                     next_state = after_action_state(state, next_action)
---> 37                     game_status = check_game_status(next_state)
     38                     if game_status > 0 & tomark(game_status)==self.mark:
     39                         return next_action

g:\Code\TicTacToe\gym-tictactoe\gym_tictactoe\env.py in check_game_status(board)
     66     for t in [1, 2]:
     67         for j in range(0, 9, 3):
---> 68             if [t] * 3 == [board[i] for i in range(j, j+3)]:
     69                 return t
     70         for j in range(0, 3):

g:\Code\TicTacToe\gym-tictactoe\gym_tictactoe\env.py in <listcomp>(.0)
     66     for t in [1, 2]:
     67         for j in range(0, 9, 3):
---> 68             if [t] * 3 == [board[i] for i in range(j, j+3)]:
     69                 return t
     70         for j in range(0, 3):

IndexError: tuple index out of range

does anyone know what causes this issue? i already tried to rewrite the function only and give it a few cases and it worked absolutely fine with no index problem. if you need more of the code let me know. thanks in advance!

Answer 1

It is not a fix but explains the cause of bug... In the below snippet, you are trying to access board[i] with index i > 9 but your board is of size 9 . For instance, check for j=9 .

for j in range(0, 9, 3):
        if [t] * 3 == [board[i] for i in range(j, j+3)]:
            return t