How to convert a keras tensor to a numpy array

Question

I am trying to create a q-learning chess engine where the output of the last layer of the neural network (the density is equal to the number of legal moves) is run through a argmax() function which returns an integer that I am using as an index for the array where the legal moves are stored. Here is part of my code:

#imports

env = gym.make('ChessAlphaZero-v0')   #builds environment
obs = env.reset()
type(obs)

done = False   #game is not won

num_actions = len(env.legal_moves)   #array where legal moves are stored

obs = chess.Board() 

model = models.Sequential()

def dqn(board):
    
    #dense layers
    
    action = layers.Dense(num_actions)(layer5)
    
    i = np.argmax(action)
    move = env.legal_moves[i]

    return keras.Model(inputs=inputs, outputs=move)

But when I run the code I get the following error:

TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.

Any code examples would be appreciated, thanks.

Answer 1

The correct way to build a model and forward an input in keras is this:

1. Building the model

model = models.Sequential()
model.add(layers.Input(observation_shape))
model.add(layers.Dense(units=128, activation='relu'))
model.add(layers.Dense(units=num_actions, activation='softmax'))
return model

or

inputs = layers.Input(observation_shape)
x = layers.Dense(units=128, activation='relu')(inputs)
outputs = layers.Dense(units=num_actions, activation='softmax')(x)

model = keras.Model(inputs, output)

Both ways are equal.

2. Forward an observation & Get the best possible action

action_values = model.predict(observation)
best_action_index = tf.argmax(action_values)
best_action = action_values[best_action_index]

Implementing DQN by yourself in keras can be quite frustrating. You might wanna use a DRL framework such as tf_agents that has implementations of lots of agents: https://www.tensorflow.org/agents

This repository contains a clean and easy to understand implementation of DQN for openai gym environments. Also, it contains examples of using tf_agents library as well for more complex agents: https://github.com/kochlisGit/Tensorflow-DQN