ValueError: Model output "Tensor("activation_1/Identity:0", shape=(?, 3), dtype=float32)" has invalid shape

Question

I am trying to run the following github code for stock market prediction:

https://github.com/multidqn/deep-q-trading

using their instructions, I run the following after installing the required libraries:

python main.py 3 0 results_folder

However, when I run the above command, I get the following error:

Using TensorFlow backend.
WARNING:tensorflow:From /Users/anisschohra/.local/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:68: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

WARNING:tensorflow:From /Users/anisschohra/.local/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:508: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /Users/anisschohra/.local/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:3837: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

Traceback (most recent call last):
  File "main.py", line 92, in <module>
    ensembleFolderName=sys.argv[3]
  File "/Users/anisschohra/deep-q-trading/deepQTrading.py", line 68, in __init__
    enable_double_dqn=True,enable_dueling_network=True)
  File "/Users/anisschohra/.local/lib/python3.7/site-packages/rl/agents/dqn.py", line 107, in __init__
    raise ValueError(f'Model output "{model.output}" has invalid shape. DQN expects a model that has one dimension for each action, in this case {self.nb_actions}.')
ValueError: Model output "Tensor("activation_1/Identity:0", shape=(?, 3), dtype=float32)" has invalid shape. DQN expects a model that has one dimension for each action, in this case 3.

Could you please help me fix the issue and run the code successfully? I have looked for the error but did not find a solution that works. The model architecture in their code (main.py) is as follows:

model = Sequential()
model.add(Flatten(input_shape=(1,1,68)))
model.add(Dense(35,activation='linear'))
model.add(LeakyReLU(alpha=.001))
model.add(Dense(nb_actions))
model.add(Activation('linear'))

Thanks in advance.

Answer 1

I have the same problem with you, and I`m not so familiar with RL but I guess I reached the cause.

if list(model.output.shape) != list((None, self.nb_actions)):
     raise ValueError(f'Model output "{model.output}" has invalid shape. DQN expects a model that has one dimension for each action, in this case {self.nb_actions}.')

According to the traceback, this is the code which causes the problem. Your model`s output shape is a tensor object, so list(model.output.shape) will be like [Dimension(None), Dimension(3)], but list((None, self.nb_actions)) is [None, 3], so it will be judged as different.

So I think if there is a way that can convert shape of the model`s output to numpy or list object we can solve this problem. Sorry for my poor English!