Keras model.predict always predicts 1

Question

I'm working on some Artificial Intelligence project and I want to predict the bitcoin trend but while using the model.predict function from Keras with my test_set, the prediction is always equal to 1 and the line in my diagram is therefor always straight.

import csv
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

from cryptory import Cryptory
from keras.models import Sequential, Model, InputLayer
from keras.layers import LSTM, Dropout, Dense
from sklearn.preprocessing import MinMaxScaler


def format_to_3d(df_to_reshape):
    reshaped_df = np.array(df_to_reshape)
    return np.reshape(reshaped_df, (reshaped_df.shape[0], 1, reshaped_df.shape[1]))


crypto_data = Cryptory(from_date = "2014-01-01")
bitcoin_data = crypto_data.extract_coinmarketcap("bitcoin")

sc = MinMaxScaler()

for col in bitcoin_data.columns:
    if col != "open":
        del bitcoin_data[col]

training_set = bitcoin_data;
training_set = sc.fit_transform(training_set)

# Split the data into train, validate and test
train_data = training_set[365:]

# Split the data into x and y
x_train, y_train = train_data[:len(train_data)-1], train_data[1:]

model = Sequential()
model.add(LSTM(units=4, input_shape=(None, 1))) # 128 -- neurons**?
# model.add(Dropout(0.2))
model.add(Dense(units=1, activation="softmax"))  # activation function could be different
model.compile(optimizer="adam", loss="mean_squared_error")  # mse could be used for loss, look into optimiser

model.fit(format_to_3d(x_train), y_train, batch_size=32, epochs=15)

test_set = bitcoin_data
test_set = sc.transform(test_set)
test_data = test_set[:364]

input = test_data
input = sc.inverse_transform(input)
input = np.reshape(input, (364, 1, 1))

predicted_result = model.predict(input)
print(predicted_result)

real_value = sc.inverse_transform(input)

plt.plot(real_value, color='pink', label='Real Price')
plt.plot(predicted_result, color='blue', label='Predicted Price')
plt.title('Bitcoin Prediction')
plt.xlabel('Time')
plt.ylabel('Prices')
plt.legend()
plt.show()

The training set performance looks like this:

1566/1566 [==============================] - 3s 2ms/step - loss: 0.8572
Epoch 2/15
1566/1566 [==============================] - 1s 406us/step - loss: 0.8572
Epoch 3/15
1566/1566 [==============================] - 1s 388us/step - loss: 0.8572
Epoch 4/15
1566/1566 [==============================] - 1s 388us/step - loss: 0.8572
Epoch 5/15
1566/1566 [==============================] - 1s 389us/step - loss: 0.8572
Epoch 6/15
1566/1566 [==============================] - 1s 392us/step - loss: 0.8572
Epoch 7/15
1566/1566 [==============================] - 1s 408us/step - loss: 0.8572
Epoch 8/15
1566/1566 [==============================] - 1s 459us/step - loss: 0.8572
Epoch 9/15
1566/1566 [==============================] - 1s 400us/step - loss: 0.8572
Epoch 10/15
1566/1566 [==============================] - 1s 410us/step - loss: 0.8572
Epoch 11/15
1566/1566 [==============================] - 1s 395us/step - loss: 0.8572
Epoch 12/15
1566/1566 [==============================] - 1s 386us/step - loss: 0.8572
Epoch 13/15
1566/1566 [==============================] - 1s 385us/step - loss: 0.8572
Epoch 14/15
1566/1566 [==============================] - 1s 393us/step - loss: 0.8572
Epoch 15/15
1566/1566 [==============================] - 1s 397us/step - loss: 0.8572

I'm supposed to print a plot with the Real Price and the Predicted Price, the Real Price is displayed properly but the Predicted price is only a straight line because of that model.predict that only contains the value 1.

Thanks in advance!

Answer 1

You're trying to predict a price value, that is, you're aiming at solving a regression problem and not a classification problem.

However, in your last layer of the network ( model.add(Dense(units=1, activation="softmax")) ), you have a single neuron (which would be adequate for a regression problem), but you've chosen to use a softmax activation function. The softmax function is used in multi-class classification problems, to normalize the outputs into a probability distribution. If you have a single output neuron and you apply softmax, the final result will always 1.0, as it is the only parameter of the probability distribution.

In summary, for regression problems you do not use an activation function, as the network is intended to already output the predicted value.