Calculating the Accuracy of A Keras Neural Network in Python

Question

I have created a Keras neural network. The neural network was trained during eight epochs, and it outputs this loss value and accuracy:

Epoch 1/8
2009/2009 [==============================] - 0s 177us/step - loss: 0.0824 - acc: 4.9776e-04
Epoch 2/8
2009/2009 [==============================] - 0s 34us/step - loss: 0.0080 - acc: 4.9776e-04
Epoch 3/8
2009/2009 [==============================] - 0s 37us/step - loss: 0.0071 - acc: 4.9776e-04
Epoch 4/8
2009/2009 [==============================] - 0s 38us/step - loss: 0.0071 - acc: 4.9776e-04
Epoch 5/8
2009/2009 [==============================] - 0s 35us/step - loss: 0.0070 - acc: 4.9776e-04
Epoch 6/8
2009/2009 [==============================] - 0s 38us/step - loss: 0.0071 - acc: 4.9776e-04
Epoch 7/8
2009/2009 [==============================] - 0s 36us/step - loss: 0.0068 - acc: 4.9776e-04
Epoch 8/8
2009/2009 [==============================] - 0s 40us/step - loss: 0.0070 - acc: 4.9776e-04

How do I interpret the loss function provided within the output? Is there any way to find the variation percentage between the actual price and prediction for every single day in the data set?

Here is the neural network:

import tensorflow as tf
import keras
import numpy as np
#import quandle
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
import pandas as pd
import sklearn
import math
import pandas_datareader as web

def func_stock_prediction(stockdata, start, end):
  start = start
  end = end
  df = web.DataReader(stockdata, "yahoo", start, end)
  df = df[['Close']]


  previous = 5


  def create_dataset(df, previous):
      dataX, dataY = [], []
      for i in range(len(df)-previous-1):
          a = df[i:(i+previous), 0]
          dataX.append(a)
          dataY.append(df[i + previous, 0])
      return np.array(dataX), np.array(dataY)

  scaler = sklearn.preprocessing.MinMaxScaler(feature_range = (0, 1))
  df = scaler.fit_transform(df)


  train_size = math.ceil(len(df) * 0.5)

  train, val = df[0:train_size,:], df[train_size:len(df),:]

  X_train, Y_train = create_dataset(train, previous)


  print(X_train)
  print(Y_train)

  print(X_train.shape)
  print(Y_train.shape)

  X_val, Y_val = create_dataset(val, previous)

  X_train = np.reshape(X_train, (X_train.shape[0], 1, X_train.shape[1]))
  X_val = np.reshape(X_val, (X_val.shape[0], 1, X_val.shape[1]))

  model = keras.models.Sequential() 
  model.add(keras.layers.Dense(units = 64, activation = 'relu', input_shape = (1, 5)))
  model.add(keras.layers.Flatten())
  model.add(keras.layers.Dense(units = 1, activation = 'linear'))
  model.compile(loss='mean_absolute_error', 
                optimizer='adam', 
                metrics=['accuracy'])

  history = model.fit(X_train, Y_train, epochs=8)

  train = model.predict(X_train)
  val = model.predict(X_val)

  train = scaler.inverse_transform(train)
  Y_train = scaler.inverse_transform([Y_train])
  val = scaler.inverse_transform(val)
  Y_val = scaler.inverse_transform([Y_val])
  predictions = val


  trainPlot = np.empty_like(df)
  trainPlot[:, :] = np.nan
  trainPlot[previous:len(train)+previous, :] = train
  valPlot = np.empty_like(df)
  valPlot[:, :] = np.nan
  valPlot[len(train)+(previous*2)+1:len(df)-1, :] = val
  inversetransform, =plt.plot(scaler.inverse_transform(df))
  train, =plt.plot(trainPlot)
  val, =plt.plot(valPlot)
  plt.xlabel('Number of Days')
  plt.ylabel('Stock Price')
  plt.title("Predicted vs. Actual Stock Price Per Day")
  plt.show()

func_stock_prediction("PLAY", 2010-1-1, 2020-1-1)

Answer 1

You are using accuracy as a metric. Accuracy measures the proportion of predicted labels that match the true labels. Accuracy is used mostly (to my knowledge) for classification tasks. As far as I know, the accuracy is not really interpretable when you're predicting a continuous outcome variable.

Based on your code, it looks like you're using the neural network for a regression problem (you're predicting a continuous variable). For regression problem meterics, people often use "mean squared error", "root mean squared error", "mean absolute error", "R^2", etc.

If you're interested in percentage differences, then maybe you could try the keras loss , "mean_absolute_percentage_error".