Keras - TypeError: only integer scalar arrays can be converted to a scalar index

Question

I am trying to learn keras, specifically LSTM for anomaly detection in time series, and to do so I have been following the examples online. Yet for some reason, it is not working. I have done as was suggested on a previous post relating to TypeError: only integer scalar arrays can be converted to a scalar index , but nothing has worked. From that, I gather it has something to do with Numpy. Here is my code:

import numpy
import pandas
import matplotlib.pyplot as plt
import math
import tensorflow as tf
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error

# fix random seed for reproducibility
numpy.random.seed(7)

#load the dataset
dataframe = pandas.read_csv('international-airline-passengers.csv', usecols=[1], engine='python', skipfooter=3)
dataset = dataframe.values
dataset = dataset.astype('float32')

#normalize the dataset

scaler = MinMaxScaler(feature_range=(0, 1))
dataset = scaler.fit_transform(dataset)

# split into train and test sets
train_size = int(len(dataset)*0.67)
test_size = len(dataset) - train_size
train, test = dataset[0:train_size,:], dataset[train_size:len(dataset),:]
print(len(train), len(test))

# convert an array of values into a dataset matrix
def create_dataset(dataset, look_back=1):
    dataX, dataY = [], []
    for i in range(len(dataset) - look_back - 1):
        a = dataset[i:(i + look_back), 0]
        dataX.append(a)
        dataY.append(dataset[i + look_back, 0])
    return numpy.array(dataX), numpy.array(dataY)

# reshape into X=t and Y=t+1
look_back = 1
trainX = create_dataset(train, look_back)[0]
trainY = create_dataset(train, look_back)[0]
testX = create_dataset(test, look_back)[0]
testY = create_dataset(test, look_back)[0]

#reshape input to be [samples, time steps, features]
trainX = numpy.reshape(trainX, (trainX[0], 1, trainX.shape[1]))[0]
testX = numpy.reshape(testX)

# create and fit the LSTM network
model = Sequential()[0]
model.add(LSTM(4, input_shape=(1, look_back)))
model.add(Dense(1))[0]
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(trainX, trainY, epochs=100, batch_size=1, verbose=2)

#make predictions
trainPredict = model.predict(trainX)
testPredict = model.predict(testX)

#invert predictions
trainPredict = scaler.inverse_transform(trainPredict)
trainY = scaler.inverse_transform([trainY])[0]

# calculate root mean squared error
trainScore = math.sqrt(mean_squared_error(train[0], trainPredict[:,0]))
print('Train Score: %.2f RMSE' % (trainScore))
testScore = math.sqrt(mean_squared_error(testY[0], testPredict[:,0]))
print('Test Score: %.2f' % (testScore))

# shift train  predictions for plotting
trainPredictPlot = numpy.empty_like(dataset)
trainPredictPlot[:, :] = numpy.nan
trainPredictPlot[look_back:len(trainPredict)+look_back, :] = trainPredict
# shift test predictions for plotting
testPredictPlot = numpy.empty_like(dataset)
testPredictPlot[:, :] = numpy.nan
testPredictPlot[len(trainPredict)+(look_back*2)+1:len(dataset)-1, :] = testPredict
# plot baseline and predictions
plt.plot(scaler.inverse_transform(dataset))
plt.plot(trainPredictPlot)
plt.plot(testPredictPlot)
plt.show()

And from that i get the errors:

Using TensorFlow backend.
96 48
Traceback (most recent call last):
  File "C:\Users\fires\Anaconda3\envs\python3.5\lib\site-packages\numpy\core\fromnumeric.py", line 57, in _wrapfunc
    return getattr(obj, method)(*args, **kwds)
TypeError: only integer scalar arrays can be converted to a scalar index


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/fires/PycharmProjects/RSI/Test 1.py", line 52, in <module>
    trainX = numpy.reshape(trainX, (trainX[0], 1, trainX.shape[1]))[0]
  File "C:\Users\fires\Anaconda3\envs\python3.5\lib\site-packages\numpy\core\fromnumeric.py", line 232, in reshape
    return _wrapfunc(a, 'reshape', newshape, order=order)
  File "C:\Users\fires\Anaconda3\envs\python3.5\lib\site-packages\numpy\core\fromnumeric.py", line 67, in _wrapfunc
    return _wrapit(obj, method, *args, **kwds)
  File "C:\Users\fires\Anaconda3\envs\python3.5\lib\site-packages\numpy\core\fromnumeric.py", line 47, in _wrapit
    result = getattr(asarray(obj), method)(*args, **kwds)
TypeError: only integer scalar arrays can be converted to a scalar index

Answer 1

You are trying to reshape your array into a non integer length.

You've written the following code;

#reshape input to be [samples, time steps, features]
trainX = numpy.reshape(trainX, (trainX[0], 1, trainX.shape[1]))[0]
testX = numpy.reshape(testX)

However I suspect you mean trainX.shape[0] instead of trainX[0] . This fixes the only integer arrays can be converted to a scalar index error. However in the line below this, you have written testX = numpy.reshape(testX) , which is invalid, as numpy.reshape requires a shape argument. I'm not sure exactly what you are trying to achieve with that line, but hopefully bringing this to your attention fixes your problem!