Using a trained Keras model to make predictions on new csv data

Question

so I'm making a project where basically i have to predict whether or not a house price is above or below its median price and to do that, I'm using this dataset from Kaggle( https://drive.google.com/file/d/1GfvKA0qznNVknghV4botnNxyH-KvODOC/view ). 1 means "Above Median" and 0 means "Below Median". I wrote this code to train a neural network and save it as a .h5 file:

import pandas as pd
from sklearn import preprocessing
from sklearn.model_selection import train_test_split
from keras.models import Sequential
from keras.layers import Dense
import h5py

df = pd.read_csv('housepricedata.csv')
dataset = df.values

X = dataset[:,0:10]
Y = dataset[:,10]

min_max_scaler = preprocessing.MinMaxScaler()

X_scale = min_max_scaler.fit_transform(X)

X_train, X_val_and_test, Y_train, Y_val_and_test = train_test_split(X_scale, Y, test_size=0.3)
X_val, X_test, Y_val, Y_test = train_test_split(X_val_and_test, Y_val_and_test, test_size=0.5)

model = Sequential([
    Dense(32, activation='relu', input_shape=(10,)),
    Dense(32, activation='relu'),
    Dense(1, activation='sigmoid'),
])

model.compile(optimizer='sgd',
              loss='binary_crossentropy',
              metrics=['accuracy'])

hist = model.fit(X_train, Y_train,
          batch_size=32, epochs=100,
          validation_data=(X_val, Y_val))

model.save("house_price.h5")

After running it, it successfully saves the .h5 file to my directory. What I want to do now is use my trained model to make predictions on a new .csv file and determine whether or not each of those are above or below median price. This is an image of the csv file in VSCode that i want it to make predictions on: csv file image As you can see, this file doesn't contain a 1(above median) or 0(below median) because that's what I want it to predict. This is the code I wrote to do that:

import pandas as pd
from sklearn import preprocessing
from sklearn.model_selection import train_test_split
from keras.models import Sequential
from keras.layers import Dense
from keras.models import load_model
import h5py

df = pd.read_csv('data.csv')
dataset = df.values

X = dataset[:,0:10]
Y = dataset[:,10]

min_max_scaler = preprocessing.MinMaxScaler()

X_scale = min_max_scaler.fit_transform(X)

X_train, X_val_and_test, Y_train, Y_val_and_test = train_test_split(X_scale, Y, test_size=0.3)
X_val, X_test, Y_val, Y_test = train_test_split(X_val_and_test, Y_val_and_test, test_size=0.5)

model = load_model("house_price.h5")

y_pred = model.predict(X_test)

print(y_pred)

It's output is [[0.00101464]] I have no clue what that is and why it's only returning one value even though the csv file has 4 rows. Does anyone know how I can fix that and be able to predict either a 1 or a 0 for each row in the csv file? Thank You!

Answer 1

As much I understand what you want! Let's Try ! This code work for me

 import tensorflow
 model = tensorflow.keras.models.load_model("house_price.h5")
 y_pred=model.predict(X_test)

still you are not able to get visit following site 1: answer1 2: answer2

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

# Importing the dataset
dataset = pd.read_csv('C:\\Users\\acer\\Downloads\\housepricedata.csv')
dataset.head()

X=dataset.iloc[:,0:10]
y=dataset.iloc[:,10]

X.head()
from sklearn.preprocessing import StandardScaler
obj=StandardScaler()
X=obj.fit_transform(X)

from sklearn.model_selection import train_test_split
 X_train,X_test,y_train,y_test=train_test_split
                                        (X,y,random_state=2020,test_size=0.25)

  print(X_train.shape)
  print(X_test.shape)
  print(y_train.shape)
  print(y_test.shape)

  import keras
  from keras.models import Sequential
  from keras.layers import Dense
  from keras.layers import Dropout
  classifier = Sequential()

    # Adding the input layer and the first hidden layer
  classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 
                                                      'relu', input_dim = 10))
   # classifier.add(Dropout(p = 0.1))

   # Adding the second hidden layer
   classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation 
                                                                   = 'relu'))
   # classifier.add(Dropout(p = 0.1))

   # Adding the output layer
   classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation 
                                                               = 'sigmoid'))

       # Compiling the ANN
  classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics 
                                                          = ['accuracy'])

  classifier.fit(X_train, y_train, batch_size = 10, epochs = 100)
  y_pred = classifier.predict(X_test)
  y_pred = (y_pred > 0.5)
  print(y_pred)

  classifier.save("house_price.h5")

  import tensorflow
  model = tensorflow.keras.models.load_model("house_price.h5")
  y_pred=model.predict(X_test)
  y_pred = (y_pred > 0.5)
  print(y_pred)

Both y_pred produce same output for me

Here one thing you not y_pred not contain 0 and 1 because you use sigmoid function which determine predication in probability so if(y_pred>0.5) it mean value is one

  #True rep one

  #false rep zero

  #you can use replace function or map function of pandas  to get convert true 
 into 1