我的神经网络只预测一件事

Question

I am new to deep neural networks and trying to implement it on python from scratch. I tried a lot but couldn't find the bug in my implementation. Whenever I use my 'predict' function it always outputs, 0. I have also tested each and every function that you'll see in my code provided below using random arrays of the same shape as x and y(you'll see below) and all of them seems to work perfectly. I also have previously cleaned the data.

import os
os.chdir(r'path where my data is store')#This block of code changes directory to where my data set

Creating a dataframe and assigning values to the input and target vector

import pandas as pd
import numpy as np
df = pd.read_csv('clean_data.csv')
X = df[['radius_mean', 'texture_mean', 'perimeter_mean',
   'area_mean', 'smoothness_mean', 'compactness_mean', 'concavity_mean',
   'concave points_mean', 'symmetry_mean', 'fractal_dimension_mean',
   'radius_se', 'texture_se', 'perimeter_se', 'area_se', 'smoothness_se',
   'compactness_se', 'concavity_se', 'concave points_se', 'symmetry_se',
   'fractal_dimension_se', 'radius_worst', 'texture_worst',
   'perimeter_worst', 'area_worst', 'smoothness_worst',
   'compactness_worst', 'concavity_worst', 'concave points_worst',
   'symmetry_worst', 'fractal_dimension_worst']].values
Y = df['diagnosis'].values 
Y = Y.reshape(569,1)

Splitting the data in training and testing data(x and y are training set and xt and yt are test set)

from sklearn.model_selection import train_test_split
x, xt, y, yt = train_test_split(X, Y, test_size = 0.2, random_state = 40)
x, xt, y, yt = x.T, xt.T, y.T, yt.T

initializing parameters

def iniparams(layer_dims):
params = {}
for l in range(1,len(layer_dims)):
    params['W' + str(l)] = np.random.randn(layer_dims[l],layer_dims[l - 1])*0.01
    params['b' + str(l)] = np.zeros((layer_dims[l],1))
return params

Writing helper functions #1

def sigmoid(Z):
return 1/(1 + np.exp(-Z)), Z

#2

def relu(Z):
return np.maximum(0, Z), Z

Linear forward

def linearfwd(W, A, b):
Z = np.dot(W, A) + b
linear_cache = (W, A, b)
return Z, linear_cache

Forward activation

def fwdactivation(W, A_prev, b, activation):
if activation == 'sigmoid':
    Z, linear_cache = linearfwd(W, A_prev, b)
    A, activation_cache = sigmoid(Z)
elif activation == 'relu':
    Z, linear_cache = linearfwd(W, A_prev, b)
    A, activation_cache = relu(Z)
cache = (linear_cache, activation_cache)
return A, cache

Forward model

def fwdmodel(x, params):
caches = []
L = len(params)//2
A = x
for l in range(1, L):
    A_prev = A
    A, cache = fwdactivation(params['W' + str(l)], A_prev, params['b' + str(l)], 'relu')
    caches.append(cache)
AL, cache = fwdactivation(params['W' + str(L)], A, params['b' + str(L)], 'sigmoid')
caches.append(cache)
return AL, caches

Computing cost

def J(AL, y):
return -np.sum(np.multiply(np.log(AL), y) + np.multiply(np.log(1 - AL), (1 - y)))/y.shape[1]

backward sigmoid

def sigmoidbkwd(dA, cache):
Z = cache
s = 1/(1 + np.exp(-Z))
dZ = dA*s*(1 - s)
return dZ

backward relu`

def sigmoidbkwd(dA, cache):
Z = cache
s = 1/(1 + np.exp(-Z))
dZ = dA*s*(1 - s)
return dZ

linear bkwd

def linearbkwd(dZ, cache):
W, A_prev, b = cache
m = A_prev.shape[1]
dW = np.dot(dZ, A_prev.T)/m
db = np.sum(dZ, axis = 1, keepdims = True)/m
dA_prev = np.dot(W.T, dZ)
return dW, dA_prev, db

backward activation

def bkwdactivation(dA, cache, activation):
linear_cache, activation_cache = cache
if activation == 'sigmoid':
    dZ = sigmoidbkwd(dA, activation_cache)
    dW, dA_prev, db = linearbkwd(dZ, linear_cache)
if activation == 'relu':
    dZ = relubkwd(dA, activation_cache)
    dW, dA_prev, db = linearbkwd(dZ, linear_cache)
return dW, dA_prev, db

backward model

def bkwdmodel(AL, y, cache):
grads = {}
L = len(cache)
dAL = -(np.divide(y, AL) - np.divide(1 - y,1 - AL))
current_cache = cache[L - 1]
grads['dW' + str(L)], grads['dA' + str(L - 1)], grads['db' + str(L)] = bkwdactivation(dAL, current_cache, 'sigmoid')
for l in reversed(range(L - 1)):
    current_cache = cache[l]
    dW_temp, dA_prev_temp, db_temp = bkwdactivation(grads['dA' + str(l + 1)], current_cache, 'relu')
    grads['dW' + str(l + 1)] = dW_temp
    grads['dA' + str(l)] = dA_prev_temp
    grads['db' + str(l + 1)] = db_temp
return grads

Optimizing parameters using gradient descent

def optimize(grads, params, alpha):
L = len(params)//2
for l in range(1, L + 1):
    params['W' + str(l)] = params['W' + str(l)] - alpha*grads['dW' + str(l)]
    params['b' + str(l)] = params['b' + str(l)] - alpha*grads['db' + str(l)]
return params

Neural Network Model

def model(x, y, layer_dims, iters):
costs = []
params = iniparams(layer_dims)
for i in range(1, iters):
    AL, caches = fwdmodel(x, params)
    cost = J(AL, y)
    costs.append(cost)
    grads = bkwdmodel(AL, y, caches)
    params = optimize(grads, params, 1.2)
    if i%100 == 0:
        print('Cost after', i,'iterations is:', cost)
        costs.append(cost)
return costs, params

calculation (The cost does gets mitigated Cost Vs Iterations(Y,X) curve )

costs, params = model(x, y, [30,8,5,4,4,3,1], 3000)

Prediction function

def predict(x,params):

AL, cache = fwdmodel(x,params)
predictions = AL >= 0.5

return predictions

And finally when I do this

predictions = predict(xt,params)
predictions

I get this:

array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])

Please tell me where I am wrong

here's the link to the dataset

Please help me out:D

Answer 1

I don't'see why you've transposed your train-test-split output. Why use xt.T, xT anyway? You should try printing your params(array) output and xt(array) output and see how they are. Are they similar? Does your params output give the right result? Check all of that.

Answer 2

The problem with me was that my Neural Network was too deep. It's a mistake that newbies like me tend to make. I found this awesome resource that helped me realize this mistake: http://theorangeduck.com/page/neural-network-not-working