Can't figure out why I'm only getting one value from predict function?

Question

I have built and trained a sequential binary classification model using keras layers. Everything seems to work fine until I start using the predict method. This function starts to give me a weird exponential value rather than probabilities of the two classes. This what I get after training and using predict method on the model

This classification model has two classes, let's say a cat or a dog, so I was expecting the result to be something like [99.9999, 0.0001] suggesting that it's a cat. I'm not sure how to interpret the value that I'm getting back instead.

Here is the code I have:

# Get the data.
    (train_texts, train_labels), (val_texts, val_labels) = data
    train_labels = np.asarray(train_labels).astype('float32')
    val_labels = np.asarray(val_labels).astype('float32')

    # Vectorizing data
    train_texts,val_texts, word_index = vectorize_data.sequence_vectorize(
        train_texts, val_texts)

    # Building the model architecture( adding layers to the model)
    model = build_model.simple_model_layers(train_texts.shape[1:])

    # Setting and compiling with the features like the optimizer, loss and metrics functions
    model = build_model.simple_model_compile(model=model)

    # This is when the learning happens
    history = model.fit(train_texts,
                        train_labels,
                        epochs=EPOCHS,
                        validation_data=(val_texts, val_labels),
                        verbose=VERBOSE_OFF, batch_size=BATCH_SIZE)

    print('Validation accuracy: {acc}, loss: {loss}'.format(
        acc=history['val_acc'][-1], loss=history['val_loss'][-1]))

    # loading data to predict on
    test_text = any
    with open('text_req.pickle', 'rb') as pickle_file:
        test_text = pickle.load(pickle_file)


    print('Lets make a prediction of this requirement:')
    prediction = model.predict(test_text, batch_size=None, verbose=0, steps=None)
    print(prediction)

Here is how the simple model function looks like:

model = models.Sequential()
    model.add(Dense(26, activation='relu', input_shape=input_shape))
    model.add(Dense(16, activation='relu'))
    model.add(Dense(10, activation='relu'))
    model.add(Dense(1, activation='sigmoid'))
    return model

Gradient desent functions: optimizer='adam', loss='binary_crossentropy'

Sample data is of String type which I convert to constant size matrices of 1's and 0's using padding and all. The features have two classes, so labels are simply 1 and 0. That's all for data. In my opinion, data doesn't seem to be the problem it could be something more trivial than that which I'm overlooking and have failed to recognize.

Thank you guys, This last problem was resolved, but I need better understanding at this:

I read that sigmoid returns the probability of all possible classes and all the probabilities should add up to 1. The values that I am getting back are:

Validation accuracy: 0.792168688343232, loss: 2.8360600299145804
Let's make a prediction of this requirement:

    [[2.7182817, 1.       ]
     [2.7182817, 1.       ]
     [1.,        2.7182817]
     [1. ,       2.7182817]]

They don't add up to 1 and looking at these values 1 or otherwise is not intuitive enough in what to make of it.

Answer 1

Your model only has one output. If your training labels are set to 0 for cat and 1 for dog then that means the network thinks its a cat if the output is [[2.977094e-12]] . If you want the probabilities of the two classes like you were expecting then you need to change the output of your model as follows:

model = models.Sequential()
model.add(Dense(26, activation='relu', input_shape=input_shape))
model.add(Dense(16, activation='relu'))
model.add(Dense(10, activation='relu')
model.add(Dense(2, activation='softmax'))

Your labels would also need to change to [1, 0] and [0, 1] for cat and dog.

Answer 2

I want to clarify that you don't get a weird exponential value, you just get a weird value. The E is scientific notation for x10, so you basically get 2.7 x 10^-12. I'd love to help but I can't check your data nor your model. I tried to Google some parts of your code, in the hope to find some clarification but I can't seem to find out what's under the hood of these two lines:

model = build_model.simple_model_layers(train_texts.shape[1:])
model = build_model.simple_model_compile(model=model)

I've no clue what network has been build, I'd like to know at least the loss function and the full final layer, that'd already be much to go by. Are you also sure that your data is correct?

EDIT:

sigmoid does not do what you describe, softmax does that. Sigmoid is often used as multilabel classification, since it can detect multiple labels as True. Sigmoid output could look for example like [0.99, 0.3], it has the ability to look at each label separately. Softmax on the other hand doesn't, softmax could look like [0.99, 0.01], and the sum of all probabilities is always 1.

That solved that confusion, now about your output, I've no clue what that is, it should be between 1 and 0, unless I'm missing something here.

To answer your data question you asked to K. Streutker: The goal of a neural network is to create the labels you feed it, on new data. If you want a probability distribution, then you also need to feed one. Every image should have a label [1, 0] and dog [0, 1], or reversed, whatever you like. Then once the model is trained it will be able to give you two outputs that make sense. The loss function, most likely cross entropy takes these labels and the output of your model, and tries to minimize the difference over time. So this is sort of what you need:

image (dog)--> model --> loss --> optimizer that updates the weights

labels ([0,1]) ------------------┘

then predicting will look like this

image --> model --> labels

Hope I helped a bit!