getting both “'numpy.ndarray' object is not callable” and “'Tensor' object is not callable”

Question

I'm working on building a binary classifier: I'm inexperienced in ML so, using code adapted from the Iris classification tutorial on TensorFlow.org I'm getting 85% accuracy on the test set. However, this evaluation is run using a threshold value of 0.5: I'd like to be able to try different threshold values just to see if I can get better accuracy. So I dug into the tensorflow website and found the following command:

tf.metrics.precision_at_thresholds(
    labels,
    predictions,
    thresholds,
    weights=None,
    metrics_collections=None,
    updates_collections=None,
    name=None
)

which looks just like what I need because it would allow me to evaluate the accuracy using whatever custom threshold values I want. Thus, I added this bit to the code for my model and the end result is looking like this:

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import numpy as np
import tensorflow as tf
train_file = "/home/javier/train.csv"
test_file = "/home/javier/test.csv"
def main():
    # Load datasets.
    training_set = tf.contrib.learn.datasets.base.load_csv_with_header(
        filename=train_file,
        target_dtype=np.int,
        features_dtype=np.float32)
    test_set = tf.contrib.learn.datasets.base.load_csv_with_header(
        filename=test_file,
        target_dtype=np.int,
        features_dtype=np.float32)
    # Specify that all features have real-value data
    feature_columns = [tf.contrib.layers.real_valued_column("", dimension=15)]
    # Build 3 layer DNN with 10, 20, 10 units respectively.
    classifier = tf.contrib.learn.DNNClassifier(feature_columns=feature_columns,
            hidden_units=[15,20,15],
            optimizer=tf.train.ProximalAdagradOptimizer(learning_rate=0.05,l2_regularization_strength=0.2),
            n_classes=2,
            model_dir="/home/javier/tf_tinkering")
    # Define the training inputs
    def get_train_inputs():
        x = tf.constant(training_set.data)
        y = tf.constant(training_set.target)
        return x, y
    # Fit model.
    classifier.fit(input_fn=get_train_inputs, steps=50)
    # Define the test inputs
    def get_test_inputs():
        x = tf.constant(test_set.data)
        y = tf.constant(test_set.target)
        return x, y
    # Evaluate accuracy.
    tf.metrics.precision_at_thresholds(
        tf.constant(test_set.target),
        classifier.predict(input_fn=tf.constant(test_set.data)),
        thresholds=[0.5,0.4,0.6],
    )

if __name__ == "__main__":
    main()

The problem is that tf.metrics isn't interpreting the "predictions" bit. I've tried different ways of calling "predictions" and they all return an error. Using

tf.metrics.precision_at_thresholds(
        tf.constant(test_set.target),
        classifier.predict_classes(input_fn=get_test_inputs),
        thresholds=[0.5,0.4,0.6],
    )

gives me

TypeError: Expected binary or unicode string, got <generator object <genexpr> at 0x7ff38d2c5af0>

Using

tf.metrics.precision_at_thresholds(
        tf.constant(test_set.target),
        classifier.predict_classes(input_fn=tf.constant(test_set.data)),
        thresholds=[0.5,0.4,0.6],
    )

results in

TypeError: 'Tensor' object is not callable

and using

tf.metrics.precision_at_thresholds(
        tf.constant(test_set.target),
        classifier.predict_classes(input_fn=test_set.data),
        thresholds=[0.5,0.4,0.6],
    )

outputs

TypeError: 'numpy.ndarray' object is not callable

I even tried defining a new matrix "feature_columns_matrix" and pasted all the values into it from the csv file and ran "classifier.predict_classes(input_fn=feature_columns_matrix)" it also didn't work. How do I pass the values of the output layer of my network when run on the test set onto the tf.metrics subroutine?

I've already read about 10 other similar questions on this website and none have helped me (just so you know I'm not asking a redundant question). Any help will be greatly appreciated! Thanks

UPDATE: I found out that running

print(list(classifier.predict(input_fn=get_test_inputs)))

correctly returns the predicted classes for each sample in the test file. However, that's not quite what I need for tf.metrics to evaluate accuracy because the above command returns classes precisely based on the 0.5 threshold! It doesn't give the actual output of the final layer of the net. I get this when I ran the command above:

[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

But what I really need is the actual float32 values it produces when the net runs on the test set. That way I can feed that into the tf.metrics and test different threshold values. Does anybody know how to do that?

Answer 1

TypeError: 'numpy.ndarray' object is not callable

This means you have a numpy array at this point, and you are trying to use as though it were a function. That is, you are 'calling' it with arr(...) syntax. Either you should be indexing it arr[...] , or this object shouldn't be an array in the first place.

Same for the tensor object.

TypeError: Expected binary or unicode string, got <generator object <genexpr> at 0x7ff38d2c5af0>

means that the function expected a string as a parameter, but you are giving it something else (the concept of a generator is probably too advanced for you.)

Ideally when programming in Python you need to understand what each variable is referencing, specifically what kind of object. Is is a function, a string, a number, an array, and tensorflow object. When you have bugs like this you need to add diagnostic prints that check what the objects. Don't just assume; test.

Apparently you are trying different things in this expression:

classifier.predict_classes(input_fn=get_test_inputs)
classifier.predict_classes(input_fn=tf.constant(test_set.data))
classifier.predict_classes(input_fn=test_set.data)

What does this function accept? I'm guessing test_set.dat is a numpy array. Wrapping that in tf.constant() turns it into a tensor object. While get_test_inputs is a generator/function.