TensorFlow import data for linear regression

Question

I'm working on project on TensorFlow and I'm trying to train a model with the linear regressor. To add my data on the estimator I'm using the function tf.estimator.inputs.pandas_input_fn() but I can't launch the trainning because I have some problems. I got this error:

TypeError: Failed to convert object of type <class 'dict'> to Tensor. Contents: {'DispositionSoldAmount': <tf.Tensor 'random_shuffle_queue_DequeueMany:4' shape=(128,) dtype=float64>}. Consider casting elements to a supported type.

I tried to change the yData to a pandas.core.series.Series but that didnt change the result.

Did someone have a solution to solve my problem?

Also, I trained another model with the sklearn.linear_regression with the same DataSet and that work correctly.

This is my code:

FEATURES = ["DispositionMileage", "PurchasePrice", "Age"] # X

feature_cols = [tf.feature_column.numeric_column(k) for k in FEATURES]

estimator = tf.estimator.LinearRegressor(feature_columns=feature_cols,model_dir="train")

def get_input_fn( num_epochs=None, n_batch = 128, shuffle=True):
         return tf.estimator.inputs.pandas_input_fn(
            x=Xdata,                    
            y=ydata,                                         
            batch_size=n_batch,            
            num_epochs=num_epochs,         
            shuffle=shuffle)               

estimator.train(input_fn=get_input_fn(num_epochs=None,n_batch = 128,shuffle=True),steps=1000)

Data used:

Xdata type is pandas.core.frame.DataFrame:

        DispositionMileage  PurchasePrice  Age
9741                  3849        16472.0    0
9744                  3849        16472.0    0
9745                  3849        16472.0    0
9748                  3849        16472.0    0
                  ...
[18105 rows x 3 columns]



ydata type is pandas.core.frame.DataFrame:

        DispositionSoldAmount
9741                   1650.0
9744                   1650.0
9745                   1650.0
9748                   1650.0
13465                  7750.0
                  ...
[18105 rows x 1 columns]

The full traceback:

WARNING:tensorflow:From /home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/inputs/queues/feeding_queue_runner.py:62: QueueRunner.__init__ (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
WARNING:tensorflow:From /home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/inputs/queues/feeding_functions.py:500: add_queue_runner (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
Traceback (most recent call last):
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 527, in make_tensor_proto
    str_values = [compat.as_bytes(x) for x in proto_values]
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 527, in <listcomp>
    str_values = [compat.as_bytes(x) for x in proto_values]
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/util/compat.py", line 61, in as_bytes
    (bytes_or_text,))
TypeError: Expected binary or unicode string, got {'DispositionSoldAmount': <tf.Tensor 'random_shuffle_queue_DequeueMany:4' shape=(128,) dtype=float64>}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tuto.py", line 85, in <module>
    estimator.train(input_fn=get_input_fn(num_epochs=None,n_batch = 128,shuffle=True),steps=1000)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 354, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 1207, in _train_model
    return self._train_model_default(input_fn, hooks, saving_listeners)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 1237, in _train_model_default
    features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 1195, in _call_model_fn
    model_fn_results = self._model_fn(features=features, **kwargs)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/canned/linear.py", line 537, in _model_fn
    sparse_combiner=sparse_combiner)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/canned/linear.py", line 215, in _linear_model_fn
    logits=logits)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/canned/head.py", line 239, in create_estimator_spec
    regularization_losses))
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/canned/head.py", line 1482, in _create_tpu_estimator_spec
    features=features, mode=mode, logits=logits, labels=labels)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/canned/head.py", line 1381, in create_loss
    expected_labels_dimension=self._logits_dimension)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/estimator/canned/head.py", line 305, in _check_dense_labels_match_logits_and_reshape
    labels = sparse_tensor.convert_to_tensor_or_sparse_tensor(labels)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/framework/sparse_tensor.py", line 279, in convert_to_tensor_or_sparse_tensor
    value, dtype=dtype, name=name)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1146, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 229, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 208, in constant
    value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/home/USER/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 531, in make_tensor_proto
    "supported type." % (type(values), values))
TypeError: Failed to convert object of type <class 'dict'> to Tensor. Contents: {'DispositionSoldAmount': <tf.Tensor 'random_shuffle_queue_DequeueMany:4' shape=(128,) dtype=float64>}. Consider casting elements to a supported type.

Answer 1

You need to convert your ydata Dataframe to pandas.Series

ydata = pd.Series(ydata[column_name])

Checked with random data, it's working. I'm actually surprised, it look like in newer TF versions tf.estimator.inputs.pandas_input_fn does not accept dataframe as labels.

Answer 2

Ok that work with the pandas.Series . Thanks for your answer. The pandas.DataFrame look like that didn't work with the y parametter on the tf.estimator.inputs.pandas_input_fn function .