Tensorflow 2.0 警告-dense_features 正在将输入张量从 dtype float64 转换为 float32 层的 dtype

Question

I am reading over the Tensorflow 2.0 google website tutorial , where they discuss the Feature Columns API. In the second in which they discuss numeric columns, the sample code generates the warning below. The warning seems to be about casting some data, but the message does not exactly explain how to fix the problem--that is, where should the user explicitly cast the data so that this warning is avoided.:

WARNING:tensorflow:Layer dense_features is casting an input tensor from dtype 
float64 to the layer's dtype of float32, which is new behavior in TensorFlow 
2.  The layer has dtype float32 because it's dtype defaults to floatx.

If you intended to run this layer in float32, you can safely ignore this 
warning. If in doubt, this warning is likely only an issue if you are porting 
a TensorFlow 1.X model to TensorFlow 2.

To change all layers to have dtype float64 by default, call 
`tf.keras.backend.set_floatx('float64')`. To change just this layer, pass 
dtype='float64' to the layer constructor. If you are the author of this layer, 
you can disable autocasting by passing autocast=False to the base Layer 
constructor.

I am trying to figure out how to fix this warning, because it is showing up on some of my own code too. The code to generate this warning is:

from __future__ import absolute_import, division, print_function, unicode_literals

import numpy as np
import pandas as pd

import tensorflow as tf

from tensorflow import feature_column
from tensorflow.keras import layers
from sklearn.model_selection import train_test_split

URL = 'https://storage.googleapis.com/applied-dl/heart.csv'
dataframe = pd.read_csv(URL)
train, test = train_test_split(dataframe, test_size=0.2)
train, val = train_test_split(train, test_size=0.2)
print(len(train), 'train examples')
print(len(val), 'validation examples')
print(len(test), 'test examples')

def df_to_dataset(dataframe, shuffle=True, batch_size=32):
  dataframe = dataframe.copy()
  labels = dataframe.pop('target')
  ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
  if shuffle:
    ds = ds.shuffle(buffer_size=len(dataframe))
  ds = ds.batch(batch_size)
  return ds

batch_size = 5 # A small batch sized is used for demonstration purposes
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

# We will use this batch to demonstrate several types of feature columns
example_batch = next(iter(train_ds))[0]

# A utility method to create a feature column
# and to transform a batch of data
def demo(feature_column):
  feature_layer = layers.DenseFeatures(feature_column)
  print(feature_layer(example_batch).numpy())

age = feature_column.numeric_column("age")
demo(age) # <-- SHOULD TRIGGER OR DISPLAY THE WARNING

Any suggestions about how to fix this?

Answer 1

tl;dr to silence this warning, manually cast your input to float32

X = tf.cast(X, tf.float32) 
y = tf.cast(y, tf.float32)

or with numpy :

X = np.array(X, dtype=np.float32)
y = np.array(y, dtype=np.float32)

Explanation

By default, Tensorflow uses floatx , which defaults to float32 , because it's more than enough for deep learning. You can verify this:

import tensorflow as tf
tf.keras.backend.floatx()

Out[3]: 'float32'

The input you provided, is of dtype float64 , so there is a mismatch between Tensorflow's default dtype for weights, and the input. Tensorflow doesn't like that, because casting (changing the dtype) is costly. Tensorflow will generally throw an error when manipulating tensors of different dtypes (eg, comparing float32 logits and float64 labels).

The new behavior it's talking about:

Layer my_model_1 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2

Is that it will automatically cast the input dtype to float32 . Tensorflow 1.X probably threw an exception in this situation, although I can't say I've ever used it.