如何将Dense层的参数的数据类型设置为float16？

Question

I want to use Tensorflow Dense layer with float16 parameters. The default data types of bias and weights are both float32, I tried setting the data type by setting the initializer tf.truncated_normal_initializer(dtype=tf.float16) but it doesn't seem to have any effect.

import tensorflow as tf
A = tf.get_variable(name='foo', shape=[3, 3])
dense = tf.layers.dense(inputs=A, units=3, kernel_initializer=tf.truncated_normal_initializer(dtype=tf.float16))
varis = tf.trainable_variables(scope=None)
print(varis[1])  # <tf.Variable 'dense/kernel:0' shape=(3, 3) dtype=float32_ref>

How can I use Tensorflow Dense with float16 parameters?

Answer 1

I figured out a way to do this, that is to set the input data type of the dense layer to tf.float16

import tensorflow as tf
A = tf.get_variable(name='foo', shape=[3, 3], dtype=tf.float16)
dense = tf.layers.dense(inputs=A, units=3)
varis = tf.trainable_variables(scope=None)
print(varis[1])  # <tf.Variable 'dense/kernel:0' shape=(3, 3) dtype=float16_ref>