![](/img/trans.png)
[英]How to use TensorFlow LSTM tutorial for character-level language modeling?
[英]Tensorflow character-level CNN - input shape
我正在嘗試將2個堆棧的字符級CNN添加到一個較大的神經網絡系統中,但是在輸入維數時出現ValueError。
我想要實現的是通過替換字符(根據大寫字母,數字或字母)並將輸入的字符輸入CNN中來獲得輸入字的正交表示。 我知道可以使用LSTM / RNN來實現,但是要求表明使用CNN,因此使用其他NN並不是可選的。
那里的大多數示例自然使用圖像數據集(MNIST等),而不使用文本數據集。 因此,我感到困惑,不確定如何對字符嵌入進行“重塑”,以使它們可以成為CNN的有效輸入。
因此,這是我嘗試運行的代碼的一部分:
# ...
# shape = (batch size, max length of sentence, max length of word)
self.char_ids = tf.placeholder(tf.int32, shape=[None, None, None],
name="char_ids")
# ...
# Char embedding lookup
_char_embeddings = tf.get_variable(
name="_char_embeddings",
dtype=tf.float32,
shape=[self.config.nchars, self.config.dim_char])
char_embeddings = tf.nn.embedding_lookup(_char_embeddings,
self.char_ids, name="char_embeddings")
# Reshape for CNN?
s = tf.shape(char_embeddings)
char_embeddings = tf.reshape(char_embeddings, shape=[s[0]*s[1], self.config.dim_char, s[2]])
# Conv #1
conv1 = tf.layers.conv1d(
inputs=char_embeddings,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
# Conv #2
conv2 = tf.layers.conv1d(
inputs=conv1,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
pool2 = tf.layers.max_pooling1d(inputs=conv2, pool_size=2, strides=2)
# Dense Layer
output = tf.layers.dense(inputs=pool2, units=32, activation=tf.nn.relu)
# ...
這是我得到的錯誤:
File "/home/emre/blstm-crf-ner/model/ner_model.py", line 159, in add_word_embeddings_op activation=tf.nn.relu)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/convolutional.py", line 411, in conv1d return layer.apply(inputs)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/base.py", line 809, in apply return self.__call__(inputs, *args, **kwargs)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/base.py", line 680, in __call__ self.build(input_shapes)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/convolutional.py", line 132, in build raise ValueError('The channel dimension of the inputs '
ValueError: The channel dimension of the inputs should be defined. Found `None`.
任何幫助,將不勝感激。
謝謝。
更新
所以通過一些博客文章下面后1 , 2 ,感謝維傑男,我明白,我們必須提供輸入尺寸事先(與提供sequence_length
s的RNN / LSTM)。 所以這是最終的代碼片段:
# Char embedding lookup
_char_embeddings = tf.get_variable(
name="_char_embeddings",
dtype=tf.float32,
shape=[self.config.nchars, self.config.dim_char])
char_embeddings = tf.nn.embedding_lookup(_char_embeddings,
self.char_ids, name="char_embeddings")
# max_len_of_word: 20
# Just pad shorter words and truncate the longer ones.
s = tf.shape(char_embeddings)
char_embeddings = tf.reshape(char_embeddings, shape=[-1, self.config.dim_char, self.config.max_len_of_word])
# Conv #1
conv1 = tf.layers.conv1d(
inputs=char_embeddings,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
# Conv #2
conv2 = tf.layers.conv1d(
inputs=conv1,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
pool2 = tf.layers.max_pooling1d(inputs=conv2, pool_size=2, strides=2)
# Dense Layer
output = tf.layers.dense(inputs=pool2, units=32, activation=tf.nn.relu)
conv1d
期望在圖形創建期間定義通道尺寸。 因此,您不能將維度傳遞為None
。
您需要進行以下更改:
char_ids = tf.placeholder(tf.int32, shape=[None, max_len_sen, max_len_word],
name="char_ids")
#max_len_sen and max_len_word has to be set.
#Reshapping for CNN, should be
s = char_embeddings.get_shape()
char_embeddings = tf.reshape(char_embeddings, shape=[-1, dim_char, s[2]])
Conv1d中輸入的默認格式具有形狀(批,長度,通道),也許char_embeddings應該像這樣:
s = char_embeddings.get_shape()
char_embeddings = tf.reshape(char_embeddings, shape=[-1, s[2], dim_char])
謝謝!
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.