[英]Basic 1d convolution in tensorflow
好的,我想在Tensorflow中對時間序列數據進行一維卷積。 根據這些 票證和手冊 ,顯然可以使用tf.nn.conv2d
支持此操作 。 唯一的要求是設置strides=[1,1,1,1]
。 聽起來很簡單!
但是,即使在非常小的測試用例中,我也無法弄清楚該如何做。 我究竟做錯了什么?
讓我們進行設置。
import tensorflow as tf
import numpy as np
print(tf.__version__)
>>> 0.9.0
好的,現在在兩個小的數組上生成基本的卷積測試。 我將通過使用1的批量大小來簡化該操作,並且由於時間序列是一維的,因此我的“圖像高度”將為1。並且由於它是單變量時間序列,因此顯然“通道”的數量也是1,所以這很簡單,對吧?
g = tf.Graph()
with g.as_default():
# data shape is "[batch, in_height, in_width, in_channels]",
x = tf.Variable(np.array([0.0, 0.0, 0.0, 0.0, 1.0]).reshape(1,1,-1,1), name="x")
# filter shape is "[filter_height, filter_width, in_channels, out_channels]"
phi = tf.Variable(np.array([0.0, 0.5, 1.0]).reshape(1,-1,1,1), name="phi")
conv = tf.nn.conv2d(
phi,
x,
strides=[1, 1, 1, 1],
padding="SAME",
name="conv")
繁榮。 錯誤。
ValueError: Dimensions 1 and 5 are not compatible
好的,首先,我不知道在任何維度上該如何發生,因為我已經指定要在卷積OP中填充參數。
但是很好,也許對此有限制。 我一定對文檔感到困惑,並且在張量的錯誤軸上設置了這種卷積。 我將嘗試所有可能的排列:
for i in range(4):
for j in range(4):
shape1 = [1,1,1,1]
shape1[i] = -1
shape2 = [1,1,1,1]
shape2[j] = -1
x_array = np.array([0.0, 0.0, 0.0, 0.0, 1.0]).reshape(*shape1)
phi_array = np.array([0.0, 0.5, 1.0]).reshape(*shape2)
try:
g = tf.Graph()
with g.as_default():
x = tf.Variable(x_array, name="x")
phi = tf.Variable(phi_array, name="phi")
conv = tf.nn.conv2d(
x,
phi,
strides=[1, 1, 1, 1],
padding="SAME",
name="conv")
init_op = tf.initialize_all_variables()
sess = tf.Session(graph=g)
sess.run(init_op)
print("SUCCEEDED!", x_array.shape, phi_array.shape, conv.eval(session=sess))
sess.close()
except Exception as e:
print("FAILED!", x_array.shape, phi_array.shape, type(e), e.args or e._message)
結果:
FAILED! (5, 1, 1, 1) (3, 1, 1, 1) <class 'ValueError'> ('Filter must not be larger than the input: Filter: (3, 1) Input: (1, 1)',)
FAILED! (5, 1, 1, 1) (1, 3, 1, 1) <class 'ValueError'> ('Filter must not be larger than the input: Filter: (1, 3) Input: (1, 1)',)
FAILED! (5, 1, 1, 1) (1, 1, 3, 1) <class 'ValueError'> ('Dimensions 1 and 3 are not compatible',)
FAILED! (5, 1, 1, 1) (1, 1, 1, 3) <class 'tensorflow.python.framework.errors.InvalidArgumentError'> No OpKernel was registered to support Op 'Conv2D' with these attrs
[[Node: conv = Conv2D[T=DT_DOUBLE, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](x/read, phi/read)]]
FAILED! (1, 5, 1, 1) (3, 1, 1, 1) <class 'tensorflow.python.framework.errors.InvalidArgumentError'> No OpKernel was registered to support Op 'Conv2D' with these attrs
[[Node: conv = Conv2D[T=DT_DOUBLE, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](x/read, phi/read)]]
FAILED! (1, 5, 1, 1) (1, 3, 1, 1) <class 'ValueError'> ('Filter must not be larger than the input: Filter: (1, 3) Input: (5, 1)',)
FAILED! (1, 5, 1, 1) (1, 1, 3, 1) <class 'ValueError'> ('Dimensions 1 and 3 are not compatible',)
FAILED! (1, 5, 1, 1) (1, 1, 1, 3) <class 'tensorflow.python.framework.errors.InvalidArgumentError'> No OpKernel was registered to support Op 'Conv2D' with these attrs
[[Node: conv = Conv2D[T=DT_DOUBLE, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](x/read, phi/read)]]
FAILED! (1, 1, 5, 1) (3, 1, 1, 1) <class 'ValueError'> ('Filter must not be larger than the input: Filter: (3, 1) Input: (1, 5)',)
FAILED! (1, 1, 5, 1) (1, 3, 1, 1) <class 'tensorflow.python.framework.errors.InvalidArgumentError'> No OpKernel was registered to support Op 'Conv2D' with these attrs
[[Node: conv = Conv2D[T=DT_DOUBLE, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](x/read, phi/read)]]
FAILED! (1, 1, 5, 1) (1, 1, 3, 1) <class 'ValueError'> ('Dimensions 1 and 3 are not compatible',)
FAILED! (1, 1, 5, 1) (1, 1, 1, 3) <class 'tensorflow.python.framework.errors.InvalidArgumentError'> No OpKernel was registered to support Op 'Conv2D' with these attrs
[[Node: conv = Conv2D[T=DT_DOUBLE, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](x/read, phi/read)]]
FAILED! (1, 1, 1, 5) (3, 1, 1, 1) <class 'ValueError'> ('Dimensions 5 and 1 are not compatible',)
FAILED! (1, 1, 1, 5) (1, 3, 1, 1) <class 'ValueError'> ('Dimensions 5 and 1 are not compatible',)
FAILED! (1, 1, 1, 5) (1, 1, 3, 1) <class 'ValueError'> ('Dimensions 5 and 3 are not compatible',)
FAILED! (1, 1, 1, 5) (1, 1, 1, 3) <class 'ValueError'> ('Dimensions 5 and 1 are not compatible',)
嗯 好的,看來現在有兩個問題。 首先,我猜ValueError
是關於沿錯誤的軸應用過濾器的,盡管有兩種形式。
但是,隨后我可以沿其應用過濾器的軸也令人困惑-請注意,它實際上構造了具有輸入形狀(5,1,1,1,1)和過濾器形狀(1,1,1,3)的圖。 根據文檔中的AFAICT,這應該是一個過濾器,以批次為例,一個“像素”和一個“通道”,輸出3個“通道”。 那么,當其他人不起作用時,為什么一個人起作用呢?
無論如何,有時在構建圖形時它不會失敗。 有時它會構造圖; 然后我們得到tensorflow.python.framework.errors.InvalidArgumentError
。 從一些令人困惑的github票證中,我收集到這可能是由於 我在CPU而不是GPU上運行的事實,反之亦然 卷積運算僅針對32位浮點數而不是64位浮點數進行定義的事實。 如果任何人都可以扔在哪個軸我應該對准什么上,為了與卷積內核時間序列一些輕,我會非常感激。
抱歉,您的第一個代碼幾乎是正確的。 您只是在tf.nn.conv2d
反轉了x
和phi
:
g = tf.Graph()
with g.as_default():
# data shape is "[batch, in_height, in_width, in_channels]",
x = tf.Variable(np.array([0.0, 0.0, 0.0, 0.0, 1.0]).reshape(1, 1, 5, 1), name="x")
# filter shape is "[filter_height, filter_width, in_channels, out_channels]"
phi = tf.Variable(np.array([0.0, 0.5, 1.0]).reshape(1, 3, 1, 1), name="phi")
conv = tf.nn.conv2d(
x,
phi,
strides=[1, 1, 1, 1],
padding="SAME",
name="conv")
更新:從版本r0.11開始,TensorFlow現在使用tf.nn.conv1d
支持一維卷積。 我以前在粘貼到此處的stackoverflow文檔(現已不存在)中做了一個使用它們的指南:
考慮一個輸入長度為10
且尺寸為16
的基本示例。 批處理大小為32
。 因此,我們有一個占位符,其輸入形狀為[batch_size, 10, 16]
。
batch_size = 32
x = tf.placeholder(tf.float32, [batch_size, 10, 16])
然后,我們創建一個寬度為3的過濾器,並以16
通道作為輸入,並輸出16
通道。
filter = tf.zeros([3, 16, 16]) # these should be real values, not 0
最后,我們在tf.nn.conv1d
使用一個步幅和一個填充:- 步幅 :整數s
填充 :就像在2D中一樣,您可以在SAME
和VALID
之間進行選擇。 SAME
將輸出相同的輸入長度,而VALID
將不添加零填充。
在我們的示例中,跨度為2,有效填充為空白。
output = tf.nn.conv1d(x, filter, stride=2, padding="VALID")
輸出形狀應為[batch_size, 4, 16]
。
使用padding="SAME"
,我們的輸出形狀為[batch_size, 5, 16]
。
在新版本的TF(從0.11開始)中,您具有conv1d ,因此無需使用2d卷積來進行1d卷積。 這是有關如何使用conv1d的簡單示例:
import tensorflow as tf
i = tf.constant([1, 0, 2, 3, 0, 1, 1], dtype=tf.float32, name='i')
k = tf.constant([2, 1, 3], dtype=tf.float32, name='k')
data = tf.reshape(i, [1, int(i.shape[0]), 1], name='data')
kernel = tf.reshape(k, [int(k.shape[0]), 1, 1], name='kernel')
res = tf.squeeze(tf.nn.conv1d(data, kernel, stride=1, padding='VALID'))
with tf.Session() as sess:
print sess.run(res)
要了解conv1d是如何計算的,請看各種示例
我認為我可以滿足我的需求。 代碼的注釋/詳細信息在代碼上:
import numpy as np
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
task_name = 'task_MNIST_flat_auto_encoder'
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
X_train, Y_train = mnist.train.images, mnist.train.labels # N x D
X_cv, Y_cv = mnist.validation.images, mnist.validation.labels
X_test, Y_test = mnist.test.images, mnist.test.labels
# data shape is "[batch, in_height, in_width, in_channels]",
# X_train = N x D
N, D = X_train.shape
# think of it as N images with height 1 and width D.
X_train = X_train.reshape(N,1,D,1)
x = tf.placeholder(tf.float32, shape=[None,1,D,1], name='x-input')
#x = tf.Variable( X_train , name='x-input')
# filter shape is "[filter_height, filter_width, in_channels, out_channels]"
filter_size, nb_filters = 10, 12 # filter_size , number of hidden units/units
# think of it as having nb_filters number of filters, each of size filter_size
W = tf.Variable( tf.truncated_normal(shape=[1, filter_size, 1,nb_filters], stddev=0.1) )
stride_convd1 = 2 # controls the stride for 1D convolution
conv = tf.nn.conv2d(input=x, filter=W, strides=[1, 1, stride_convd1, 1], padding="SAME", name="conv")
with tf.Session() as sess:
sess.run( tf.initialize_all_variables() )
sess.run(fetches=conv, feed_dict={x:X_train})
感謝Olivier的幫助(有關進一步的說明,請參見他的評論中的討論)。
手動檢查:
X_train_org = np.array([[0,1,2,3]])
N, D = X_train_org.shape
X_train_1d = X_train_org.reshape(N,1,D,1)
#X_train = tf.constant( X_train_org )
# think of it as N images with height 1 and width D.
xx = tf.placeholder(tf.float32, shape=[None,1,D,1], name='xx-input')
#x = tf.Variable( X_train , name='x-input')
# filter shape is "[filter_height, filter_width, in_channels, out_channels]"
filter_size, nb_filters = 2, 2 # filter_size , number of hidden units/units
# think of it as having nb_filters number of filters, each of size filter_size
filter_w = np.array([[1,3],[2,4]]).reshape(1,filter_size,1,nb_filters)
#W = tf.Variable( tf.truncated_normal(shape=[1,filter_size,1,nb_filters], stddev=0.1) )
W = tf.Variable( tf.constant(filter_w, dtype=tf.float32) )
stride_convd1 = 2 # controls the stride for 1D convolution
conv = tf.nn.conv2d(input=xx, filter=W, strides=[1, 1, stride_convd1, 1], padding="SAME", name="conv")
#C = tf.constant( (np.array([[4,3,2,1]]).T).reshape(1,1,1,4) , dtype=tf.float32 ) #
#tf.reshape( conv , [])
#y_tf = tf.matmul(conv, C)
##
x = tf.placeholder(tf.float32, shape=[None,D], name='x-input') # N x 4
W1 = tf.Variable( tf.constant( np.array([[1,2,0,0],[3,4,0,0]]).T, dtype=tf.float32 ) ) # 2 x 4
y1 = tf.matmul(x,W1) # N x 2 = N x 4 x 4 x 2
W2 = tf.Variable( tf.constant( np.array([[0,0,1,2],[0,0,3,4]]).T, dtype=tf.float32 ))
y2 = tf.matmul(x,W2) # N x 2 = N x 4 x 4 x 2
C1 = tf.constant( np.array([[4,3]]).T, dtype=tf.float32 ) # 1 x 2
C2 = tf.constant( np.array([[2,1]]).T, dtype=tf.float32 )
p1 = tf.matmul(y1,C1)
p2 = tf.matmul(y2,C2)
y = p1 + p2
with tf.Session() as sess:
sess.run( tf.initialize_all_variables() )
print 'manual conv'
print sess.run(fetches=y1, feed_dict={x:X_train_org})
print sess.run(fetches=y2, feed_dict={x:X_train_org})
#print sess.run(fetches=y, feed_dict={x:X_train_org})
print 'tf conv'
print sess.run(fetches=conv, feed_dict={xx:X_train_1d})
#print sess.run(fetches=y_tf, feed_dict={xx:X_train_1d})
輸出:
manual conv
[[ 2. 4.]]
[[ 8. 18.]]
tf conv
[[[[ 2. 4.]
[ 8. 18.]]]]
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.