[英]tensorflow: how to interleave columns of two tensors (e.g. using tf.scatter_nd)?
I've read the tf.scatter_nd documentation and run the example code for 1D and 3D tensors... and now I'm trying to do it for a 2D tensor.我已经阅读了tf.scatter_nd 文档并运行了 1D 和 3D 张量的示例代码......现在我正在尝试为 2D 张量执行此操作。 I want to 'interleave' the columns of two tensors.
我想“交错”两个张量的列。 For 1D tensors, one can do this via
对于一维张量,可以通过
'''
We want to interleave elements of 1D tensors arr1 and arr2, where
arr1 = [10, 11, 12]
arr2 = [1, 2, 3, 4, 5, 6]
such that
desired result = [1, 2, 10, 3, 4, 11, 5, 6, 12]
'''
import tensorflow as tf
with tf.Session() as sess:
updates1 = tf.constant([1,2,3,4,5,6])
indices1 = tf.constant([[0], [1], [3], [4], [6], [7]])
shape = tf.constant([9])
scatter1 = tf.scatter_nd(indices1, updates1, shape)
updates2 = tf.constant([10,11,12])
indices2 = tf.constant([[2], [5], [8]])
scatter2 = tf.scatter_nd(indices2, updates2, shape)
result = scatter1 + scatter2
print(sess.run(result))
(aside: is there a better way to do this? I'm all ears.) (旁白:有没有更好的方法来做到这一点?我全神贯注。)
This gives the output这给出了输出
[ 1 2 10 3 4 11 5 6 12]
Yay!好极了! that worked!
那行得通!
Now lets' try to extend this to 2D.现在让我们尝试将其扩展到 2D。
'''
We want to interleave the *columns* (not rows; rows would be easy!) of
arr1 = [[1,2,3,4,5,6],[1,2,3,4,5,6],[1,2,3,4,5,6]]
arr2 = [[10 11 12], [10 11 12], [10 11 12]]
such that
desired result = [[1,2,10,3,4,11,5,6,12],[1,2,10,3,4,11,5,6,12],[1,2,10,3,4,11,5,6,12]]
'''
updates1 = tf.constant([[1,2,3,4,5,6],[1,2,3,4,5,6],[1,2,3,4,5,6]])
indices1 = tf.constant([[0], [1], [3], [4], [6], [7]])
shape = tf.constant([3, 9])
scatter1 = tf.scatter_nd(indices1, updates1, shape)
This gives the error ValueError: The outer 1 dimensions of indices.shape=[6,1] must match the outer 1 dimensions of updates.shape=[3,6]: Dimension 0 in both shapes must be equal, but are 6 and 3. Shapes are [6] and [3]. for 'ScatterNd_2' (op: 'ScatterNd') with input shapes: [6,1], [3,6], [2].
这给出了错误
ValueError: The outer 1 dimensions of indices.shape=[6,1] must match the outer 1 dimensions of updates.shape=[3,6]: Dimension 0 in both shapes must be equal, but are 6 and 3. Shapes are [6] and [3]. for 'ScatterNd_2' (op: 'ScatterNd') with input shapes: [6,1], [3,6], [2].
ValueError: The outer 1 dimensions of indices.shape=[6,1] must match the outer 1 dimensions of updates.shape=[3,6]: Dimension 0 in both shapes must be equal, but are 6 and 3. Shapes are [6] and [3]. for 'ScatterNd_2' (op: 'ScatterNd') with input shapes: [6,1], [3,6], [2].
Seems like my indices
is specifying row indices instead of column indices, and given the way that arrays are "connected" in numpy and tensorflow (ie row-major order), does that mean I need to explicitly specify every single pair of indices for every element in updates1
?似乎我的
indices
指定的是行索引而不是列索引,并且考虑到数组在 numpy 和 tensorflow 中“连接”的方式(即行主序),这是否意味着我需要为每个索引明确指定每对索引? updates1
元素? Or is there some kind of 'wildcard' specification I can use for the rows?或者我可以对行使用某种“通配符”规范吗? (Note
indices1 = tf.constant([[:,0], [:,1], [:,3], [:,4], [:,6], [:,7]])
gives syntax errors, as it probably should.) (注意
indices1 = tf.constant([[:,0], [:,1], [:,3], [:,4], [:,6], [:,7]])
给出语法错误,正如它可能应该的那样。)
Would it be easier to just do a transpose, interleave the rows, then transpose back?只是进行转置,交错行,然后转回会更容易吗? Because I tried that...
因为我试过了...
scatter1 = tf.scatter_nd(indices1, tf.transpose(updates1), tf.transpose(shape))
print(sess.run(tf.transpose(scatter1)))
...and got a much longer error message, that I don't feel like posting unless someone requests it. ......并获得更长的错误信息,说我不喜欢发帖,除非有人请求它。
PS- I searched to make sure this isn't a duplicate -- I find it hard to imagine that someone else hasn't asked this before -- but turned up nothing. PS-我搜索以确保这不是重复的-我发现很难想象其他人以前没有问过这个问题-但什么也没找到。
This is pure slicing but I didn't know that syntax like arr1[0:,:][:,:2]
actually works.这是纯粹的切片,但我不知道像
arr1[0:,:][:,:2]
这样的语法实际上有效。 It seems it does but not sure if it is better.似乎有,但不确定是否更好。
This may be the wildcard slicing mechanism you are looking for.这可能是您正在寻找的通配符切片机制。
arr1 = tf.constant([[1,2,3,4,5,6],[1,2,3,4,5,7],[1,2,3,4,5,8]])
arr2 = tf.constant([[10, 11, 12], [10, 11, 12], [10, 11, 12]])
with tf.Session() as sess :
sess.run( tf.global_variables_initializer() )
print(sess.run(tf.concat([arr1[0:,:][:,:2], arr2[0:,:] [:,:1],
arr1[0:,:][:,2:4],arr2[0:, :][:, 1:2],
arr1[0:,:][:,4:6],arr2[0:, :][:, 2:3]],axis=1)))
Output is输出是
[[ 1 2 10 3 4 11 5 6 12]
[ 1 2 10 3 4 11 5 7 12]
[ 1 2 10 3 4 11 5 8 12]]
So, for example,所以,例如,
arr1[0:,:]
returns arr1[0:,:]
返回
[[1 2 3 4 5 6]
[1 2 3 4 5 7]
[1 2 3 4 5 8]]
and arr1[0:,:][:,:2]
returns the first two columns和
arr1[0:,:][:,:2]
返回前两列
[[1 2]
[1 2]
[1 2]]
axis is 1.轴为 1。
Some moderators might have regarded my question as a duplicate of this one , not because the questions are the same, but only because the answers contain parts one can use to answer this question -- ie specifying every index combination by hand.一些版主可能认为我的问题是这个问题的重复,不是因为问题是相同的,而只是因为答案包含可以用来回答这个问题的部分——即手动指定每个索引组合。
A totally different method would be to multiply by a permutation matrix as shown in the last answer to this question .一个完全不同的方法是乘以一个置换矩阵,如该问题的最后一个答案所示。 Since my original question was about scatter_nd, I'm going to post this solution but wait to see what other answers come in... (Alternatively, I or someone could edit the question to make it about reordering columns, not specific to scatter_nd --EDIT: I have just edited the question title to reflect this ).
由于我原来的问题是关于scatter_nd,我要去发布此解决方案,但等待,看看有什么其他的答案进来......(另外,我还是有人可以编辑的问题,使之约重排序列,不是针对scatter_nd - - 编辑:我刚刚编辑了问题标题以反映这一点)。
Here, we concatenate the two different arrays/tensors...在这里,我们连接两个不同的数组/张量......
import numpy as np
import tensorflow as tf
sess = tf.Session()
# the ultimate application is for merging variables which should be in groups,
# e.g. in this example, [1,2,10] is a group of 3, and there are 3 groups of 3
n_groups = 3
vars_per_group = 3 # once the single value from arr2 (below) is included
arr1 = 10+tf.range(n_groups, dtype=float)
arr1 = tf.stack((arr1,arr1,arr1),0)
arr2 = 1+tf.range(n_groups * (vars_per_group-1), dtype=float)
arr2 = tf.stack((arr2,arr2,arr2),0)
catted = tf.concat((arr1,arr2),1) # concatenate the two arrays together
print("arr1 = \n",sess.run(arr1))
print("arr2 = \n",sess.run(arr2))
print("catted = \n",sess.run(catted))
Which gives output这给出了输出
arr1 =
[[10. 11. 12.]
[10. 11. 12.]
[10. 11. 12.]]
arr2 =
[[1. 2. 3. 4. 5. 6.]
[1. 2. 3. 4. 5. 6.]
[1. 2. 3. 4. 5. 6.]]
catted =
[[10. 11. 12. 1. 2. 3. 4. 5. 6.]
[10. 11. 12. 1. 2. 3. 4. 5. 6.]
[10. 11. 12. 1. 2. 3. 4. 5. 6.]]
Now we build the permutation matrix and multiply...现在我们构建置换矩阵并乘...
start_index = 2 # location of where the interleaving begins
# cml = "column map list" is the list of where each column will get mapped to
cml = [start_index + x*(vars_per_group) for x in range(n_groups)] # first array
for i in range(n_groups): # second array
cml += [x + i*(vars_per_group) for x in range(start_index)] # vars before start_index
cml += [1 + x + i*(vars_per_group) + start_index \
for x in range(vars_per_group-start_index-1)] # vars after start_index
print("\n cml = ",cml,"\n")
# Create a permutation matrix using p
np_perm_mat = np.zeros((len(cml), len(cml)))
for idx, i in enumerate(cml):
np_perm_mat[idx, i] = 1
perm_mat = tf.constant(np_perm_mat,dtype=float)
result = tf.matmul(catted, perm_mat)
print("result = \n",sess.run(result))
Which gives output这给出了输出
cml = [2, 5, 8, 0, 1, 3, 4, 6, 7]
result =
[[ 1. 2. 10. 3. 4. 11. 5. 6. 12.]
[ 1. 2. 10. 3. 4. 11. 5. 6. 12.]
[ 1. 2. 10. 3. 4. 11. 5. 6. 12.]]
Even though this doesn't use scatter_nd as the original question asked, one thing I like about this is, you can allocate the perm_mat
once in some __init__()
method, and hang on to it, and after that initial overhead it's just matrix-matrix multiplication by a sparse, constant matrix, which should be pretty fast.尽管这不使用 scatter_nd 作为原始问题,但我喜欢的一件事是,您可以在某些
__init__()
方法中分配perm_mat
一次,并坚持下去,在初始开销之后它只是矩阵 -矩阵乘以稀疏的常量矩阵,这应该非常快。 (?) (?)
Still happy to wait and see what other answers might come in.仍然很高兴等待,看看其他答案可能会出现。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.