[英]Transforming a 3 Column Matrix into an N x N Matrix in Numpy
I have a 2D numpy
array with 3 columns. 我有3列的2D numpy
数组。 Columns 1 and 2 are a list of connections between ID's. 第1列和第2列是ID之间的连接列表。 Column 3 is a the strength of that connection. 第3栏是这种联系的力量。 I would like to transform this 3 column matrix into a weighted adjacency matrix (an N x N matrix where cells represent the strength of connection between each ID). 我想将此3列矩阵转换为加权邻接矩阵 (N x N矩阵,其中单元代表每个ID之间的连接强度)。
I have already done this in my code below. 我已经在下面的代码中完成了此操作。 matrix
is the 3 column 2D array and t1
is the weighted adjacency matrix. matrix
是3列2D数组, t1
是加权邻接矩阵。 My problem is this code is very slow because I am using nested for loops. 我的问题是这段代码非常慢,因为我使用嵌套的for循环。 I am familiar with the pandas
function melt
which does this, but I am not able to use pandas
. 我对执行此操作的pandas
函数melt
,但是我不能使用pandas
。 Is there a faster implementation not using pandas
? 是否有不使用pandas
的更快实现?
import numpy as np
a = np.arange(2000)
np.random.shuffle(a)
b = np.arange(2000)
np.random.shuffle(b)
c = np.random.rand(2000,1)
matrix = np.column_stack((a,b,c))
#get unique value list of nm
flds = list(np.unique(matrix[:,0]))
flds.extend(list(np.unique(matrix[:,1])))
flds = np.asarray(flds)
flds = np.unique(flds)
#make lookup dict
lookup = dict(zip(np.arange(0,len(flds)), flds))
lookup_rev = dict(zip(flds, np.arange(0,len(flds))))
#make empty n by n matrix with unique lists
t1 = np.zeros([len(flds) , len(flds)])
#map values into the n by n matrix and make the rest 0
'''this takes a long time to run'''
#iterate through rows
for i in np.arange(0,len(lookup)):
#iterate through columns
for k in np.arange(0,len(lookup)):
val = matrix[(matrix[:,0] == lookup[i]) & (matrix[:,1] == lookup[k])][:,2]
if val:
t1[i,k] = sum(val)
Assuming that I understood the question correctly and that val
is a scalar, you could use a vectorized approach that involves initializing with zeros
and then indexing
, like so - 假设我正确理解了这个问题,并且val
是一个标量,则可以使用矢量化方法,该方法涉及用zeros
初始化,然后进行indexing
,如下所示:
out = np.zeros((len(flds),len(flds)))
out[matrix[:,0].astype(int),matrix[:,1].astype(int)] = matrix[:,2]
Please note that by my observation it looks like you can avoid using lookup
. 请注意,根据我的观察,您似乎可以避免使用lookup
。
The main acceleration you can get is by not iterating through each element of the NxN matrix but instead iterate trough your connection list, which is much smaller. 您可以获得的主要加速效果是,不对NxN矩阵的每个元素进行迭代,而是对连接列表进行迭代,该连接列表要小得多。
I tried to simplify your code a bit. 我试图简化您的代码。 It use the list.index
method, which can be slow, but it should still be faster that what you had. 它使用list.index
方法,该方法可能很慢,但仍应比您拥有的方法快。
import numpy as np
a = np.arange(2000)
np.random.shuffle(a)
b = np.arange(2000)
np.random.shuffle(b)
c = np.random.rand(2000,1)
matrix = np.column_stack((a,b,c))
lookup = np.unique(matrix[:,:2]).tolist() # You can call unique only once
t1 = np.zeros((len(lookup),len(lookup)))
for i,j,val in matrix:
t1[lookup.index(i),lookup.index(j)] = val # Fill the matrix
You need to iterate your matrix only once: 您只需要迭代一次矩阵:
import numpy as np
size = 2000
a = np.arange(size)
np.random.shuffle(a)
b = np.arange(size)
np.random.shuffle(b)
c = np.random.rand(size,1)
matrix = np.column_stack((a,b,c))
#get unique value list of nm
fields = np.unique(matrix[:,:2])
n = len(fields)
#make reverse lookup dict
lookup = dict(zip(fields, range(n)))
#make empty n by n matrix
t1 = np.zeros([n, n])
for src, dest, val in matrix:
i = lookup[src]
j = lookup[dest]
t1[i, j] += val
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.