试图从 python 字典创建一个二维数组

Question

I am trying to create a 2D array from dictionary in python.

mydictionary={
'a':['review','read','study'],
'b':['read'],
'c':['review','dictate']}

I want to have a 2D array that shows the number of items matching.(ie compare the keys and their values and store the matching values in a 2D array)

Output Format:

       a       b       c
  ___|___________________
  a  |  3       1       1
     |
  b  |  1       1       0
     |
  c  |  1       0       2

My dictionary has around 7000 items. What is the best way to achieve this? Thanks

Answer 1

A sweet way to obtain result is to use pandas , the numpy big brother :

In [6]: md=mydictionary
In [7]: df=pd.DataFrame([[len(set(md[i])&set(md[j])) for j in md] for i in md],md,md)
In [8]: df
Out[8]: 
   c  a  b
c  2  1  0
a  1  3  1
b  0  1  1

If order matter :

In [9]: df.sort_index(0).sort_index(1)
Out[9]: 
   a  b  c
a  3  1  1
b  1  1  0
c  1  0  2

Answer 2

For starters, you can use the fact that the diagonals are just the lengths of the individual lists.

Then the matrix is perfectly symmetric, so you only need to compute the value for (a,b) and not both (a,b), (b,a)

Past that, you can compute the size of their intersection for each pair:

len([filter(lambda x: x in arr1, subArr) for subArr in arr2])

Answer 3

You can form the list however you like but forming the sets first will be faster than repeatedly creating sets:

new = {k: set(v) for k, v in mydictionary.items()}
out = OrderedDict()
for k, v in new.items():
    out[k] = [k, len(v)]
    for k2, v2 in new.items():
        if k2 == k:
            continue
        out[k].append(sum(val in v for val in v2))


print(list(out.values()))

Output:

[['a', 3, 1, 1], ['c', 2, 1, 0], ['b', 1, 1, 0]]

Answer 4

The other solutions offered here are suitable for smaller lists of inputs, but as the list grows they will scale as O[N^2] (at best) which may be relatively slow in your case. Here's an approach using scikit-learn's DictVectorizer that should be faster for large inputs with small amounts of overlap.

The idea is to construct a one-hot encoding of the input, and then use a matrix product to compute the final result:

from sklearn.feature_extraction import DictVectorizer

keys, vals = zip(*mydictionary.items())
valsdict = [dict(zip(val, repeat(1))) for val in vals]

V = DictVectorizer().fit_transform(valsdict)
result = V.dot(V.T)

The result will be a scipy.sparse matrix, which only explicitly stores nonzero elements. You can convert it to a dense array form with result.toarray() ; using pandas you can also apply the labels to the rows and columns:

import pandas as pd
pd.DataFrame(result.toarray(), keys, keys)
#    a  c  b
# a  3  1  1
# c  1  2  0
# b  1  0  1

I expect this will be significantly faster than the other solutions posted here as the size of the inputs grow.

---

Edit: here's a benchmark on a 1000-item input where about half of the pairs have some overlap:

import numpy as np
import pandas as pd
from itertools import repeat
from sklearn.feature_extraction import DictVectorizer

def dense_method(md):
    return pd.DataFrame([[len(set(md[i]) & set(md[j]))
                          for j in md]
                         for i in md], md, md)

def sparse_method(mydictionary):
    keys, vals = zip(*mydictionary.items())
    valsdict = [dict(zip(val, repeat(1))) for val in vals]
    V = DictVectorizer().fit_transform(valsdict)
    return pd.DataFrame(V.dot(V.T).toarray(), keys, keys)


mydictionary = {i: np.random.randint(0, 20, 3)
                for i in range(1000)}

print(np.allclose(dense_method(mydictionary),
                  sparse_method(mydictionary)))
# True

%timeit sparse_method(mydictionary)
# 100 loops, best of 3: 19.5 ms per loop

%timeit dense_method(mydictionary)
# 1 loops, best of 3: 3.41 s per loop

The sparse method is two orders of magnitude faster here.

Answer 5

Surely not the most elegant way to perform that task, but it works

import numpy as np
N = len(mydictionary)
freqs = np.zeros(shape=(N, N), dtype=np.int)
mykeys = sorted(mydictionary.keys())
for i, x in enumerate(mykeys):
    freqs[i, i] = len(mydictionary[x])
    for j in range(i+1, N):
        for elem in mydictionary[x]:
            if elem in mydictionary[mykeys[j]]:
                freqs[i, j] += 1
                freqs[j, i] += 1
print freqs
#[[3 1 1]
# [1 2 0]
# [1 0 1]]