加快图表中的采样

Question

I've been recently working with graph sampling in Python. My working example reads as:

for enx, wlen in enumerate(wlen_dist):
    for j in range(wlen):
        node_container = queue.Queue(maxsize=200000000)
        node_container.put(node_name)
        tmp_walk = [] # [node_name]
        while not node_container.empty():
            nod = node_container.get()
            neighs = list(network.neighbors(nod))
            tar = random.choice(neighs)
            node_container.put(tar)
            if len(tmp_walk) > enx+1:
                break
            tmp_walk.append(tar)
       some_container.append(tmp_walk)

where wlen is the number of samples of path of length enx, and I am simply saving the walks to some_container (not really important here). The wlen_dist is for example:

[1000,500,100]

and here, 1000 samples of a walk of length two, 500 of length 3 and 100 of length 4 are obtained. The networkx is a networkX graph. I was wondering, how does one speed up code like this (I am new to this part).

My ideas:

1. Use Numba and wrap individual walks into a method

2. Use Cython somehow

3. Rewrite it alltogether in C++ and call it somehow

I would be glad for any ideas and feedback, thanks!

Answer 1

One idea often used in graph embedding is the idea of reusing parts of random walks:

If you have a random walk visiting the nodes a_1, a_2, a_3 , you can regard this as one random walk of length 3 and 2 random walks of length 2 ( a_1, a_2 and a_2, a_3 ).

This can be generalised to longer walks, so your random walks of length 4 contain 2 random walks of length 3 and 3 random walks of length 2.