randomizing two lists(numpy in) and maintaining order in python

Question

I have two 2d numpy lists. I want to shuffle it, but just outer side shuffle.

If i randomize order list a, I want list b to follow list a's order.

I have seen randomizing two lists and maintaining order in python but this looks not work for me.

The below code is how I'm doing now.

But it's too slow for big numpy lists.

import numpy as np
import random    

a = np.array([[1,2,3], [4,5,6], [7,8,9], [10,11,12]])
b = np.array([[100,200,300,400,500], [600,700,800,900,901], [101,102,103,104,105], [501,502,503,504,505]])
r = [i for i in range(4)]
random.shuffle(r)
newa = np.empty((0, 3))
newb = np.empty((0, 5))
for rr in r:
    newa = np.append(newa, [a[rr]], axis=0)
    newb = np.append(newb, [b[rr]], axis=0)
print(newa)
print(newb)

Any pythonic or faster way to do this?

Thanks for answer.

Answer 1

You have the right idea, but appending to an array is very time consuming, since it reallocates the entire buffer every time. Instead, you can just use the shuffled index:

a = np.array([[1,2,3], [4,5,6], [7,8,9], [10,11,12]])
b = np.array([[100,200,300,400,500], [600,700,800,900,901], [101,102,103,104,105], [501,502,503,504,505]])

r = np.arange(4)
np.random.shuffle(r)

newa = a[r]
newb = b[r]

Answer 2

I am not 100% on performance but I think this may work better for you:

newa = np.copy(a)
np.random.shuffle(newa)
newb = np.copy(b)
np.random.shuffle(newb)

Do you need them to be in the same random order? So if element 1 moves to position 3 in newa should that move also occur in newb ? That is how your code is but you don't specify in your question. If you want the same shuffle for both you need to do something like this:

indexes = np.arrange(len(a))
np.random.shuffle(indexes)
newa = a[indexes]
newb = b[indexes]

This will go faster than what you have since all of the append calls are slowing you down.

Answer 3

Use the shuffle option in numpy itself, it would be much more efficient.

np.random.shuffle(a)
np.random.shuffle(b)

print(a)
#
[[ 4  5  6]
 [10 11 12]
 [ 7  8  9]
 [ 1  2  3]]
print(b)
#
[[600 700 800 900 901]
 [100 200 300 400 500]
 [501 502 503 504 505]
 [101 102 103 104 105]]