How can I create np.ndarray from four arrays of different lengths without using loops?

Question

I have 5 arrays, that I need to create two large arrays of shape ((1000,5000)) to use in regression analysis. I'm currently using the following code.

# data shapes are as follows
bbeta.shape = ((5000,2))
rand1.shape = ((1000))
rand2.shape = ((1000))
rand3.shape = ((1000))
depths.shape = ((1000))

# creating x and y for regression
for i in np.arange(0,1000,1):
    for j in np.arange(0,5000,1):
        y[i,j] = np.log((rand1[i] * (depths[i]/3500)**bbeta[j,0]))
        x[i,j] = np.log(rand2[i] + (bbeta[j,1] * rand3[i]))

This code takes ~30 seconds to run, which is fine I guess, but I need to run this over 1000 times to find the bootstrap standard error, which means my code will currently take > 8 hours. I've cut down the number created datasets significantly (from bbeta.shape = ((17801,2)), but it's not sufficient.

If I can stick with numpy, or at least if it's faster to convert from and back to numpy, as the rest of my code is written using numpy.

I wondered if anyone knew of a way to do the same thing as above but faster, as I'm aware that loops aren't computationally efficient. I've looked through stackoverflow but I couldn't find anyone that answers this question (at least in a way that I can recognise helps me).

Sorry if any of this isn't clear - I'm dyslexic and a new-ish to python. Any help anyone can give me would be greatly appreciated

Answer 1

You could take advantage of broadcasting and do the following:

y2 = np.power((depths / 3500)[:, np.newaxis], bbeta[:, 0])
y2 = np.log(rand1[:, np.newaxis] * y2)

x2 = bbeta[:, 1] * rand3[:, np.newaxis]
x2 = np.log(rand2[:, np.newaxis] + x2)

print(np.allclose(y, y2))
print(np.allclose(x, x2))

Output

True
True

The test data was generated with the following code:

bbeta = np.random.random((5000, 2))
rand1 = np.random.random((1000))
rand2 = np.random.random((1000))
rand3 = np.random.random((1000))
depths = np.random.random((1000))