Fastest way to compute vector in Python

Question

I have the following (using Python's pandas):

y: n by 1 dataframe

x: n by k dataframe

theta: k by 1 dataframe

Each of the elements in the above dataframes contains a real number.

I need a dataframe w, where w = y'x (' denotes transpose), but w only contains the observations for which y multiplied element-wise by (x * theta) is less than 1. In other words, the dimension of w is at most n by k, and there will be fewer rows if there are some observations that do not meet the criteria.

What's the fastest way (in terms of time) to get w?

Answer 1

Use .values to access underlying numpy arrays

Y = y.values
X = x.values
Th = theta.values

W = Y.T.dot(X)

mask = Y * X.dot(Th) < 1

w = pd.DataFrame(W[mask], y.index[mask])