Why is `numpy` slower than python for left bit shifts?

Question

I am trying to do bit shifts on numpy integers (specifically, numpy.uint64 objects) and I need them to be fast. In my implementation below, I put the object in a numpy.array only because that's the only object that can accept bit left shifts. If there is any faster implementation I will accept it.

from timeit import timeit
print(timeit("a << 1", "a = int(2**60)"))
print(timeit("a << 1", "import numpy as np; a = np.array([2 ** 60], dtype=np.uint64)"))
print(timeit("np.left_shift(a, 1)", "import numpy as np; a = np.array([2 ** 60], dtype=np.uint64)"))

returns:

0.056681648000000084
1.208092987
1.1685176299999998

Why is python so much faster than numpy for this operation? Is there a way to get comparable speeds in numpy ?

Answer 1

About the performance difference, it seems logical: you're applying a vectorized shift on one element. There's a big overhead just reaching the shift part and changing the numpy struct. native code shifts just faster.

Okay, I googled the error message that you get when you try to do that on one element, which is:

>>> a = numpy.uint64(2**60)
>>> a << 3
Traceback (most recent call last):
  File "<string>", line 301, in runcode
  File "<interactive input>", line 1, in <module>
TypeError: ufunc 'left_shift' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

and I found this github issue: https://github.com/numpy/numpy/issues/2524

This is because the shift number is converted as a signed type and there is no signed integer type big enough to hold a uint64.

now a good workaround (as seen in this github issue comment ) is this:

a << numpy.uint64(1)

(maybe build the "1" constant once and for all and use it in all your code to save the object creation)