Timeit showing that regular python is faster than numpy?

Question

I'm working on a piece of code for a game that calculates the distances between all the objects on the screen using their in-game coordinate positions. Originally I was going to use basic Python and lists to do this, but since the number of distances that will need calculated will increase exponentially with the number of objects, I thought it might be faster to do it with numpy .

I'm not very familiar with numpy , and I've been experimenting on basic bits of code with it. I wrote a little bit of code to time how long it takes for the same function to complete a calculation in numpy and in regular Python, and numpy seems to consistently take a good bit more time than the regular python.

The function is very simple. It starts with 1.1 and then increments 200,000 times, adding 0.1 to the last value and then finding the square root of the new value. It's not what I'll actually be doing in the game code, which will involve finding total distance vectors from position coordinates; it's just a quick test I threw together. I already read here that the initialization of arrays takes more time in NumPy, so I moved the initializations of both the numpy and python arrays outside their functions, but Python is still faster than numpy .

Here is the bit of code:

#!/usr/bin/python3

import numpy
from timeit import timeit
#from time import process_time as timer
import math

thing = numpy.array([1.1,0.0], dtype='float')
thing2 = [1.1,0.0]

def NPFunc():

    for x in range(1,200000):
        thing[0] += 0.1
        thing[1] = numpy.sqrt(thing[0])


    print(thing)
    return None


def PyFunc():

    for x in range(1,200000):
        thing2[0] += 0.1
        thing2[1] = math.sqrt(thing2[0])

    print(thing2)
    return None


print(timeit(NPFunc, number=1))
print(timeit(PyFunc, number=1))

It gives this result, which indicates normal Python is 3x faster:

[ 20000.99999999    141.42489173]
0.2917748889885843

[20000.99999998944, 141.42489172698504]
0.10341173503547907

Am I doing something wrong, is is this calculation just so simple that it isn't a good test for numpy ?

Answer 1

Am I doing something wrong, is is this calculation just so simple that it isn't a good test for NumPy?

It's not really that the calculation is simple, but that you're not taking any advantage of NumPy.

The main benefit of NumPy is vectorization: you can apply an operation to every element of an array in one go, and whatever looping is needed happens inside some tightly-optimized C (or Fortran or C++ or whatever) loop inside NumPy, rather than in a slow generic Python iteration.

But you're only accessing a single value, so there's no looping to be done in C.

On top of that, because the values in an array are stored as "native" values, NumPy functions don't need to unbox them, pulling the raw C double out of a Python float , and then re-box them in a new Python float , the way any Python math functions have to.

But you're not doing that either. In fact, you're doubling that work: You're pulling the value out o the array as a float (boxing it), then passing it to a function (which has to unbox it, and then rebox it to return a result), then storing it back in an array (unboxing it again).

And meanwhile, because np.sqrt is designed to work on arrays, it has to first check the type of what you're passing it and decide whether it needs to loop over an array or unbox and rebox a single value or whatever, while math.sqrt just takes a single value. When you call np.sqrt on an array of 200000 elements, the added cost of that type switch is negligible, but when you're doing it every time through the inner loop, that's a different story.

---

So, it's not an unfair test.

You've demonstrated that using NumPy to pull out values one at a time, act on them one at a time, and store them back in the array one at a time is slower than just not using NumPy.

But, if you compare it to actually taking advantage of NumPy—eg, by creating an array of 200000 floats and then calling np.sqrt on that array vs. looping over it and calling math.sqrt on each one—you'll demonstrate that using NumPy the way it was intended is faster than not using it.

Answer 2

you're comparing it wrong

a_list = np.arange(0,20000,0.1)
timeit(lambda:np.sqrt(a_list),number=1)