将值附加到空列表还是将值分配给预定义数组？

Question

In Python, for several applications I normally have to store values to an array, like:

results = []
for i in range(num_simulations):
    ...<calculate results_new>...
    results.append(results_new)

Yet I have seen most of other sample codes declaring a zero-value array first:

results = np.zeros(len(num_simulations))
for i in range(num_simulations):
    ...<calculate results_new>...
    results[i] = results_new

Which one is better for common practice? Or even if you have to make a performance comparison, is there really a significant difference in time and memory between the two methods?

DISCLAIMER: I more or less only use Python codes for simulations, and hence just want to achieve better practice as I go along.

Answer 1

There are a few things you should know about using numpy arrays:

1. If you will be playing around with matrices, they are way faster. In your application, for just storing the data, they do not provide any exceptional additional benefit
2. HOWEVER, initializing them comes at both a cost and a benefit. The benefit of using pre allocated space is that you need not worry about running into memory issues, etc. The con is that the overload of allocating that big a memory space is more significant (See end of code)

So in your application, if you are just storing the results in a list, and not performing any numerical methods, then it is fine if you do not use numpy. In fact, it is more efficient to do so, as seen below

In [29]: %%timeit
    ...: results=[]
    ...: num_simulations=10000
    ...: for i in range(num_simulations):
    ...:     results.append(i)
    ...: 
1000 loops, best of 3: 984 µs per loop

In [30]: %%timeit
    ...: num_simulations = 10000
    ...: results=np.zeros(num_simulations)
    ...: for i in range(num_simulations):
    ...:     results[i]=i
    ...: 
1000 loops, best of 3: 1.44 ms per loop

In [31]: %%timeit
    ...: results=[]
    ...: num_simulations=100000
    ...: for i in range(num_simulations):
    ...:     results.append(i)
    ...: 
100 loops, best of 3: 10.1 ms per loop

In [32]: %%timeit
    ...: num_simulations = 100000
    ...: results=np.zeros(num_simulations)
    ...: for i in range(num_simulations):
    ...:     results[i]=i
    ...: 
100 loops, best of 3: 15.4 ms per loop

In [33]: %%timeit
    ...: results=[]
    ...: num_simulations=1000000
    ...: for i in range(num_simulations):
    ...:     results.append(i)
    ...: 
10 loops, best of 3: 103 ms per loop

In [34]: %%timeit
    ...: num_simulations = 1000000
    ...: results=np.zeros(num_simulations)
    ...: for i in range(num_simulations):
    ...:     results[i]=i
    ...: 
10 loops, best of 3: 156 ms per loop

Just to sum up the results,

Normal list vs     Numpy
984         vs     1440 microsecond     for 10000 simulations
10.1        vs     15.4 millisecond     for 100000 simulations
103         vs     156 millisecond      for 1000000 simulations

Evidently, using just lists, purely for storing is faster, as it does not involve heap memory allocation overhead

However, for pretty much any and every numerical method you will want to perform on a matrix, Numpy offers a much more overshadowing benefit