python迭代列表性能差异列表

Question

When iterating a list of lists in python 2.7.3 I noticed performance differences when changing the order of the iteration:

I have a list of 200 lists of 500000 strings. I then iterate in the following ways:

numberOfRows = len(columns[0])
numberOfColumns = len(columns)

t1 = time.clock()
for i in xrange(numberOfRows):
    for j in xrange(numberOfColumns):
        cell = columns[j][i]
print time.clock() - t1

t1 = time.clock()
for i in xrange(numberOfColumns):
    for j in xrange(numberOfRows):
        cell = columns[i][j]
print time.clock() - t1

The program repeatedly produces outputs similar to this:

33.97
29.39

Now I expected to have efficient random access on the lists . Where do these 4 seconds come from; is it only caching?

Answer 1

I get something like

30.509407822896037
29.88344778700383

for

columns = [[0] * 500000 for x in range(200)]

If I replace the cell = ... lines with pass , I get

8.44722739915369
10.23647023463866

So it's definitely not an issue with creating the xrange objects or something alike.

It's the caching (not by Python, by the computer) of the columns: If I use

columns = [[0] * 500000] * 200

I get

27.725353873145195
29.592749434295797

Here, always the same column object is used, and there is (almost) no difference in caching. Thus (about) the same timing difference as in the pass variant shows.