Python 中的可变迭代器？
[英]Mutable iterators in Python?
问题描述
While benchmarking my application, I noticed accessing arrays items by index in relatively expensive in Python, making for v in lst: v signicantly faster than for i in range(len(lst): lst[i] :
在对我的应用程序进行基准测试时，我注意到在 Python 中按索引访问数组项相对昂贵，这使得for v in lst: v明显快于for i in range(len(lst): lst[i] :
from array import array
a_ = array('f', range(1000))
def f1():
a = a_
acc = 0
for v in a:
acc += v
return acc
def f2():
a = a_
acc = 0
for i in range(len(a)):
acc += a[i]
return acc
from dis import dis
from timeit import timeit
for f in f1,f2:
dis(f)
print(timeit(f, number=20000))
print()
Producing:生产：
9 0 LOAD_GLOBAL 0 (a_)
3 STORE_FAST 0 (a)
10 6 LOAD_CONST 1 (0)
9 STORE_FAST 1 (acc)
11 12 SETUP_LOOP 24 (to 39)
15 LOAD_FAST 0 (a)
18 GET_ITER
>> 19 FOR_ITER 16 (to 38)
22 STORE_FAST 2 (v)
12 25 LOAD_FAST 1 (acc)
28 LOAD_FAST 2 (v)
31 INPLACE_ADD
32 STORE_FAST 1 (acc)
35 JUMP_ABSOLUTE 19
>> 38 POP_BLOCK
14 >> 39 LOAD_FAST 1 (acc)
42 RETURN_VALUE
0.6036834940023255
17 0 LOAD_GLOBAL 0 (a_)
3 STORE_FAST 0 (a)
18 6 LOAD_CONST 1 (0)
9 STORE_FAST 1 (acc)
19 12 SETUP_LOOP 40 (to 55)
15 LOAD_GLOBAL 1 (range)
18 LOAD_GLOBAL 2 (len)
21 LOAD_FAST 0 (a)
24 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
27 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
30 GET_ITER
>> 31 FOR_ITER 20 (to 54)
34 STORE_FAST 2 (i)
20 37 LOAD_FAST 1 (acc)
40 LOAD_FAST 0 (a)
43 LOAD_FAST 2 (i)
46 BINARY_SUBSCR
47 INPLACE_ADD
48 STORE_FAST 1 (acc)
51 JUMP_ABSOLUTE 31
>> 54 POP_BLOCK
22 >> 55 LOAD_FAST 1 (acc)
58 RETURN_VALUE
1.0093544629999087
The core of the loop differs only in the presence of the extra LOAD_FAST BINARY_SUBSCR opcodes when using indexed access.循环的核心仅在使用索引访问时存在额外的LOAD_FAST BINARY_SUBSCR操作码时有所不同。 However, this suffices for the iterator-based solution to be about 40% faster than using indexed access.然而，这足以让基于迭代器的解决方案比使用索引访问快 40%。
Unfortunately, in this form, iterators are only usable for reading the input array.不幸的是，在这种形式中，迭代器只能用于读取输入数组。 Is there a way to use a "fast" iterator to change in place the item of an array, or do I have to stick to the "slow" indexed access?有没有办法使用一个“快”迭代到位数组的项目变更的方式，或者说我要坚持到“慢”索引访问？
3 个解决方案
解决方案1
3 已采纳 2019-11-28 15:04:12
For full loops, you can split the difference with enumerate , using indexed access for setting a value, and the name for reading it:对于完整循环，您可以使用enumerate分割差异，使用索引访问来设置值，以及读取它的名称：
for i, value in enumerate(mysequence):
mysequence[i] = do_stuff_with(value)
You can't avoid the indexed reassignment in a general loop structure though;但是，您无法避免通用循环结构中的索引重新分配； Python has no equivalent to C++ reference semantics where assignment changes the value referred to, rather than rebinding the name. Python 没有等效于 C++ 引用语义，其中赋值更改引用的值，而不是重新绑定名称。
That said, if the work is simple enough, a list comprehension avoids the need for an index, by building the new list and wholesale replacing the old one:也就是说，如果工作足够简单， list推导式通过构建新list并大量替换旧list避免对索引的需要：
mysequence[:] = [do_stuff_with(value) for value in mysequence]
Assigning to the complete slice of mysequence ensures it's modified in place, so other references to it see the change.分配给mysequence的完整切片确保它被修改到位，因此对它的其他引用会看到更改。 You can leave off the [:] if you don't want that behavior (you'll rebind to a new list with no other references to it).如果您不想要这种行为，您可以不使用[:] （您将重新绑定到一个没有其他引用的新list ）。
解决方案2
1 2019-11-28 16:48:44
Here are some timeit results for the different methods:以下是不同方法的一些时间结果：
#for v in a #for i in range(len(a)) #for i,v in enumerate(a)
[[0.47590930000296794, 0.8639191000038409, 0.7616558000008808],
[0.43640120000054594, 0.832395199999155, 0.7896779000002425],
[0.44416509999427944, 0.8366088000038872, 0.7590674000020954]]
Note that using numpy arrays is very fast, but only if you build the array inside numpy and only use the native numpy functions:请注意，使用 numpy 数组非常快，但前提是您在 numpy 内部构建数组并且仅使用本机 numpy 函数：
import numpy as np
def f4():
N = 1000
vect = np.arange(float(N))
return np.sum(vect)
timeit gives:时间给出：
[0.09995190000336152
0.10408379999716999
0.09926139999879524]
Attempting to modify a numpy array by explicit indexing seems to be give no advantage, however.然而，尝试通过显式索引修改 numpy 数组似乎没有任何好处。 Also copying any native Python structure into a numpy array is expensive.将任何原生 Python 结构复制到 numpy 数组中也是很昂贵的。
解决方案3
0 2019-11-28 16:43:38
As a complement to @ShadowRunner's answer, I've done some extra benchmarking to compare the different solutions to modify an array in place.作为对@ShadowRunner 答案的补充，我做了一些额外的基准测试来比较不同的解决方案来修改数组。
Even for relatively large arrays, it's cheaper to build a new array from a comprehension list, then overwriting the original array, rather than trying to modify in its turn the individual items of the array.即使对于相对较大的数组，从推导式列表构建新数组，然后覆盖原始数组，而不是尝试依次修改数组的各个项目，成本更低。
import itertools
from functools import reduce
import operator
from array import array
a_ = array('f', range(100000))
b_ = array('f', range(100000))
def f1():
a = a_
b = b_
for i,v in enumerate(a):
b[i] = v*2
def f2():
a = a_
b = b_
for i in range(len(a)):
b[i] = a[i]*2
def f3():
a = a_
b = b_
getter = a.__getitem__
setter = b.__setitem__
for i in range(len(a)):
setter(i, getter(i)*2)
def f4():
a = a_
b = b_
b[:] = array('f', [v*2 for v in a])
from dis import dis
from timeit import timeit
for f in f1,f2,f3,f4:
dis(f)
print(timeit(f, number=2000))
print()
Result:结果：
10 0 LOAD_GLOBAL 0 (a_)
3 STORE_FAST 0 (a)
11 6 LOAD_GLOBAL 1 (b_)
9 STORE_FAST 1 (b)
12 12 SETUP_LOOP 40 (to 55)
15 LOAD_GLOBAL 2 (enumerate)
18 LOAD_FAST 0 (a)
21 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
24 GET_ITER
>> 25 FOR_ITER 26 (to 54)
28 UNPACK_SEQUENCE 2
31 STORE_FAST 2 (i)
34 STORE_FAST 3 (v)
13 37 LOAD_FAST 3 (v)
40 LOAD_CONST 1 (2)
43 BINARY_MULTIPLY
44 LOAD_FAST 1 (b)
47 LOAD_FAST 2 (i)
50 STORE_SUBSCR
51 JUMP_ABSOLUTE 25
>> 54 POP_BLOCK
>> 55 LOAD_CONST 0 (None)
58 RETURN_VALUE
24.717656177999743
16 0 LOAD_GLOBAL 0 (a_)
3 STORE_FAST 0 (a)
17 6 LOAD_GLOBAL 1 (b_)
9 STORE_FAST 1 (b)
18 12 SETUP_LOOP 44 (to 59)
15 LOAD_GLOBAL 2 (range)
18 LOAD_GLOBAL 3 (len)
21 LOAD_FAST 0 (a)
24 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
27 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
30 GET_ITER
>> 31 FOR_ITER 24 (to 58)
34 STORE_FAST 2 (i)
19 37 LOAD_FAST 0 (a)
40 LOAD_FAST 2 (i)
43 BINARY_SUBSCR
44 LOAD_CONST 1 (2)
47 BINARY_MULTIPLY
48 LOAD_FAST 1 (b)
51 LOAD_FAST 2 (i)
54 STORE_SUBSCR
55 JUMP_ABSOLUTE 31
>> 58 POP_BLOCK
>> 59 LOAD_CONST 0 (None)
62 RETURN_VALUE
25.86994492100348
22 0 LOAD_GLOBAL 0 (a_)
3 STORE_FAST 0 (a)
23 6 LOAD_GLOBAL 1 (b_)
9 STORE_FAST 1 (b)
24 12 LOAD_FAST 0 (a)
15 LOAD_ATTR 2 (__getitem__)
18 STORE_FAST 2 (getter)
25 21 LOAD_FAST 1 (b)
24 LOAD_ATTR 3 (__setitem__)
27 STORE_FAST 3 (setter)
26 30 SETUP_LOOP 49 (to 82)
33 LOAD_GLOBAL 4 (range)
36 LOAD_GLOBAL 5 (len)
39 LOAD_FAST 0 (a)
42 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
45 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
48 GET_ITER
>> 49 FOR_ITER 29 (to 81)
52 STORE_FAST 4 (i)
27 55 LOAD_FAST 3 (setter)
58 LOAD_FAST 4 (i)
61 LOAD_FAST 2 (getter)
64 LOAD_FAST 4 (i)
67 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
70 LOAD_CONST 1 (2)
73 BINARY_MULTIPLY
74 CALL_FUNCTION 2 (2 positional, 0 keyword pair)
77 POP_TOP
78 JUMP_ABSOLUTE 49
>> 81 POP_BLOCK
>> 82 LOAD_CONST 0 (None)
85 RETURN_VALUE
42.435717200998624
30 0 LOAD_GLOBAL 0 (a_)
3 STORE_FAST 0 (a)
31 6 LOAD_GLOBAL 1 (b_)
9 STORE_FAST 1 (b)
32 12 LOAD_GLOBAL 2 (array)
15 LOAD_CONST 1 ('f')
18 LOAD_CONST 2 (<code object <listcomp> at 0x7fab3b1e9ae0, file "t.py", line 32>)
21 LOAD_CONST 3 ('f4.<locals>.<listcomp>')
24 MAKE_FUNCTION 0
27 LOAD_FAST 0 (a)
30 GET_ITER
31 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
34 CALL_FUNCTION 2 (2 positional, 0 keyword pair)
37 LOAD_FAST 1 (b)
40 LOAD_CONST 0 (None)
43 LOAD_CONST 0 (None)
46 BUILD_SLICE 2
49 STORE_SUBSCR
50 LOAD_CONST 0 (None)
53 RETURN_VALUE
19.328940125000372
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.