[英]Fastest way to make a list-like object containing integers in python
This means I want to have an object that supports the two (very) basic operations of a list: getting an object in a certain index (1) and changing its value (2). 这意味着我想拥有一个支持列表的两个(非常)基本操作的对象:获取某个索引(1)中的对象并更改其值(2)。
I came across these two: [1] [2] 我遇到了这两个: [1] [2]
They didn't solve my problem because all the solutions to them were simply too slow: in my PC array.array('i',(0,)*10 ** 8)
resulted in an error (lol); 他们没有解决我的问题,因为他们的所有解决方案都太慢了:在我的PC
array.array('i',(0,)*10 ** 8)
导致错误(lol); [0 for _ in range(10**8)]
took about 15 seconds (wow!); [0 for _ in range(10**8)]
花了大约15秒(哇!); [0] * 10 ** 8
took 2.3 seconds; [0] * 10 ** 8
耗时2.3秒; [None] * 10 ** 8
took 1.8 seconds; [None] * 10 ** 8
了1.8秒; (1.8sec could be faster...) (1.8秒可能更快...)
I tried using the ctypes
module 我尝试使用
ctypes
模块
from ctypes import c_int
array = (c_int * 10 ** 8)()
The code above took only 0.7 seconds ... but is there a way to make it faster? 上面的代码只花了0.7秒......但有没有办法让它更快? Besides being fast it has some disadvantages:
除了快速,它有一些缺点:
Is it really possible to do what I'm asking? 真的有可能做我要问的事吗? Is there a faster way rather than using the
ctypes
module? 有没有比使用
ctypes
模块更快的方法? If so make sure that you are using a 'built-in' / 'pre-installed' module. 如果是这样,请确保使用“内置”/“预安装”模块。
I'm using python for competitive programming and most interpreters/judges just won't allow external libraries. 我正在使用python进行竞争性编程,大多数口译/评委都不允许使用外部库。
I can see many of the answers use the array
function of the array
module. 我可以看到很多答案都使用了
array
模块的array
功能。 They all use 'i' to specify we want to store integers. 他们都使用'i'来指定我们想要存储整数。 Is it possible to make a class and create an `array.array' containing it?
是否可以创建一个类并创建一个包含它的`array.array'? For example:
例如:
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
# make array.array object with all indexes containing a Point with atributes x and y with value 0
# an example with a list of what I want to do is this:
# l = [Point(0, 0) for _ in range(10**3)]
array.array('i',(0,) * 10**8)
resulted in an error (lol)array.array('i',(0,) * 10**8)
导致错误(lol)
You didn't specify what error you got - that works for me, although it's not very fast because it builds an intermediate tuple and immediately discards it. 你没有指定你得到的错误 - 这对我有用,虽然它不是很快,因为它构建了一个中间元组并立即丢弃它。 Using Python's built-in types,
array.array
will probably result in best performance, provided you avoid the tuple: 使用Python的内置类型,如果你避免使用元组,
array.array
可能会产生最佳性能:
a = array.array('i', (0,)) * 10**8
The code above took only 0.7 seconds ... but is there a way to make it faster?
上面的代码只花了0.7秒......但有没有办法让它更快?
It will be hard to beat array.array
if you're not allowed to create or import C extensions. 如果不允许创建或导入C扩展,那将很难击败
array.array
。 On my several years old machine the above takes 0.6 seconds. 在我几年前的机器上,上面需要0.6秒。 You can further optimize it by increasing the size of the initial array.
您可以通过增加初始数组的大小来进一步优化它。 For example, this produces the same result, but is almost 3x faster (!):
例如,这会产生相同的结果,但几乎快3倍(!):
# 0.22 s
a = array.array('i', (0,) * 10) * 10**7
On my machine the following version works best: 在我的机器上,以下版本效果最佳:
# 0.19 s
a = array.array('i', (0,) * 100) * 10**6
Further increasing the initial array size doesn't help and soon starts degrading performance. 进一步增加初始阵列大小并没有帮助,很快就会开始降低性能。
To get better efficiency, consider alternative approaches, such as a lazy list or an altogether different data structure tailored for your use case. 为了获得更高的效率,请考虑其他方法,例如惰性列表或为您的用例量身定制的完全不同的数据结构。 Given the context of a competition, that might be what is actually being sought.
鉴于竞争的背景,这可能是实际上正在寻求的。
Be aware, however, that each solution will have different tradeoffs. 但请注意,每种解决方案都会有不同的权衡。 For example, a lazy array such as one provided by @KonstantinNikitin will be extremely efficient to construct, but its
__getitem__
and __setitem__
, implemented in pure Python, will be several orders of magnitude slower than those of list or array.array
. 例如,像@KonstantinNikitin提供的惰性数组将非常有效地构造,但是在纯Python中实现的
__getitem__
和__setitem__
将比list或array.array
慢几个数量级。 Which is better for you boils down to what operations are more frequent in your program, and that is up to you to find out. 对您来说哪个更好,归结为您的计划中更频繁的操作,这取决于您找出答案。
I would just use the numpy
module, which supports fast array operations. 我会使用
numpy
模块,它支持快速数组操作。
For example making an array with numbers 0 to 10**8: 例如,制作一个数字为0到10 ** 8的数组:
import numpy as np
import time
b = time.time()
a = np.linspace(0, 10**8, 10**8)
c = time.time()
print(c-b)
>>>0.5000154972076416
Or making an array of 0s that is 10**8 long: 或者制作一个长度为10 ** 8的0数组:
b = time.time()
a = np.zeros(shape=(10**8,))
c = time.time()
print(c-b)
>>>0.0
The main reason why numpy is this fast is because it is implemented in C. numpy这么快的主要原因是因为它是用C实现的。
EDIT: If you want to do it with only pre-installed packages, you can try using the array
package: 编辑:如果你想只使用预安装的包,你可以尝试使用
array
包:
import array
import time
r = time.time()
a = array.array('i', [0]) * (10**8)
print(time.time()-r)
>>>0.15627217292785645
I'd say that you can try different approaches: 我会说你可以尝试不同的方法:
1) numpy
. 1)
numpy
。 It's really a standard for arrays. 它确实是阵列的标准。 It comes with a cost of crossing Python <-> C boundary for each operation, but it really depends on your task.
它为每个操作带来了跨越Python < - > C边界的成本,但它实际上取决于您的任务。
x = numpy.array(10 ** 8)
timeit.timeit('x = numpy.array(10 ** 8)', 'import numpy', number=1)
4.195800283923745e-05
2) lazy initialization (like JavaScript arrays). 2)延迟初始化(如JavaScript数组)。
class LazyArray:
def __init__(self, size):
self.storage = {}
self.size = size
def check(self, i):
if i < 0 or i >= self.size:
raise RuntimeError()
def __getitem__(self, i):
self.check(i)
return self.storage.get(i, 0)
def __setitem__(self, i, value):
self.check(i)
self.storage[i] = value
x = LazyArray(10 ** 8)
x[10]
>> 0
x[10] = 5
x[10]
>> 0
If you really only want these two properties: 如果你真的只想要这两个属性:
getting an object in a certain index (1) and changing its value (2)
获取某个索引(1)中的对象并更改其值(2)
then you can just use a collections.defaultdict
: 然后你可以使用
collections.defaultdict
:
import collections
my_list = collections.defaultdict(lambda: 0)
which is rather fast (~0.4 μs): 相当快(~0.4μs):
$ python3 -m timeit -s 'import collections' 'collections.defaultdict(lambda: 0)'
1000000 loops, best of 3: 0.417 usec per loop
however, actually using it will probably be quite a bit slower than any of the types suggested in other answers. 但是,实际使用它可能比其他答案中建议的任何类型都要慢一些。
For cases where you only need integers from 0 to 255, bytearray
objects are quite fast to create: 对于只需要0到255之间的整数的情况,可以非常快速地创建
bytearray
对象:
>>> timeit.timeit('bytearray(100000)', number=1000)
0.005567271093696036
>>> timeit.timeit('array.array("B", [0])*100000', 'import array', number=1000)
0.36631167401839093
>>> timeit.timeit('array.array("i", [0])*100000', 'import array', number=1000)
0.56494557472422
Unlike array.array
, this zeros the allocation directly instead of copying from an object initialized with zeros. 与
array.array
不同,这会直接将分配归零,而不是从使用零初始化的对象进行复制。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.