[英]Time taken by an import in Python
我想知道内置模块和用户定义模块的导入需要多少时间。
One way to profile imports is to use the profile_imports module used in bzr source code : 分析导入的一种方法是使用bzr 源代码中使用的profile_imports模块:
# put those two lines at the top of your script
import profile_imports
profile_imports.install()
# display the results
profile_imports.log_stack_info(sys.stderr)
Besides giving you timing for imports, this also estimates time to compile regex, which is often a significant cause for slowdown in imports. 除了为您提供进口时间外,这也估计了编制正则表达式的时间,这通常是进口放缓的重要原因。
You can test this runnning 你可以测试这个运行
$ time python -c "import math"
However, what would this help you? 但是,这会对你有什么帮助? Importing only happens once and will almost never be a bottle neck.
进口只发生一次,几乎永远不会成为瓶颈。 Importing the same module over and over will not run significantly slower than importing it once, since Python tracks which modules have already been imported.
反复导入同一模块的速度不会比导入一次慢得多,因为Python会跟踪哪些模块已经导入。
What are you actually trying to achieve? 你究竟想要实现什么目标?
Starting from Python3.7 releaese, new -X importtime
option is available. 从Python3.7 releaese开始,新的
-X importtime
选项可用。 To measure import time, just simply execute your script with that option, eg python -X importtime my_script.py
. 要测量导入时间,只需使用该选项执行脚本,例如
python -X importtime my_script.py
。
For reference: 以供参考:
To find out how long an import takes, the simplest way is probably using the timeit module .. 要了解导入需要多长时间,最简单的方法可能是使用timeit模块 。
>>> import timeit
>>> t = timeit.Timer('import urllib')
>>> t.timeit(number = 1000000)
0.98621106147766113
So to import urllib 1 million times, it took just under a second (on a Macbook Pro).. 因此要导入urllib 100万次,花了不到一秒钟(在Macbook Pro上)..
i have a master script that imports other modules.I need to calculate how much time it takes
我有一个主脚本导入其他模块。我需要计算它需要多少时间
If you mean the total script execution time, on Linux/OS X/Cygwin, you can run the script using the time
command, for example: 如果您的意思是脚本执行总时间,则在Linux / OS X / Cygwin上,您可以使用
time
命令运行脚本,例如:
$ time python myscript.py
real 0m0.046s
user 0m0.024s
sys 0m0.020s
(remember that includes all the Python interpreter startup time, as well as the actual code execution time, although it's pretty trivial amount) (请记住,包括所有Python解释器的启动时间,以及实际的代码执行时间,尽管它的数量非常微不足道)
Another, possibly more useful way is to profile the script: 另一种可能更有用的方法是分析脚本:
Instead of running your code with 而不是运行您的代码
$ python myscript.py
..you use.. ..你用..
$ python -m cProfile myscript.py
1059 function calls in 0.015 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.000 0.000 <string>:1(<module>)
1 0.002 0.002 0.015 0.015 myscript.py:1(<module>)
[...]
I don't find the command line output very easy to read, so I almost always use gprof2dot , which turns the profiling information into a pretty graphviz graph: 我没有发现命令行输出非常容易阅读,所以我几乎总是使用gprof2dot ,它将分析信息转换为漂亮的graphviz图:
$ python -m cProfile -o myscript.prof myscript.py
$ python gprof2dot.py -o myscript.dot -f pstats myscript.prof
$ dot -Tpng -o profile.png prof_runtest.dot -Gbgcolor=black
使用cProfile:
python -m cProfile yourscript.py
I ran into this issue profiling a large legacy application with a multi-second startup time. 我遇到了这个问题,分析了一个具有多秒启动时间的大型遗留应用程序。 It's relatively simple to replace the builtin importer with something that does some profiling.
将内置导入器替换为执行某些分析的内容相对简单。 Below is a hacky way of showing approximately how long each module takes to execute:
下面是一个显示每个模块执行大约需要多长时间的hacky方式:
import os
import sys
import time
class ImportEventNode(object):
def __init__(self, name, start_time, children=None, end_time=None):
self.name = name
self.start_time = start_time
self.children = [] if children is None else children
self.end_time = end_time
def __repr__(self):
return 'ImportEventNode({self.name}, {self.start_time}, children={self.children}, end_time={self.end_time})'.format(self=self)
@property
def total_time(self):
return self.end_time - self.start_time
@property
def net_time(self):
return self.total_time - sum(child.total_time for child in self.children)
root_node = cur_node = None
all_nodes = []
old_import = __import__
def __import__(*args, **kwargs):
global root_node, cur_node
name = args[0]
if name not in sys.modules:
t0 = time.time()
if root_node is None:
root_node = prev_node = cur_node = lcur_node = ImportEventNode(args[0], t0)
else:
prev_node = cur_node
cur_node = lcur_node = ImportEventNode(name, t0)
prev_node.children.append(cur_node)
try:
ret = old_import(*args, **kwargs)
finally:
lcur_node.end_time = time.time()
all_nodes.append(lcur_node)
cur_node = prev_node
return ret
else:
return old_import(*args, **kwargs)
__builtins__.__import__ = __import__
Running on a simple example, here's how it looks on importing scipy.stats: 运行一个简单的例子,这是它在导入scipy.stats时的样子:
:import scipy.stats
:
:nodes = sorted(all_nodes, key=(lambda x: x.net_time), reverse=True)
:for node in nodes[:10]:
: print(node.name, node.net_time)
:
:<EOF>
('pkg_resources', 0.08431100845336914)
('', 0.05861020088195801)
('decomp_schur', 0.016885995864868164)
('PIL', 0.0143890380859375)
('scipy.stats', 0.010602712631225586)
('pkg_resources._vendor.packaging.specifiers', 0.007072925567626953)
('add_newdocs', 0.00637507438659668)
('mtrand', 0.005497932434082031)
('scipy.sparse.linalg', 0.005171060562133789)
('scipy.linalg', 0.004471778869628906)
This is extension to szymon and skjern
suggestion of combining Python Profilers and tuna .这是szymon和
skjern
建议结合Python Profilers和tuna 的扩展。
First install tuna首先安装金枪鱼
pip install tuna
Then, create a python extension file and name it to whatever you desire.然后,创建一个 python 扩展文件并将其命名为您想要的任何名称。 here we name it as
timit_package.py
.这里我们将其命名为
timit_package.py
。
Within this file, include both built-in as well as user defined modules.在此文件中,包括内置模块和用户定义模块。
Here, I only test在这里,我只测试
import mne
Then, in the terminal然后,在终端
python -mcProfile -o program.prof timit_package.py
wait for few sec, then paste the following line into the terminal等待几秒钟,然后将以下行粘贴到终端中
tuna program.prof
A browser along with the profile will be generated.将生成一个浏览器和配置文件。
Im testing this with python 3.9我用 python 3.9 测试这个
Tested on Windows in Python 2.4 - you can try it yourself. 在Python 2.4中在Windows上测试 - 您可以自己尝试。
>>> import time
>>> ## Built-in module
>>> def testTime():
now = time.clock() # use time.time() on unix-based systems
import math
print time.clock() - now
>>> testTime()
7.54285810167e-006
>>> ## My own module
>>> def testTime():
now = time.clock()
import myBuiltInModule # Not its actual name ;-)
print time.clock() - now
>>> testTime()
0.00253174635324
>>> testTime()
3.70158777141e-006
So there is a big difference between cached modules and those being brought in from scratch. 因此,缓存模块与从头开始引入的模块之间存在很大差异。 To illustrate, we can reload the module:
为了说明,我们可以重新加载模块:
>>> def testTime():
now = time.clock()
reload(myBuiltInModule )
print time.clock() - now
>>> testTime()
0.00250017809526
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.