[英]Finding Memory Usage, CPU utilization, Execution time for running a python script
I am using python, and suppose i had some code as below我正在使用 python,并假设我有一些代码如下
example.py例子.py
import os, psutil
import MySQLdb as mdb
conn = mdb.connect(user='root', passwd='redhat', db='File_Data', host='localhost', charset="utf8")
file_path = "/home/local/user/Module/File_processing/part-file.txt"
p = psutil.Process(os.getpid())
cursor = conn.cursor()
for line in file_open:
result = line.split('\t')
query = "insert into PerformaceReport (campaignID, keywordID, keyword, avgPosition)"
query += " VALUES (%s,%s,'%s',%s)%(result[0],result[1],result[2],result[3])"
cursor.execute( query )
conn.commit()
print p.get_cpu_times()
print p.get_memory_percent()
The above code reads data from text file and saves to database and its working fine上面的代码从文本文件中读取数据并保存到数据库并且它工作正常
I am running the file with the command python path/to/the/example.py
我正在使用命令python path/to/the/example.py
运行文件
Now what i am trying to do is finding/capturing the below three values/information现在我要做的是查找/捕获以下三个值/信息
execution time
taken by the python script( example.py
) to read and saves data to the database python脚本( example.py
)读取数据并将数据保存到数据库所花费的总execution time
CPU utilization
when the python script( example.py
) has run(executed) python脚本( example.py
)运行(执行)时的CPU utilization
Memory usage
when the python script( example.py
) has run(executed) python脚本( example.py
)运行(执行)时的Memory usage
I already googled a lot and found some information here , here and here , each explanation has different approach and i am really confused and unable to understand the approach on how to capture the above three values when the script example.py
has run so asked in SO我已经在谷歌上搜索了很多,在这里, 这里和这里找到了一些信息,每个解释都有不同的方法,我真的很困惑,无法理解当脚本example.py
运行时如何捕获上述三个值的方法,所以问所以
Can anyone please provide me an working example(code) that suits/captures the above values for my need i mentioned above, so that i can extend further to my further needs任何人都可以为我提供一个适合/捕获我上面提到的需要的上述值的工作示例(代码),以便我可以进一步扩展我的进一步需求
Edited已编辑
Process Followed过程遵循
1 . 1 . For finding execution time用于查找执行时间
Run the example.py file with the command as below使用如下命令运行 example.py 文件
python -m cProfile -o file.profile example.py
2 . 2 . For finding CPU Usage用于查找 CPU 使用率
Executing the top
command in a terminal after the example.py has runned在 example.py 运行后在终端中执行top
命令
Result:结果:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1354 root 20 0 138m 17m 7200 S 2.0 0.6 2:40.50 Xorg
2672 shivakri 20 0 581m 19m 12m S 1.3 0.6 0:07.60 gnome-terminal
2587 shivakri 20 0 987m 168m 33m S 1.0 5.6 5:25.26 firefox
2780 shivakri 20 0 1233m 51m 16m S 1.0 1.7 0:16.51 gedit
2163 shivakri 20 0 821m 135m 43m S 0.7 4.5 4:58.74 chrome
1938 shivakri 20 0 567m 15m 11m S 0.3 0.5 0:06.64 metacity
........
........
Now in the above result, how/where can i find the CPU/Memory utilization values, whether i need to take the values %CPU %MEM
from the line that contains gnome-terminal
?现在在上面的结果中,我如何/在哪里可以找到 CPU/内存利用率值,是否需要从包含gnome-terminal
的行中获取值%CPU %MEM
? (Because we are running the file through terminal ?) (因为我们通过终端运行文件?)
Aslo top
is used for finding total cpu utilization processes that are running, but we need to find the cpu utilization for only example.py script when it is executed/running, i mean how much CPU, Memory is utilized when we run the file(example.py) Aslo top
用于查找正在运行的总 CPU 利用率进程,但是我们只需要在 example.py 脚本执行/运行时找到它的 CPU 利用率,我的意思是我们运行文件时使用了多少 CPU、内存(例子.py)
Finally what my intention is when i run the python script (example.py) the values such as execution time,CPU utilization taken for running the script,and Memory used for running the script has to be recorded/stored some where(In to a text file) or something最后,当我运行 python 脚本 (example.py) 时,我的意图是执行时间、运行脚本所采用的 CPU 利用率以及用于运行脚本的内存等值必须被记录/存储在某个位置(进入到文本文件)或其他东西
For time profiling用于时间分析
cd
into the dir that contains example.py
(lets call this exampledir
). cd
进入包含example.py
的目录(我们称之为exampledir
)。python -m cProfile -o example.profile example.py
运行python -m cProfile -o example.profile example.py
cd
into the dir where you unpacked RunSnake cd
进入你解压 RunSnake 的目录python runsnake.py exampledir/example.profile
运行python runsnake.py exampledir/example.profile
For CPU Profiling用于 CPU 分析
psutil
使用psutil
psutil.Process
with myProcess = psutil.Process(os.getpid())
使用myProcess = psutil.Process(os.getpid())
创建一个新的psutil.Process
myProcess.get_memory_info()
or myProcess.get_ext_memory_info()
or myProcess.get_memory_percent()
as required根据需要调用myProcess.get_memory_info()
或myProcess.get_ext_memory_info()
或myProcess.get_memory_percent()
For memory profiling用于内存分析
easy_install
or pip
使用easy_install
或pip
安装 meliaeexample.py
:将以下代码行添加到example.py
的顶部:from meliae import scanner # [[source](http://www.vrplumber.com/programming/runsnakerun/)]
scanner.dump_all_objects( filename ) # you can pass a file-handle if you prefer
runsnakemem fpath
, where fpath
is the path to the file that you used in the code above.运行runsnakemem fpath
,其中fpath
是您在上面的代码中使用的文件的路径。This should get you a visual memory profiler similar to What you got with RunSnakeRun.这应该会为您提供一个类似于 RunSnakeRun 所获得的可视化内存分析器。
Hope this helps希望这可以帮助
Here is an informative blog post if anyone still searching anything regarding this requirements:如果有人仍在搜索有关此要求的任何内容,这里是一篇内容丰富的博客文章:
Monitoring memory usage of a running Python program 监控正在运行的 Python 程序的内存使用情况
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.