Python: Store command output in variable

Question

I'd like to measure the time and cpu usage of a command with the /usr/bin/time tool. But when I do os.popen( "/usr/bin/time -f \t%E MM:ss:mm ls -R" ).read() it also stores the output of ls -R . What can I do to only store the /usr/bin/time output?

I also tried it with subprocess but it doesn't work either.

out = subprocess.Popen(['/usr/bin/time', '-f', '"\t%E MM:ss:mm"', 'ls', '-R'], 
       stdout=subprocess.PIPE, 
       stderr=subprocess.STDOUT)
stdout,stderr = out.communicate()
print( stdout )

Running the command in the terminal looks like this:

Input:

/usr/bin/time -f "\t%E M:ss:mm, \t%P CPU" ls -R    

Output:

.:
Dockerfile  input  output  README.md  src  Test.py

./input:
links.txt  raw  yuv

./input/raw:

./input/yuv:

./output:

./src:
InstallEncoderArm.sh  InstallEncoderx86.sh  RunTests.py
    0:00.00 M:ss:mm,    100% CPU

Answer 1

You need to send the output of the command to /dev/null

>/usr/bin/time -f "\t%E real,\t%U user,\t%S sys" ls -Fs >/dev/null
        0:00.01 real,   0.00 user,      0.01 sys

The complexity here is we want to throw away the command output but store the output of the /usr/bin/time command.

To store the output of usr/bin/time as a variable is a little more complicated as /usr/bin/time presents its output on stderr. So we need to send the command output to dev/null , then redirect the output of time from stderr and capture it in a variable. Assuming you may want to execute more complex commands than ls -R we would normally call sh -c 'exec ' this will give you more options in the future. Thus:

result=$(/usr/bin/time -f "\t%E MM:ss:mm" sh -c 'exec ls -R >/dev/null' 2>&1 tee)

Execution Output:

>result=$(/usr/bin/time -f "\t%E MM:ss:mm" sh -c 'exec ls -R >/dev/null' 2>&1 tee
); echo $result
0:20.60 MM:ss:mm

here we capture the result as an environment variable

>echo $result
0:20.60 MM:ss:mm

finally we arrive at:

os.popen("/usr/bin/time -f '%E MM:ss:mm' sh -c 'exec ls -R >/dev/null' 2>&1 tee").read()

Execution Out:

>python3
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.popen("/usr/bin/time -f '%E MM:ss:mm' sh -c 'exec ls -R >/dev/null' 2>&1 tee").read()
'0:19.89 MM:ss:mm\n'

hoping the above points you in the right direction.

Answer 2

this code worked for me on python 2.7

import os
from datetime import datetime
CPU_Pct="Cpu Usage :"+str(round(float(os.popen('''grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage }' ''').readline()),2))+ "  time:" + datetim$

print(CPU_Pct)

the output will be like

5.74  time:2020-07-07 10:53:22

and if you wanted to get usage of memory too you can add this line to your code

tot_m, used_m, free_m = map(int, os.popen('free -t -m').readlines()[-1].split()[1:])

the final code could be this:

import os
from datetime import datetime
CPU="|| CPU Usage :"+str(round(float(os.popen('''grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage }' ''').readline()),2))
Time =  "||  time:" + datetime.now().strftime('%H:%M:%S')
tot_m, used_m, free_m = map(int, os.popen('free -t -m').readlines()[-1].split()[1:])
Memory ="|| memory :"+ str(used_m) +"/"+str(tot_m)
print(Time + CPU+Memory)

and here is the output:

||  time:11:02:33|| CPU Usage :5.74|| memory :13847/37529

Answer 3

It works with subprocess but notice that /usr/bin/time uses stderr

import subprocess

proc = subprocess.Popen(["/usr/bin/time -f \"\t%E M:ss:mm, \t%P CPU\" ls -R"], 
stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
(out, err) = proc.communicate()
print("program output:", err.decode("utf-8"))

Output:

program output:         0:00.00 M:ss:mm,        100% CPU