Execute external command and exchange variable using Python

Question

1. Introduction

I have a bunch of files in netcdf format. Each file contain the meteorology condition of somewhere in different period(hourly data).
I need to extract the first 12 h data for each file. So I select to use NCO(netcdf operator) to deal with.

NCO works with terminal environment. With >ncks -d Time 0,11 input.nc output.nc , I can get one datafile called out.nc which contain the first 12h data of in.nc .

2. My attempt

I want to keep all the process inside my ipython notebook. But I stuck on two aspects.

1. How to execute terminal code in python loop

2. How to transfer the string in python into terminal code.

Here is my fake code for example.

files = os.listdir('.')
for file in files:
    filename,extname = os.path.splitext(file)
    if extname == '.nc':   
        output = filename + "_0-12_" + extname
        ## The code below was my attempt
        !ncks -d Time 0,11 file output` 

3. Conclusion

Basically, my target was letting the fake code !ncks -d Time 0,11 file output coming true. That means:

1. execute netcdf operator directly in python loop...
2. ...using filename which is an string in python environment.

Sorry for my unclear question. Any advice would be appreciated!

Answer 1

You can use subprocess.check_output to execute external program:

import glob
import subprocess

for fn in glob.iglob('*.nc'):
    filename, extname = os.path.splitext(fn)
    output_fn = filename + "_0-12_" + extname
    output = subprocess.call(['ncks', '-d', 'Time', '0,11', fn, output_fn])
    print(output)

NOTE: updated the code to use glob.iglob ; you don't need to check extension manually.

Answer 2

You may also check out pynco which wraps the NCO with subprocess calls, similar to @falsetru's answer. Your application may look something like

nco = Nco()
for fn in glob.iglob('*.nc'):
    filename, extname = os.path.splitext(fn)
    output_fn = filename + "_0-12_" + extname
    nco.ncks(input=filename, output=output_fn, dimension='Time 0,11')