subprocess.call and os.system return not found

Question

I'm attempting to throw together a quick script that I can use to check what AWS EC2 snapshots have been made in the last x number of days.

Whilst I get an output error of

/bin/sh: 1: /usr/local/bin/aws --profile dummydata0 ec2 describe-snapshots --owner-ids 000000000000 --filters Name=start-time,Values=2014-07-08*: not found

running

/usr/local/bin/aws --profile dummydata0 ec2 describe-snapshots --owner-ids 000000000000 --filters Name=start-time,Values=2014-07-08*

at the command line works just fine, so I'm guessing that my basic Linux comprehension is falling down here.

Here's my script in full run from the command line using python ./script.py

#!/usr/bin/env python
import subprocess
import datetime

# Create your views here.

def main(numdays):
        base = datetime.date.today()
        date_list = [base - datetime.timedelta(days=x) for x in range(0, numdays)]
        environment = {'dummydata0':'000000000000', 'dummydata1':'111111111111', 'dummydata2':'222222222222'}
        data = []
        for date in date_list:
                date_string = str(date) + "*"
                # Python 3 prefers .values to .itervalues
                for key in environment:
                        call_snapshots = '"/usr/local/bin/aws --profile {0} ec2 describe-snapshots --owner-ids {1} --filters Name=start-time,Values={2}"'.format((key), (environment[key]), (date_string))
                        return subprocess.call(call_snapshots, shell=True)

main(7)

Answer 1

You have a pair of quotes inside your string:

call_snapshots = '"/usr/local/bin/aws --profile {0} ec2 describe-snapshots --owner-ids {1} --filters Name=start-time,Values={2}"'.format((key), (environment[key]), (date_string))

So the actual command that the shell will receive is:

"/usr/local/bin/aws --profile ... etc."

And these quotes instruct the shell to treat the entire line as a single item. It won't look for a program named aws , but for a program named aws --profile .... and that won't work.

You can fix this by using pipes.quote to properly quote your arguments (in Python 3 renamed to shlex.quote ):

call_snapshots = '/usr/local/bin/aws --profile {0} ec2 describe-snapshots --owner-ids {1} --filters Name=start-time,Values={2}'.format(
    *map(pipes.quote, (key, environment[key], date_string)))
return subprocess.call(call_snapshots, shell=True)

Or, better, by avoiding the shell and passing your arguments as a list:

call_snapshots = ['/usr/local/bin/aws',
  '--profile', key, 'ec2', 'describe-snapshots', '--owner-ids', environment[key],
  '--filters', 'Name=start-time,Values=' + date_string]
return subprocess.call(call_snapshots)

[And doesn't AWS have a Python API for this kind of thing?]