How to access AWS's dynamic EC2 inventory using Fabric?

Question

Fabric has a hosts setting to specify which computers to SSH into.

Amazon Web Services has more of a dynamic inventory that can be queried in python using tools like boto .

Is there a way to combine these two services? Ideally, I wanted something as simple as ansible 's approach with an inventory file and using an external file like ec2.py .

More specifically, is there a prebaked solution for this use case? Ideally, I would like to run something straightforward like this:

from fabric.api import env, task
import ec2
env.roledefs = ec2.Inventory()

@task
def command():
    run("lsb_release -a")

And run it like so, assuming env.roledefs['nginx'] exists:

$ fab -R nginx command

Answer 1

You can use fabric and boto concurrently. First you need to export the aws_secret_key, aws_secret_access_key and default regions from your console. Fabric file name should be fabfile.py and should not ec2.py/other.

import boto, urllib2
from   boto.ec2 import connect_to_region
from   fabric.api import env, run, cd, settings, sudo
from   fabric.api import parallel
import os
import sys
REGION       = os.environ.get("AWS_EC2_REGION")
env.user      = "ec2-user"
env.key_filename = ["/home/user/uswest.pem"]
@task
def command():
    run("lsb_release -a")
def _create_connection(region):
    print "Connecting to ", region
    conn = connect_to_region(
        region_name = region, 
        aws_access_key_id=os.environ.get("AWS_ACCESS_KEY_ID"), 
        aws_secret_access_key=os.environ.get("AWS_SECRET_ACCESS_KEY")
    )
    print "Connection with AWS established"
    return connection

Finally this program can be executed by using below command.

$ fab command

Answer 2

From http://docs.python-guide.org/en/latest/scenarios/admin/

You can see that if you set env.hosts = ['my_server1', 'my_server2']

You'll then be able to target those hosts.

With boto, if you just have a function that does ec2_connection.get_only_instances(filter={'tag':< whatever>}) and returns a list of their dns names, you'll be able to then set env.hosts = [< list of dns names from ec2>]

Piece of cake!