Need to write a bash script to ssh into many UNIX machines, run a python script and push results to GitHub

Question

I'm trying to write a bash script that does the following for ~200 UNIX machines:

In a for loop, ssh in, navigate to the right directory.

Then, executes a python file in the given directory with a command line argument. This command line argument is specified based off of location in the for loop. So, the first machine runs the script with a command of '0', then the second machine runs the script with a command of '1', so on and so forth.

How would I go about doing this? I'm looking into shell scripting but I've never written one before.

Also, I have to use a password to sign into the machines, so I need to account for that in a script.

Answer 1

Sparrowdo provides a neat API to do this, I wrote a small article on this topic - https://sparrowdo.wordpress.com/2017/02/01/sshscp-commands-with-sparrowdo/ . The advantage is you write scenario in high level language Perl6 and free to implement any complex logic.

PS.disclosure - I am the tool author.

Answer 2

First of all I'll assume you're as lazy as me and want to go to the easiest solution. In my opinion it is, in this case, Ansible . It is an automation tool which can ssh for you on all the machines you list on a hosts file.

hosts file :

[myServers]
myServer1
myServer2
myServer3
...

You can write playbooks, which are series of actions to execute on each machine. Here, for example your playbook.yml would look like

- name: Executing my python program
  shell: "python mypythoncommand"
  args:
    chdir: theDirectoryYouWant

At this point, we still need to increment some part in the command for each machine. In order to do this we can use the Jinja2 language, which is basically code interpreted before applying commands.

I think something like that would do the job :

hosts: myServers
  vars:
    - myVariable: 0
  tasks:
  {% for thehost in play_hosts %}
  {% if inventory_hostname==thehost %}
    - name: Executing my python program
      shell: "python mypythoncommand {{myVariable + loop.index}}"
      args:
        chdir: theDirectoryYouWantToBeIn
  {% endif %}
  {% endfor %}

This playbook ssh on all the servers, iterates on your server list and when it finds the server Ansible is ssh-ing on, it executes your command with the variable incremented for each server it passed already.

You will want to get the output of your program in a file. Ansible stores the log where you want, you juste have to :

export ANSIBLE_LOG_PATH="/path/to/my/log" && ansible-playbook myAwesomePlaybook.yml -k -u username && git commit -a -m "execution log $(date)" -k asks for password and -u username choose the user you want to ssh as.

Hopes it helps, if you really wanna go bash you may want to read this and take a look at sshpass in order to connect to each server.