AWS EMR - How to automatically edit file on all slave nodes?

Question

I am running a Python script on all slave nodes of an AWS EMR cluster. I need to configure matplotlib to use non-interactive backend on each slave node, otherwise I will run into an error ( detailed description provided here )

My current solution is to manually ssh into each slave node and to manually edit /usr/local/lib64/python2.7/site-packages/matplotlib/mpl-data/matplotlibrc:

backend : agg

Obviously, this approach is very time-consuming and inefficient.

Can anyone provide a small (pseudo) code snippet which performs this task automatically on all slave nodes?

Answer 1

Simplest solution is to provide a shell script via bootstrap action after Amazon EMR launches the instance:

#! /bin/sh
sudo sed -i 's/TkAgg/agg/g' /usr/local/lib64/python2.7/site-packages/matplotlib/mpl-data/matplotlibrc