Replace placeholders with corresponding env variables

Question

I have a configuration file with placeholders like this (stored in /tmp/var for this example)

ldap_bind_dn='${bind_dn}'

Now I'd like replace ${bind_dn} with the the environment variable of the same name (which is set inside Docker).

export $bind_dn=CN=my-user,CN=Users,DC=example,DC=com

The expected result after processing aboves test file would be

ldap_bind_dn='CN=my-user,CN=Users,DC=example,DC=com'

I tried sed but it doesn't replace it to the value of the env variable

$ sed "s#\${\(.*\)}#$\1#" /tmp/var 
bind_dn='$bind_dn'

Why sed replace with $bind_dn instead of the value? I'd expect that the variable is processed because I haven't escaped the $ sign.

The expression itself works, only the substitution doesn't:

$ sed "s#\${\(.*\)}#test123#" /tmp/var 
bind_dn='test123'

The replacement is also done correctly when the target variable is hardcoded

$ sed "s#\${\(.*\)}#$bind_dn#" /tmp/var 
bind_dn='CN=my-user,CN=Users,DC=example,DC=com'

But since we have a bunch of configuration variables, I'd like to automatically replace all env variables in the format ${NAME} automatically.

Answer 1

Not the most elegant solution, but this one works in bash:

sed 's#\${\(.*\)}#`{echo,"$\1"}`#' /tmp/var | xargs -n1 -I{} echo echo "{}" | bash -s

It is a little bit tricky because you need bash execution for the variable replacement, that's why I'm piping it to bash -s

Answer 2

While the env variable could be parsed by executing eval with mapfile , this seems not suiteable for me because its a ini configuration file. The sections marked with brackets like [general] would throw errors. And I also have concerns to just execute the WHOLE line, which allows executing any command.

This is fixed by the following awk :

awk '{while(match($0,"[$]{[^}]*}")) {var=substr($0,RSTART+2,RLENGTH -3);gsub("[$]{"var"}",ENVIRON[var])}}1' < /tmp/var > /tmp/var

Answer 3

Why sed replace with $bind_dn instead of the value?

Because sed is not supposed to do shell parameter expansion in its pattern space. That's the job of the shell mainly.

Using GNU sed :

~>  cat /tmp/var
ldap_bind_dn='${bind_dn}'
~>  export bind_dn=CN=my-user,CN=Users,DC=example,DC=com
~>  sed -E 's/^\w+=.*/echo "&"/e' /tmp/var
ldap_bind_dn='CN=my-user,CN=Users,DC=example,DC=com'

The e command (a GNU extension) in

sed -E 's/^\w+=.*/echo "&"/e'

executes the command that is found in pattern space and replaces the pattern space with the output. For this example, pattern space is ldap_bind_dn='${bind_dn}' , and is replaced with the output of echo "ldap_bind_dn='${bind_dn}'" ( & references the whole matched portion of the pattern space in echo "&" ). Since the argument of echo is in double quotes, it is subject to parameter expansion when it is executed by the shell.

Caveat : Make sure that the file (/tmp/var in this example) comes from a trusted source. Otherwise it may contain lines like foo='$(some_nasty_command)' , which is executed when the sed command above runs.