Replace all String regex matches with environment variables in bash

Question

I am trying to figure out how to replace matches of regex in a file with the value of an environment variable called a substring of that matched string. Each matched String also has a default along with it, separated by a colon. What I mean is if I have a file called myFile.properties that contains:

varWithDefault=${envVar1:default1}
varSetNoDefault=${envVar2}
varEmptyValue=${emptyEnvVar}
varEmptyValueWithDefault=${emptyEnvVar:3}
varNotSetWithDefault=${notSetEnvVar:I have : a colon}
# required but no match will prob be an error message
varNotSetNoDefault=${notSetEnvVar}

And I set the following environment variables:

export envVar1=value1
export envVar2=value2
export emptyEnvVar=""

then the file should update to be:

 varWithDefault=value1
 varSetNoDefault=value2
 varEmptyValue=
 varEmptyValueWithDefault=
 varNotSetWithDefault=I have : a colon
 # required but no match will prob be an error message
 varNotSetNoDefault=${notSetEnvVar}

I have something that for the most part works... but doesn't handle when an environment is set to empty string, nor does it handle defaults properly.

# get any variables we would like to replace in the myFile.properties file
# after the colon I say "anything except for a closing bracket" so that it will allow for colons in the value
varStrings=$( grep -Po '\${([_a-zA-Z][_a-zA-Z0-9]*)(:[^}]*)*}' myFile.properties )

# loop through the variables found and replace them with the value of the corresponding environment variable
# varString is a value that looks like:  "${my_variable:my_default}"
for varString in ${varStrings[@]} ; do

  # ideally grab these values from the matched regex, but I can't seem to figure out how to do that
  # propName would be:  "my_variable"
  # defaultValue would be:  "my_default"
  propName=$varString[0]
  defaultValue=$varString[1]

  # this technically gets the values, but I would also need to remove the "${}"
  propName="$( cut -d ':' -f 1- <<< "$varString" )"
  defaultValue="$( cut -d ':' -f 2- <<< "$varString" )"

  # $varString will be a String in the format '${my_variable:my_default}' so I need to strip the default chunk from it before doing this
  # but... to get the environment variable value if there was no default:
  envValue=`eval echo $varString`

  # if there is a matching environment variable, do the replacement; otherwise, spit out a warning
  # the -z will also fail the if check if the value is an empty string, which I don't want.  I only want it to go to the else clause if it was not set.  Not sure how to do that.
  if [ ! -z "$envValue" ]; then
    echo "Replacing $varString with environment variable value '$envValue'"
    sed -i "s|$varString|$envValue|g" myFile.properties
  else
    # set the default value
    if [[ noDefaultValueGiven ]] ; then
       echo "Warning: No environment variable defined for $envVarName.  String not replaced."
    else
       echo "Warning: No environment variable '$envVarName' defined.  Using default value '$defaultValue'."
       sed -i "s|$varString|$defaultValue|g" myFile.properties
    fi
  fi
done

The two big issues I'm having with this is:
1. How to loop through each regex match and have access to both regex groups (the sections surrounded by parenthesis)
2. How to check if an environment variable exists based on a string representation of said variable (ie check if "${my_variable}" is set, not just if it is empty)

Does anyone know how this should be done? I'm used to SpringBoot doing this for me.

Answer 1

You may do it in pure BASH regex:

re='^([^=]+=)\$\{([_a-zA-Z][_a-zA-Z0-9]*)(:([^}]*))?\}'

while IFS= read -r line; do
   if [[ $line =~ $re ]]; then                       # match regex
      #declare -p BASH_REMATCH

      var="${BASH_REMATCH[2]}"                       # var name in group #2
      if [[ -n ${!var+set} ]]; then                  # if var is set in env
         line="${BASH_REMATCH[1]}${!var}"            # use var value
      elif [[ -n ${BASH_REMATCH[4]} ]]; then.        # if default value is set
         line="${BASH_REMATCH[1]}${BASH_REMATCH[4]}" # use default string
      fi
   fi
   echo "$line"                                      # print each line
done < file.properties

Output:

varWithDefault=value1
varSetNoDefault=value2
varEmptyValue=
varEmptyValueWithDefault=
varNotSetWithDefault=I have : a colon
# required but no match will prob be an error message
varNotSetNoDefault=${notSetEnvVar}

Code Demo