How to pass a variable in a JSON string in bash script

Question

i have below script while gets some command output from a host and keeps that into a file /tmp/${stcl}_aggr.txt , further this file is placed to a variable body=$(cat /tmp/${stcl}_aggr.txt)

While calling this Variable body into a Jason arrays as "description": "$body" its value is not getting expanded and resulting in error as KeyError: 'result' , and the varible like incident_number , curloutput also not working while if i remove that body for "description": then script will run but the i need "description": value to be uploaded.

#!/bin/bash
#
DATE=$(date +"%m_%d_%Y_%H_%M")
mailto="udc@example.com"
cwd="/myenv/filer-mgt/bin"
output="/my_env/filer-mgt/rpt/$DATE"
clusters="udcl301 "

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:$PATH
incident=$1
sysid=UNKNOWN

checkvaultenv=`echo $vaultenv | tr '[:upper:]' '[:lower:]'`
if [[  "$checkvaultenv" == "" ]]
then
  vaultenv=prod
else
  if [[  "$checkvaultenv" != "dev" &&  "$checkvaultenv" != "test" && "$checkvaultenv" != "acc" && "$checkvaultenv" != "prod" ]]
  then
    echo unknown vault environment specified: $vaultenv
    exit 1
  fi
fi
vaultenvlc=`echo $vaultenv | tr '[:upper:]' '[:lower:]'`
vaultenvuc=`echo $vaultenv | tr '[:lower:]' '[:upper:]'`


SNOW_USERNAME=`export EDITOR=cat ; ansible-vault edit --vault-password-file=~/.ssh/vpf.iss.${vaultenvlc} ~/iss/vaults/vault-${vaultenvuc}.yml | grep vault_servicenow_username | grep -v -E  "^\s*#" | awk '{print $2}' FS=\"`
SNOW_PASSWORD=`export EDITOR=cat ; ansible-vault edit --vault-password-file=~/.ssh/vpf.iss.${vaultenvlc} ~/iss/vaults/vault-${vaultenvuc}.yml | grep vault_servicenow_password | grep -v -E  "^\s*#" | awk '{print $2}' FS=\"`

case $vaultenvlc in
  ram)
    SNOW_INSTANCE=udc2dev.service-now.com
    ;;
  iqa)
    SNOW_INSTANCE=udc2dev.service-now.com
    ;;
  dev)
    SNOW_INSTANCE=udc2dev.service-now.com
    ;;
  test)
    SNOW_INSTANCE=udc2trn.service-now.com
    ;;
  acc)
    SNOW_INSTANCE=udc2qa.service-now.com
    ;;
  *)
    SNOW_INSTANCE=udc2.service-now.com
    ;;
esac

SNOW_AUTH=`echo -n $SNOW_USERNAME:$SNOW_PASSWORD`

create_incident()
{
  url="https://${SNOW_INSTANCE}/api/now/table/incident"
  curloutput=`curl -sslv1 -u $SNOW_AUTH -X POST -s -H "Accept:application/json" -H "Content-Type:application/json" $url -d "$variables" 2>/dev/null`
  RETURNCODE=$?
  if [ "$RETURNCODE" == "0" ]
  then
    incident_number=`echo $curloutput |  python -c $'import sys, json\nprint json.load(sys.stdin)["result"]["number"]'`
    sysid=`echo $curloutput |  python -c $'import sys, json\nprint json.load(sys.stdin)["result"]["sys_id"]'`
    echo "OK: created incident $incident_number with sysid $sysid"
  else
    echo "ERROR creating incident:"
    echo "======================================="
    echo $curloutput
    echo "======================================="
  fi
}

#create_incident
for udcl in `echo $clusters`
do
  ssh $udcl aggr show -root false | grep -i "9[2-9]%\|100%" >  /tmp/${udcl}_aggr.txt
  snshort_description="aggr utilization above limit"
  sncmdb_ci=$udcl
  body="`cat /tmp/${udcl}_aggr.txt`"

read -r -d '' variables <<- EOM
{
  "short_description": "$snshort_description",
  "assignment_group":  "RD-DI-Infra-Storage",
  "contact_type":      "interface",
  "state":             "New",
  "urgency":           "3 - Low",
  "impact":            "3 - Low",
  "cmdb_ci":           "$sncmdb_ci",
  "u_sec_env":         "Normal Secure",
  "description":       "$body"
}
EOM

echo $variables

  create_incident


ssh $udcl "ro 0;vol show -state offline -fields aggregate"|grep stv > /tmp/${udcl}_validate_offline_volume.txt
done

Error:

i see below error while running the above..

[root@inv0104 tmp]# ./K_storage_healt_check.sh
{ "short_description": "aggr utilization above limit", "assignment_group": "RD-DI-Infra-Storage", "contact_type": "interface", "state": "New", "urgency": "3 - Low""' }mpact": "3 - Low", "cmdb_ci": "udcl301", "u_sec_env": "Normal Secure", "description": '"udc3019_ssd02 93.89TB 6.10TB 93% online 391 udc3019 raid_dp,
Traceback (most recent call last):
  File "<string>", line 2, in <module>
KeyError: 'result'
Traceback (most recent call last):
  File "<string>", line 2, in <module>
KeyError: 'result'

While running in the bebug mode i see more error like..

+ curloutput='{"error":{"detail":"Cannot decode: java.io.StringReader@6b4762","message":"Exception while reading request"},"status":"failure"}'
+ RETURNCODE=0
+ '[' 0 == 0 ']'
++ echo '{"error":{"detail":"Cannot' decode: 'java.io.StringReader@6b4762","message":"Exception' while reading 'request"},"status":"failure"}'
++ python -c 'import sys, json
print json.load(sys.stdin)["result"]["number"]'
Traceback (most recent call last):
  File "<string>", line 2, in <module>
KeyError: 'result'
+ incident_number=
++ echo '{"error":{"detail":"Cannot' decode: 'java.io.StringReader@6b4762","message":"Exception' while reading 'request"},"status":"failure"}'
++ python -c 'import sys, json
print json.load(sys.stdin)["result"]["sys_id"]'
Traceback (most recent call last):
  File "<string>", line 2, in <module>
KeyError: 'result'
+ sysid=
+ echo 'OK: created incident  with sysid '
OK: created incident  with sysid

looks like an issue with json or python which i am unable to get here.

Problematic Section:

 ssh $udcl aggr show -root false | grep -i "9[2-9]%\|100%" >  /tmp/${udcl}_aggr.txt
  body="`cat /tmp/${udcl}_aggr.txt`"

read -r -d '' variables <<- EOM
{
  "description":       "$body"
}
EOM

If i'll remove the $body then whole script will run just fine hence its all about the variable $body which is not getting intrepreted.

Answer 1

I can reproduce similar to my lab, So, as per my understanding below should work for you.

ssh $udcl aggr show -root false | grep -i "9[2-9]%\|100%"  | sed 's/\(.*\)\r/"\1"/g' >  /tmp/${udcl}_aggr.txt

Explanation:

Your input file has carriage returns at the end of the lines, so to remove the same we can use sed as above after "grep" and then you dont need to double quote your body variable as you data line inside the variable already been quoted and that json likes to be the way.