How to use quotes in environment variables expansion

Question

In How can I use environment variables in body of a curl PUT request? , I was given the great advise to always use " using doing environment variables.

Say I want do following query:

curl -XPUT http://"${HOST}"/create/"${USER}" -d'{"user":"'"${USER}"'"}'

I enclosed ${USER} between " to ensure that spaces in the user name are possible. I did the same for ${HOST} , although that was strictly not required, since hostnames cannot contain spaces as far as I know.

I am wondering if the following request is equal to the previous request:

curl -XPUT "http://${HOST}/create/${USER}" -d'{"user":"'"${USER}"'"}'

Are they equal? Which one is preferred/most standard?

Answer 1

Yes, they are equal.

I'd prefer

curl -XPUT "http://${HOST}/create/${USER}" -d"{\"user\":\"${USER}\"}"

first because:

1. it is shorter as @Ryan said in comment
2. second literal is more readable when in one chunk rather than concatenatig two styles of quotes
3. some editors will highlight them in more readable way (for example vim )

Answer 2

As you've seen, dealing with quoting conventions in Bash when you have arbitrary data is difficult. However, there's a third way of quoting in cases like this than can make life a lot easier: "here documents" .

Using <<TOKEN in a shell command indicates that the lines after the command will be read as the standard input to the command, terminated with TOKEN . Within the here document the usual quoting characters lose their special meaning and are interpreted literally, but variable substitution still happens normally.

To demonstrate, start a netcat "server" to display requests in one terminal with

nc -kl localhost 8888

Now, in another terminal, run this shell script:

name="Alice's Restaurant"
password="quote is ' and doublequote is \\\"."
curl -XPUT http://localhost:8888/create/user --data-binary @- <<EOF
{
  "name": "$name",
  "password": "$password",
  "created": "$(date --iso-8601)"
}
EOF

When a --data argument is given @ that requests that curl read the data from the filename specified immediately after the @ , and using - as the filename reads from stdin.

Note that here I use --data-binary to make the server's output easier to understand; in production use you'd want to use --data-urlencode or, if the server accepts data in another format, ensure you're setting the Content-type header to that format rather than leaving it at the default application/x-www-form-urlencoded .

When you run the above, you'll see the following in your netcat terminal:

PUT /create/user HTTP/1.1
Host: localhost:8888
User-Agent: curl/7.52.1
Accept: */*
Content-Length: 112
Content-Type: application/x-www-form-urlencoded

{
  "name": "Alice's Restaurant",
  "password": "quote is ' and doublequote is \".",
  "created": "2018-02-20"
}

As you can see, normal quoting characters are not treated specially, you need not do any special quoting on individual shell variables that get expanded within the here document, and you can even use $() to run shell commands whose output will be substituted within the document.

(By the way, I specified the double quote within the password variable as \\\\\\" , setting it to \\" in the variable after shell interpolation of a double-quoted string, because that's necessary to produce valid JSON. Oh, you can never escape the quoting issues.)