Bash Array expansion with entries containing whitespaces

Question

Having a strange issue passing an array that has entires containing whitespaces to another function. The error occurs when the array in question is subsequently expanded from within the context of target function, it appears that whitespaces are erroneously interpreted as delimiters and thus:

1. Spaces between each word in each array entry are not preserved.
2. Each word directly preceded by a whitespace, occupies a separate index in the resulting array.

How can one ensure that arrays passed to a function have any spaces in their entries preserved following subsequent expansion?

The following code may be used to reproduce the error:

function caller(){

  #string to convert to array.
  str="var1=val1,var2=val2,var3=value\ with\ a\ space"

  #convert argument string to full name value pairs array
  fullNameValPairsArr=($(toNameValuePairsArray "$str"))

  callee fullNameValPairsArr    
}


#called by the top-level "caller" function and passed the array as an argument
function callee(){

  passedArray=$1[@]

  expandedPassedArray=("${!passedArray}")

  for curEntry in "${expandedPassedArray[@]}"
  do
    
       echo $curEntry
  
  done    

 }



#helper function converts arg string to name value pairs array.
function toNameValuePairsArray(){

  #store args string to local
  fullNameValPairsString="$1"

  #first split the full name-value pair parameters(via comma delimiter).
  IFS=','

  #Read the split words into an array based on space delimiter
  read -a nameValPairsArr <<< "$fullNameValPairsString"

 
  echo "${nameValPairsArr[@]}"      

}

caller

The output from the call to the top-level "caller" function is:

var1=val1                                                                                                                                                    
var2=val2                                                                                                                                                    
var3=value                                                                                                                                                   
with                                                                                                                                                         
a                                                                                                                                                            
space 

As you can see the last parameter that contains a space (ie value with a space) is split into several separate entries in the array; the spaces in the string are not preserved.

Answer 1

This was your main error:

  fullNameValPairsArr=($(toNameValuePairsArray "$str"))

Because the command substsitution is unquoted, the shell will perform word splitting on the output, breaking the argument with spaces into separate words.

---

I would use "namerefs" that allow you to pass variable names to functions.
Requires bash version 4.3+.
See Shell Parameters in the manual

if declare -n a=b 2>/dev/null; then
    unset a
else
    echo "This bash version ($BASH_VERSION) does not implement namerefs." >&2
    exit 1
fi

caller() {
    local str="var1=val1,var2=val2,var3=value\ with\ a\ space"
    local -a fullNameValPairsArr    

    toNameValuePairsArray "$str" fullNameValPairsArr    

    callee fullNameValPairsArr    
}


callee() {
    local -n passedArray=$1
    for curEntry in "${passedArray[@]}"; do
        echo "$curEntry"
    done    
}


toNameValuePairsArray() {
    local fullNameValPairsString="$1"
    local -n _ary=$2      # cannot use same varname as caller

    # don't set *global* IFS
    IFS=',' read -a _ary <<< "$fullNameValPairsString"
}

caller

outputs

var1=val1
var2=val2
var3=value with a space

---

I would go a step further and parse the input into an associative array:

# check bash version...

caller() {
    local str="var1=val1,var2=val2,var3=value\ with\ a\ space"
    #local -a fullNameValPairsArr    
    local -A fullNameValPairsArr    

    toNameValuePairsArray "$str" fullNameValPairsArr    

    callee fullNameValPairsArr    
}


callee() {
    local -n passedArray=$1
    for idx in "${!passedArray[@]}"; do
        echo "$idx => ${passedArray[$idx]}"
    done    
}


toNameValuePairsArray() {
    local fullNameValPairsString="$1"
    local -n _ary=$2      # cannot use same varname as caller

    # don't set *global* IFS
    IFS=',' read -a pairs <<< "$fullNameValPairsString"
    for pair in "${pairs[@]}"; do
        IFS="=" read var value <<<"$pair"
        _ary[$var]=$value
    done
}

caller

outputs

var1 => val1
var2 => val2
var3 => value with a space

---

One last note, bash 5.1 provides a new loadable csv command

BASH_LOADABLES_PATH="${BASH%/bin/bash}/lib/bash"
enable -f csv csv

str="var1=val1,var2=val2,var3=value\ with\ a\ space"
csv -a pairs "$str"

declare -p pairs

result

declare -a pairs=([0]="var1=val1" [1]="var2=val2" [2]="var3=value\\ with\\ a\\ space")

Documentation:

$ help csv
csv: csv [-a ARRAY] string
    Read comma-separated fields from a string.

    Parse STRING, a line of comma-separated values, into individual fields,
    and store them into the indexed array ARRAYNAME starting at index 0.
    If ARRAYNAME is not supplied, "CSV" is the default array name.

Answer 2

Here's what you're currently doing:

1. Carefully split the string into a correct array with read -a
2. Immediately join all the elements with spaces with echo
3. Split all the elements on spaces with word splitting

If you simply skip step 2 and 3, you won't have this problem.

There's no great way to return values from bash functions, but you can eg use a global variable:

function caller(){
  #string to convert to array.
  str="var1=val1,var2=val2,var3=value\ with\ a\ space"

  #convert argument string to full name value pairs array
  toNameValuePairsArray "$str"

# Copy from the global variable to an array of our choice
  fullNameValPairsArr=( "${toNameValuePairsArray_result[@]}" )
  callee fullNameValPairsArr
}

#called by the top-level "caller" function and passed the array as an argument
function callee(){
  passedArray=$1[@]
  expandedPassedArray=("${!passedArray}")
  for curEntry in "${expandedPassedArray[@]}"
  do
       echo $curEntry
  done
 }

#helper function converts arg string to name value pairs array.
function toNameValuePairsArray(){
  #store args string to local
  fullNameValPairsString="$1"
  #first split the full name-value pair parameters(via comma delimiter).
  IFS=','
  # "return" by assigning to a global variable
  read -a toNameValuePairsArray_result <<< "$fullNameValPairsString"
}

caller

This results in:

var1=val1
var2=val2
var3=value with a space