如何用反斜杠和空格“\\”替换空格，以便 bash shell 脚本可以从 .tsv 文件中读取文件名并执行 rsync 复制

Question

I have a script which takes source and destination information from a tsv file separated by space. The first column indicates source file path, and second column is destination. My rsync command reads source and destination information and performs copy operation.

But the issue is source and destination files both contain filename with white space (header-background copy.jpg) and as we know that when bash shell reads a file name (with space), it replace the space with backslash followed by space “\\ ”

/data-prod/bigdata/abc/test/1143-1003-004_1143-1003-905/static/common/images/header-background copy.jpg /mapped-data/data20/data3/header-background copy.jpg

My question is how I can replace the space with “\\ ” so shell can read it. I tried with using below sed command

sed -r 's/^\\s+//;s/\\s+/\\\\ /g' test2.tsv

but there is a problem as above sed command also adds a backslash after the source path. As I have mentioned that my script takes source and destination information from the .tsv file so having a slash added is a problem here. Below is the output of the sed command.

/data-prod/bigdata/abc/test/1143-1003-004_1143-1003-905/static/common/images/header-background\\ copy.jpg\\ /mapped-data/data20/data3/header-background\\ copy.jpg

what I want is something like covert from

/data-prod/bigdata/abc/test/1143-1003-004_1143-1003-905/static/common/images/header-background copy.jpg /mapped-data/data20/data3/header-background copy.jpg

to

/data-prod/bigdata/abc/test/1143-1003-004_1143-1003-905/static/common/images/header-background\\ copy.jpg /mapped-data/data20/data3/header-background\\ copy.jpg

Answer 1

tsv file separated by space

At least use a tab , because a tab is less likely to appear in a path than a space .

Remark : Did you know that only the / and the NULL char are forbidden in filenames in Linux filesystems? That means that everything but the NULL char can appear in a path...

Let's say that your file is now tab delimited and your paths don't include newlines nor tabs . Here's how you can read it in BASH:

while IFS=$'\t' read filepath1 filepath2
do
    declare -p filepath1 filepath2
done <<< "/data-prod/bigdata/abc/test/1143-1003-004_1143-1003-905/static/common/images/header-background copy.jpg"$'\t'"/mapped-data/data20/data3/header-background copy.jpg"

Output:

declare -- filepath1="/data-prod/bigdata/abc/test/1143-1003-004_1143-1003-905/static/common/images/header-background copy.jpg"
declare -- filepath2="/mapped-data/data20/data3/header-background copy.jpg"

If you need to explicitly escape a variable then you can use printf '%q'

filepath1="/data-prod/bigdata/abc/test/1143-1003-004_1143-1003-905/static/common/images/header-background copy.jpg"
filepath2="/mapped-data/data20/data3/header-background copy.jpg"

rsync -av "$filepath1" user@server:"$(printf '%q' "$filepath2")"

Answer 2

Using sed , one way would be to group the match and return it with a back reference appending the back slash

sed 's/\([A-Za-z0-9\/][^\.]*\) /\1\\ /g' input_file
/data-prod/bigdata/abc/test/1143-1003-004_1143-1003-905/static/common/images/header-background\ copy.jpg   /mapped-data/data20/data3/header-background\ copy.jpg

Answer 3

Split the columns with parameter substitution:

while IFS= read -r line; do
    # count raw tabs to validate format
    tabs=${line//[!$'\t']}
    tabs=${#tabs}

    if ((tabs!=1)); then
        echo "${line//$'\t'/TAB}: less or more than one raw tab, skipping…" >&2
    else
        # split by tab
        source=${line%$'\t'*}
        target=${line#*$'\t'}

        rsync "$source" "$target"
    fi

done < list.tsv

The loop checks that each line has exactly one tab, and skips it if it doesn't. Add in your rsync options etc.

Make sure the list doesn't have symbolic escape characters (like \\t or \\n ), or any other escapes/quotes that aren't part of a literal file name. tsv and csv can have all sorts of different non-literal quoting, that needs to be parsed correctly.

Also note this usage of rsync , for copying files names from a list file to a single target: rsync [opts] —files-from="my-source-list" /my/target/