使用Sed Mac终端查找和替换空格

Question

I have a .CSV file with over 500,000 lines that I need to:

1. find all 'space double quote space' sequences and replace with nothing
2. find all 'space double quote' sequences and replace with nothing
3. find all double quotes and replace with nothing

Example of .CSV line:

"DISH Hartford & New Haven  (Hartford)", "206", "FBNHD", " 06028", " East Windsor Hill", "CT", "Hartford County"

** Required output**

DISH Hartford & New Haven  (Hartford),206,FBNHD,06028,East Windsor Hill,CT,Hartford County

I need to remove all double quotes ( " ) and spaces in front of and behind the commas ( , ).

I've tried

$ cd /Users/Leonna/Downloads/
$ cat bs-B2Bformat.csv | sed s/ " //g

This gives me the 'command incomplete' greater than prompt, so I then tried:

$ cat bs-B2Bformat.csv | sed s/ " //g
sed: 1: "s/": unterminated substitute pattern
$ cat bs-B2Bformat.csv |sed s/ \" //g
sed: 1: "s/": unterminated substitute pattern
$

There are too many lines for me to edit in Excel (Excel won't load all the lines) or even a text editor. How can I fix this?

Answer 1

Quoted from here :

For POSIX compliance, use the character class [[:space:]] instead of \\s, since the latter is a GNU sed extension.

Based on that, I would suggest the following, which, as Jonathan Leffler pointed out, is portable across GNU and BSD implementations.

sed -E 's/[[:space:]]?"[[:space:]]?//g' <path/to/file>

The -E flag enables extended regular expressions on BSD implementations. On GNU sed it is undocumented, but as discussed here , it enables compatibility with the BSD standard.

Quoted from the manual for BSD sed :

-E Interpret regular expressions as extended (modern) regular expressions rather than basic regular expressions (BRE's).

Applying the above command on a file containing the following single line

"DISH Hartford & New Haven (Hartford)", "206", "FBNHD", " 06028", " East Windsor Hill", "CT", "Hartford County"

it yields

DISH Hartford & New Haven (Hartford),206,FBNHD,06028,East Windsor Hill,CT,Hartford County

Answer 2

这应该这样做：

sed -i 's/\(\s\|\)"\(\|\s\)//g' bs-B2Bformat.csv

Answer 3

This works for me. Is this what you want ?

 sed -e 's|", "|,|g' -e 's|^"||g' -e 's|"$||g' file.csv

 echo '"DISH Hartford & New Haven (Hartford)", "206", "FBNHD", " 06028", " East Windsor Hill", "CT", "Hartford County"' | sed -e 's|", "|,|g' -e 's|^"||g' -e 's|"$||g'

 DISH Hartford & New Haven (Hartford),206,FBNHD, 06028, East Windsor Hill,CT,Hartford County

Answer 4

One way is to use python and its csv module:

import csv 
import sys 

## Open file provided as argument.
with open(sys.argv[1], 'r') as f:

    ## Create the csv reader and writer. Avoid to quote fields in output.
    reader = csv.reader(f, skipinitialspace=True)
    writer = csv.writer(sys.stdout, quoting=csv.QUOTE_NONE, escapechar='\\')

    ## Read file line by line, remove leading and trailing white spaces and
    ## print.
    for row in reader:
        row = [field.strip() for field in row]
        writer.writerow(row)

Run it like:

python3 script.py csvfile

That yields:

DISH Hartford & New Haven  (Hartford),206,FBNHD,06028,East Windsor Hill,CT,Hartford County

Answer 5

What all of the current answers seemed to miss:

 $ cat bs-B2Bformat.csv | sed s/ " //g sed: 1: "s/": unterminated substitute pattern $ cat bs-B2Bformat.csv |sed s/ \\" //g sed: 1: "s/": unterminated substitute pattern $ 

The problem in the above is missing single quotes. It should have been:

$ cat bs-B2Bformat.csv | sed 's/ " //g'
                             ^        ^

Without the single quotes, bash splits at the spaces and sends three separate arguments (well at least for the case of \\" ). sed was seeing its first argument as just s/ .

Edit: FYI, single quotes are not required, they just make this case easier. If you want to use double quotes, just escape the one you want to keep for matching:

$ cat bs-B2Bformat.csv | sed "s/ \" //g"