Sed / Awk / Cut GNU将文本行转换为单行

Question

I have the following type of data:

3869|Jennifer Smith
10413 NE 71st Street
Vancouver, WA
98662
360-944-9578
jsmith@yahoo.com|1234567890123456|03-2013|123
--
3875|Joan L Doe
422 1/2 14th Ave E
Seattle, WA
98112
206-322-7666
jldoe@comcast.net|1234-1234-1234-1234|03-2013|123
--
3862|Dana Doe
24235 NE 7th Pl
Sammamish, WA
98074
425 868-2227
jsmith@hotmail.com|1234567890123456|03-2013|123
--
3890|John Smith
10470 SW 67th Ave
Tigard, OR
97223
5032205213
john.smith@gmail.com|1234567890123456|03-2013|123

I need to transform it to:

3869|Jennifer Smith|10413 NE 71st Street|Vancouver, WA|98662|360-944-9578|jsmith@yahoo.com|1234567890123456|03-2013|123
3875|Joan L Doe|422 1/2 14th Ave E|Seattle, WA|98112|206-322-7666|jldoe@comcast.net|1234-1234-1234-1234|03-2013|123
3862|Dana Doe|24235 NE 7th Pl|Sammamish, WA|98074|425 868-2227|jsmith@hotmail.com|1234567890123456|03-2013|123
3890|John Smith|10470 SW 67th Ave|Tigard, OR|97223|5032205213|john.smith@gmail.com|1234567890123456|03-2013|123

or better:

3869|Jennifer Smith|10413 NE 71st Street|Vancouver|WA|98662|360-944-9578|jsmith@yahoo.com|1234567890123456|03-2013|123
3875|Joan L Doe|422 1/2 14th Ave E|Seattle|WA|98112|206-322-7666|jldoe@comcast.net|1234-1234-1234-1234|03-2013|123
3862|Dana Doe|24235 NE 7th Pl|Sammamish|WA|98074|425 868-2227|jsmith@hotmail.com|1234567890123456|03-2013|123
3890|John Smith|10470 SW 67th Ave|Tigard|OR|97223|5032205213|john.smith@gmail.com|1234567890123456|03-2013|123

any idea how to automate this using GNU sed, awk, cu or perl/python whatever... Thank you!

Answer 1

Using sed

sed -n ':a;$!N;/--/!s/\n/|/g;ta;P' inputFile


$ sed -n ':a;$!N;/--/!s/\n/|/g;ta;P' temp 
3869|Jennifer Smith|10413 NE 71st Street|Vancouver, WA|98662|360-944-9578|jsmith@yahoo.com|1234567890123456|03-2013|123
3875|Joan L Doe|422 1/2 14th Ave E|Seattle, WA|98112|206-322-7666|jldoe@comcast.net|1234-1234-1234-1234|03-2013|123
3862|Dana Doe|24235 NE 7th Pl|Sammamish, WA|98074|425 868-2227|jsmith@hotmail.com|1234567890123456|03-2013|123
3890|John Smith|10470 SW 67th Ave|Tigard, OR|97223|5032205213|john.smith@gmail.com|1234567890123456|03-2013|123

Explaination:

• :a Create a label a.
• $! If not the last line; do
• N get a new line
• /--/! if line does not match this regex; do
• /s/\\n/|/g substitute new line with pipe
• ta branch back to label if the substitution was successful
• P print the line.

Note: Here is the difference between p , P , n and N .

• The n command will print out the current pattern space and read in the next line of input.
• The N command does not print out the current pattern space. It reads in the next line, but appends a new line character along with the input line itself to the pattern space.
• The p command prints the entire pattern space.
• The P command only prints the first part of the pattern space, up to the NEWLINE character.

Answer 2

I don't think it is very nice, but it nearly works (missing last line):

$ awk '{if (/^--/) {print a; a=""} else { a=a"|"$0}}' file
|3869|Jennifer Smith|10413 NE 71st Street|Vancouver, WA|98662|360-944-9578|jsmith@yahoo.com|1234567890123456|03-2013|123
|3875|Joan L Doe|422 1/2 14th Ave E|Seattle, WA|98112|206-322-7666|jldoe@comcast.net|1234-1234-1234-1234|03-2013|123
|3862|Dana Doe|24235 NE 7th Pl|Sammamish, WA|98074|425 868-2227|jsmith@hotmail.com|1234567890123456|03-2013|123

Update

If you add an extra

--

at the end of your file, it completly works:

$ awk '{if (/^--/) {print a; a=""} else { a=a"|"$0}}' file
|3869|Jennifer Smith|10413 NE 71st Street|Vancouver, WA|98662|360-944-9578|jsmith@yahoo.com|1234567890123456|03-2013|123
|3875|Joan L Doe|422 1/2 14th Ave E|Seattle, WA|98112|206-322-7666|jldoe@comcast.net|1234-1234-1234-1234|03-2013|123
|3862|Dana Doe|24235 NE 7th Pl|Sammamish, WA|98074|425 868-2227|jsmith@hotmail.com|1234567890123456|03-2013|123
|3890|John Smith|10470 SW 67th Ave|Tigard, OR|97223|5032205213|john.smith@gmail.com|1234567890123456|03-2013|123

This happens because my code waits for an -- to print what is buffering.

Answer 3

A slightly more idiomatic awk solution:

awk -F'\n' -vRS='\n--\n' -vOFS='|' '{$1=$1;print}' test.in

Tell it incoming records are separated by a line consisting of -- and fields are separated by newlines, and outgoing fields should be separated by | and records should be separated by the standard newline. $1 = $1 forces a reformatting conforming to this.

If the file doesn't end with a -- , you will get an extra | on the end, if you need to avoid this you can change this slightly:

awk -F'\n' -vRS='\n--\n' -vOFS='|' '{if($NF==""){NF--}$1=$1;print}' test.in