如何grep和从文件中删除分隔符之间的所有行

Question

I have a file that looks like this:

===SEPARATOR===
line2
line3
===SEPARATOR===
line5
line6
===SEPARATOR===
line8
...
lineX
===SEPARATOR===

How can I do a while loop and go through the file, dump anything between two ===SEPARATOR=== occurrences into another file for further processing? I want to add only line2, line3 to the second file on the first iteration. I will parse the file; and on the next iteration I want line5 line6 in second file to do the same parsing again but on different data.

Answer 1

You can exclude all lines matching ===SEPARATOR=== with grep -v and redirect the rest to a file:

grep -vx '===SEPARATOR===' file > file_processed

-x makes sure that only lines completely matching ===SEPARATOR=== are excluded.

Answer 2

This uses sed to find lines between separators, and then grep -v to delete the separators.

$ sed -n '/===SEPARATOR===/,/===SEPARATOR===/ p' file | grep -v '===SEPARATOR==='
line2
line3
line8
...
lineX

There's got to be a more elegant answer that doesn't repeat the separator three times, but I'm drawing a blank.

Answer 3

I am assuming that you do not need the line5 and line6 . You can do it with awk like this:.

awk '$0 == "===SEPARATOR===" {interested = ! interested; next} interested {print}'

Credit goes to https://www.gnu.org/software/gawk/manual/html_node/Boolean-Ops.html#Boolean-Ops

Output:

[root@hostname ~]# cat /tmp/1 | awk '$0 == "===SEPARATOR===" {interested = ! interested; next} interested {print}' /tmp/1
line2
line3
line8
...
lineX

Answer 4

awk to the rescue!

with multi-char support (eg gawk)

$ awk -v RS='\n?===SEPARATOR===\n' '!(NR%2)' file

line2
line3
line8
...
lineX

or without that

$ awk '/===SEPARATOR===/{p=!p;next} p' file

line2
line3
line8
...
lineX

which is practically the same with @Jay Rajput's answer.

Answer 5

It sounds like you want to save each block of lines to a separate file .

The following solutions create output files f1 , f2 , containing the (non-empty) blocks of lines betwen the ===SEPARATOR=== lines.

With GNU Awk or Mawk:

awk -v fnamePrefix='f' -v RS='(^|\n)===SEPARATOR===(\n|$)' \
  'NF { fname = fnamePrefix (++n); print > fname; close(fname) }' file

Pure bash - which will be slow :

#!/usr/bin/env bash

fnamePrefix='f'; i=0
while IFS= read -r line; do
  [[ $line == '===SEPARATOR===' ]] && { (( ++i )); > "${fnamePrefix}${i}"; continue; }
  printf '%s\n' "$line" >> "${fnamePrefix}${i}"
done < file