使用 sed 重命名文本文件中的特定行

Question

I have a file that looks like this:

>alks|keep1|aoiuor|lskdjf
ldkfj
alksj
asdflkj
>jhoj_kl|keep2|kjghoij|adfjl
aldskj
alskj
alsdkj

I would like to edit just the lines starting with > , ideally in-place, to get a file:

>keep1
ldkfj
alksj
asdflkj
>keep2
aldskj
alskj
alsdkj

I know that in principle this is achievable with various combinations of sed/awk/cut, but I haven't been able to figure out the right combination. Ideally it should be fast - the file has many millions of lines, and many of the lines are also very long.

Key things about the lines I want to edit:

• Always start with >
• The bit I want to keep is always between the first and second pipe symbol | (hence thinking cut is going to help
• The bit I want to keep has alphanumeric symbols and sometimes underscores. The rest of the string on the same line can have any symbols

What I've tried that seems helpful

(Most of my sed attempts are pure garbage)

cut -d '|' -f 2  test.txt

Gets me the bit of the string that I want, and it keeps the other lines too. So it's close, but (of course) it doesn't preserve the initial > on the lines where cut applies, so it's missing a crucial part of the solution.

Answer 1

With sed :

sed -E '/^>/ s/^[^|]+\|([^|]+).*/>\1/'

• /^>/ to select lines starting with > , not strictly necessary for given sample but sometimes this provides faster result than using s alone
• ^[^|]+\| this will match non | characters from the start of line
• ([^|]+) capture the second field
• .* rest of the line
• >\1 replacement string where \1 will have the contents of ([^|]+)

If your input has only ASCII character, this would give you much faster results:

LC_ALL=C sed -E '/^>/ s/^[^|]+\|([^|]+).*/>\1/'

Timing

• Checking the timing results by creating a huge file from given input sample, awk is much faster and mawk is even faster
• However, OP reports that the sed solution is faster for the actual data

Answer 2

With your shown samples, you could simply try following. In this code, we are setting field separator as | for all the lines of Input_file then in main program checking if line starts from > then print 2nd field else print the complete line.

awk -F'|' '/^>/{print ">"$2;next} 1' Input_file

Explanation: Adding detailed explanation for above.

awk -F'|' '     ##Starting awk program from here and setting field separator as | here.
/^>/{           ##Checking condition if line starts from > then do following.
  print ">"$2   ##Printing 2nd field of current line here.
  next          ##next will skip all further statements from here.
}
1               ##Will print current line.
' Input_file    ##mentioning Input_file name here.

Answer 3

You can also use the following awk command:

awk  -F\| '/^>/{print ">"$2} !/^>/{print}' file
# Inplace replacement with gawk (GNU awk)
gawk -i inplace  -F\| '/^>/{print ">"$2} !/^>/{print}' file
# "Inline-like" replacement with any awk
awk -F\| '/^>/{print ">"$2} !/^>/{print}' file > tmp && mv tmp file

Here,

• -F\| - sets the field separator to a | char
• /^>/ is the condition: if line starts with < (and !/^>/ means the opposite)
• {print ">"$2} prints the Field 2 value with a > char prepended to it
• {print} simply prints the full line.

Note that since !/^>/{print} can be reduced to !/^>/ as print is the default action.

See an online demo :

s='>alks|keep1|aoiuor|lskdjf
ldkfj
alksj
asdflkj
>jhoj_kl|keep2|kjghoij|adfjl
aldskj
alskj
alsdkj'
awk  -F\| '/^>/{print ">"$2} !/^>/{print}' <<< "$s"

Output:

>keep1
ldkfj
alksj
asdflkj
>keep2
aldskj
alskj
alsdkj