sed如何删除文件中的前17行和后8行

Question

I have a big file 150GB CSV file and I would like to remove the first 17 lines and the last 8 lines. I have tried the following but seems that's not working right

sed -i -n -e :a -e '1,8!{P;N;D;};N;ba' 

and

sed -i '1,17d' 

I wonder if someone can help with sed or awk, one liner will be great?

Answer 1

head和tail比sed或awk更适合工作。

tail -n+18 file | head -n-8 > newfile

Answer 2

awk -v nr="$(wc -l < file)" 'NR>17 && NR<(nr-8)' file

Answer 3

所有awk：

awk 'NR>y+x{print A[NR%y]} {A[NR%y]=$0}' x=17 y=8 file

Answer 4

Try this :

sed '{[/]<n>|<string>|<regex>[/]}d' <fileName>       
sed '{[/]<adr1>[,<adr2>][/]d' <fileName>

where

1. /.../=delimiters

2. n = line number

3. string = string found in in line

4. regex = regular expression corresponding to the searched pattern

5. addr = address of a line (number or pattern )

6. d = delete

Refer this link

Answer 5

LENGTH=`wc -l < file`
head -n $((LENGTH-8)) file | tail -n $((LENGTH-17)) > file

Edit: As mtk posted in comment this won't work. If you want to use wc and track file length you should use:

LENGTH=`wc -l < file`
head -n $((LENGTH-8)) file | tail -n $((LENGTH-8-17)) > file

or:

LENGTH=`wc -l < file`
head -n $((LENGTH-8)) file > file
LENGTH=`wc -l < file`
tail -n $((LENGTH-17)) file > file

What makes this solution less elegant than that posted by choroba :)

Answer 6

I learnt this today for the shell.

{
  ghead -17  > /dev/null
  sed -n -e :a -e '1,8!{P;N;D;};N;ba'
} < my-bigfile > subset-of

One has to use a non consuming head , hence the use of ghead from the GNU coreutils.

Answer 7

Similar to Thor's answer , but a bit shorter:

sed -i '' -e $'1,17d;:a\nN;19,25ba\nP;D' file.txt

The -i '' tells sed to edit the file in place. (The syntax may be a bit different on your system. Check the man page.)

If you want to delete front lines from the front and tail from the end, you'd have to use the following numbers:

1,{front}d;:a\nN;{front+2},{front+tail}ba\nP;D

(I put them in curly braces here, but that's just pseudocode. You'll have to replace them by the actual numbers. Also, it should work with {front+1} , but it doesn't on my machine (macOS 10.12.4). I think that's a bug.)

I'll try to explain how the command works. Here's a human-readable version:

1,17d     # delete lines 1 ... 17, goto start
:a        # define label a
N         # add next line from file to buffer, quit if at end of file
19,25ba   # if line number is 19 ... 25, goto start (label a)
P         # print first line in buffer
D         # delete first line from buffer, go back to start

First we skip 17 lines. That's easy. The rest is tricky, but basically we keep a buffer of eight lines. We only start printing lines when the buffer is full, but we stop printing when we reach the end of the file, so at the end, there are still eight lines left in the buffer that we didn't print - in other words, we deleted them.