Remove row from a file perl

Question

I have file with | delimited row in that i want to add check on the value of 8th position if the value matches i want to remove that row from the file and if it not matching i want to leave that in file.

Below is the file format , i want to remove all the rows which have U value on the 8th position

A|B|DADD|H|O| |123 A Street; Apt.2|U|M
A|B|DADD|H|O| |123 A Street; Apt.2|A|M
A|B|DADD|H|O| |123 A Street; Apt.2|B|M
A|B|DADD|H|O| |123 A Street; Apt.2|U|M 

How we can do it this Perl or is there any way we can use Awk or Sed. But after removing i want to print them as well .

I have tried sed but is matching through out the file i want to match at specific position.

sed -i '' "/$pattern/d" $file

Answer 1

perl -F'\|' -wlane'print if $F[7] ne "U"' file  > new

With -a switch each line is split into words, available in @F array. The separator to split on can be set with -F option (default is whitespace) and here it's | . See switches in perlrun . Then we just check for the 8th field and print.

In order to change the input file in-place add -i switch

perl -i -F'\|' -wlane'print if $F[7] ne "U"' file

or use -i.bak to keep ( .bak ) backup as well.

---

I see that a question popped up about logging those lines that aren't kept in the file.

One way is to hijack the STDERR stream for them

perl -i -F'\|' -wlane'$F[7] ne "U" ? print : print STDERR $_' file 2> excluded

where the file excluded gets the STDERR stream, redirected (in bash) using 2> . However, that can be outright dangerous since now possible warnings are hidden and corrupt the file intended for excluded lines (as they also go to that file).

So better collect those lines and print them at the end

perl -i -F'\|' -wlanE'
    $F[7] ne "U" ? print : push @exclude, $_; 
    END { say for @exclude }
' input > excluded

where file excluded gets all omitted (excluded) lines. (I switched -e to -E so to have say .)

Answer 2

Sounds like this might be what you want:

$ cat file
A|B|DADD|H|O| |123 A Street; Apt.2|U|M
A|B|DADD|H|O| |123 A Street; Apt.2|A|M
A|B|DADD|H|O| |123 A Street; Apt.2|B|M
A|B|DADD|H|O| |123 A Street; Apt.2|U|M

$ awk -i inplace -F'[|]' '$8=="U"{print|"cat>&2"; next} 1' file
A|B|DADD|H|O| |123 A Street; Apt.2|U|M
A|B|DADD|H|O| |123 A Street; Apt.2|U|M

$ cat file
A|B|DADD|H|O| |123 A Street; Apt.2|A|M
A|B|DADD|H|O| |123 A Street; Apt.2|B|M

The above uses GNU awk for -i inplace . With other awks you'd just do:

awk -F'[|]' '$8=="U"{print|"cat>&2"; next} 1' file > tmp && mv tmp file

To log the deleted line to a file named log1 :

awk -F'[|]' '$8=="U"{print >> "log1"; next} 1' file

To log it and print it to stderr:

awk -F'[|]' '$8=="U"{print|"tee -a log1 >&2"; next} 1' file