有条件地将一些行从一个文件写入另一个文件

Question

I have a text file (fixed width) with following format:

a1   b   c1    d     -> header
1    2    3    4
6    4    3    5
a2   b    c2   d2    -> header
7    9    1    4
a    b1   c6   d2    -> header
8    9    3    4

From this file, I want to create another file with filtered rows. If the column with c has value 3 AND column b does not have value 2 then I want that row including the header. If the column c does not have value 3 then I do not want any rows including its header. The new file must therefore look like this:

a1   b   c1    d
6    4    3    5
a    b1   c6   d2
8    9    3    4

Also, the value 3 can only occur in column with c and 2 only in column with b . So, we don't even have to check the columns, as long as the filtering condition is met in a row. The only important thing is, if the conditions are met I also need the header for that row. If not, I also don't want header for that.

How can achieve this?

What I have tried is to read through the file in pandas with .read_fwf() and save each dataframe. After that, I filter the dataframe and write them to a file with .to_string() . This does exactly what I want (kinda) but the number of whitespaces are not consistent. Since, it is a fixed-width file, I want the new file to have the exact same format as the old file. I also tried to write with np.savetxt() but it also has issue with whitespaces. Shame, pandas does not have write_fwf .

So, maybe instead of pandas I could use plain python to do this? Or even a bash(powershell) script? Anything that works:)

Answer 1

This might work for you (GNU sed):

sed -nE ':a;/^a/{h;:b;n;/^\S+ +[^2 ]+ +3 /H;$bc;/^a/{:c;x;/\n/p;x;ba};bb}' file

This is a filtering operation, so set -n and extend regexps by using -E .

Make a copy of the current header and loop through the data rows appending those that meet the criteria to it.

At end of file or the next header, print the previous header and any rows.