过滤txt文件是否满足python中的某些条件？

Question

I have a txt file contained the subjectid_num_[dog/cat]_[option] .

ID1_0123_CAT_ANIMAL_3
ID1_0123_CAT_ANIMAL_GOOD_3
ID1_0123_ABC_3
ID2_1234_CAT_ANIMAL_3
ID2_1234_CAT_ANIMAL_GOOD_3
ID2_1234_DOG_ANIMAL_2
ID2_1234_DOG_ANIMAL_GOOD_0
ID2_1234_ABCD_3
ID3_4321_DOG_ANIMAL_1
ID3_4321_DOG_ANIMAL_GOOD_4
ID3_4321_DOG_3

I want to filter the file to get the output that satisfies the condition. For example, the below code will file the output has CAT and GOOD in the name, and does not contains the DOG and GOOD in the name. The name is determined by same subject_id and same number num . However, the code does not show my expected output. How should I fix it?

This is my code

with open("./cat_dog.txt", 'r') as f:
    files_list = [line.rstrip('\n') for line in f]
file_filter = []
for i, cat in enumerate(files_list):
    if 'GOOD' in cat and 'CAT' in cat:
        subject_id = cat.split('_')[0]
        num_id = cat.split('_')[1]
        subject_num = subject_id + '_' + num_id
        for j, dog in enumerate(files_list):
                if subject_num in dog and 'GOOD' in dog:
                    if 'GOOD' in dog and 'DOG' in dog:
                        continue;
                    else:
                        file_filter.append(cat)

The current output is

ID1_0123_CAT_ANIMAL_GOOD_3
ID2_1234_CAT_ANIMAL_GOOD_3

While the expected is

ID1_0123_CAT_ANIMAL_GOOD_3

Answer 1

Your code is wrong. Consider what happens when you check line ID2_1234_CAT_ANIMAL_GOOD_3 in the inner loop:

subject_id = cat.split('_')[0]            #ID2
num_id = cat.split('_')[1]                # 1234
subject_num = subject_id + '_' + num_id   #ID2_1234
for j, dog in enumerate(files_list):
        # when dog is the line ID2_1234_CAT_ANIMAL_GOOD_3
        if subject_num in dog and 'GOOD' in dog:   # this is true
            if 'GOOD' in dog and 'DOG' in dog:   # this is false
                continue;
            else:
                file_filter.append(cat)   # then it outputs it

The problem is that every line with GOOD and CAT in it will "match itself" in the inner loop.

IMHO I'd use itertools.groupby . Something along the lines of:

from itertools import groupby

def key(line):
    return line.split('_')[:2]

for key, lines in groupby(sorted(files_list, key=key), key=key):
    good_lines = [line for line in lines if 'GOOD' in line]
    if len(good_lines) == 1 and 'CAT' in good_lines[0]:
        file_filter.append(good_lines[0])

This should also be more efficient O(nlog n) vs O(n^2), although it needs all the contents of the file in RAM.

---

If you have other "classes" other than CAT and DOG and you want to output all GOOD CAT lines except if the subject_id is also a GOOD DOG you can modify the code above in this way:

is_good_cat = any('CAT' in line for line in good_lines)
is_good_dog = any('DOG' in line for line in good_lines)
if is_good_cat and not is_good_dog:
    file_filter.extend(line for line in good_lines if 'CAT' in good_lines)

(You need to use .extend and the loop because we no longer know which is the line to write, so you have to filter them.