列表理解中的多个条件

Question

I have a list of nested dictionaries that looks as follows:

messages_all = [{'type': 'message',
      'subtype': 'bot_message',
      'text': "This content can't be displayed.",
      'ts': '1573358255.000100',
      'username': 'Userform',
      'icons': {'image_30': 'www.example.com'},
      'bot_id': 'JOD4K22SJW',
      'blocks': [{'type': 'section',
        'block_id': 'yCKUB',
        'text': {'type': 'mrkdwn',
         'text': 'Your *survey* has a new response.',
         'verbatim': False}},
       {'type': 'section',
        'block_id': '37Mt4',
        'text': {'type': 'mrkdwn',
         'text': '*Thanks for your response. Where did you first hear about us?*\nFriend',
         'verbatim': False}},
       {'type': 'section',
        'block_id': 'hqps2',
        'text': {'type': 'mrkdwn',
         'text': '*How would you rate your experience?*\n9',
         'verbatim': False}},
       {'type': 'section',
        'block_id': 'rvi',
        'text': {'type': 'mrkdwn', 'text': '*city*\nNew York', 'verbatim': False}},
       {'type': 'section',
        'block_id': 'q=L+',
        'text': {'type': 'mrkdwn',
         'text': '*order_id*\n123456',
         'verbatim': False}}]},

{'type': 'message',
  'subtype': 'channel_join',
  'ts': '1650897290.290259',
  'user': 'T01CTZE4MB6',
  'text': '<@U03CTDZ4MA6> has joined the channel',
  'inviter': 'A033AHJCK'},

{'type': 'message',
  'subtype': 'channel_leave',
  'ts': '1650899175.290259',
  'user': 'T01CTZE4MB6',
  'text': '<@U03CTDZ4MA6> has left the channel',
  'inviter': 'A033AHJCK'},

{'client_msg_id': '123456jk-a19c-97fe-35c9-3c9f643cae19',
  'type': 'message',
  'text': '<@ABC973RJD>',
  'user': 'UM1922AJG',
  'ts': '1573323860.000300',
  'team': 'B09AJR39A',
  'reactions': [{'name': '+1', 'users': ['UM1927AJG'], 'count': 1}]},

{'client_msg_id': '1234CAC1-FEC8-4F25-8CE5-C135B7FJB2E',
  'type': 'message',
  'text': '<@UM1922AJG> ',
  'user': 'UM1922AJG',
  'ts': '1573791416.000200',
  'team': 'AJCR23H',
  'thread_ts': '1573791416.000200',
  'reply_count': 3,
  'reply_users_count': 2,
  'latest_reply': '1573829538.002000',
  'reply_users': ['UM3HRC74J', 'UM1922AJG'],
  'is_locked': False,
  'subscribed': False}

]

I'd like to be able to filter out dictionaries with the following

client_msg_id
channel_join
channel_leave
reply_users_count

My code to do so is:

filtered_messages = [elem for elem in messages_all if not elem.get('client_msg_id')
                     or (elem.get('type') == 'message' and elem.get('subtype') == 'channel_join') 
                     or (elem.get('type') == 'message' and elem.get('subtype') == 'channel_leave')
                     or (elem.get('type') == 'message' and elem.get('reply_users_count') == 2)
                ]

From testing, it seems as though only the client_msg_id is being filtered out. The others are not.

Would someone please assist me with the syntax of this list comprehension?

Answer 1

IIUC, you're simply missing parentheses to negate the union of all the conditions:

filtered_messages = [elem for elem in messages_all if not (elem.get('client_msg_id')
                     or (elem.get('type') == 'message' and elem.get('subtype') == 'channel_join') 
                     or (elem.get('type') == 'message' and elem.get('subtype') == 'channel_leave')
                     or (elem.get('type') == 'message' and elem.get('reply_users_count') == 2))
                ]

This would keep only the first element of your input in the example.

output:

[{'type': 'message', 'subtype': 'bot_message', 'text': "This content can't be displayed.", 'ts': '1573358255.000100', 'username': 'Userform', 'icons': {'image_30': 'www.example.com'}, 'bot_id': 'JOD4K22SJW', 'blocks': [{'type': 'section', 'block_id': 'yCKUB', 'text': {'type': 'mrkdwn', 'text': 'Your *survey* has a new response.', 'verbatim': False}}, {'type': 'section', 'block_id': '37Mt4', 'text': {'type': 'mrkdwn', 'text': '*Thanks for your response. Where did you first hear about us?*\nFriend', 'verbatim': False}}, {'type': 'section', 'block_id': 'hqps2', 'text': {'type': 'mrkdwn', 'text': '*How would you rate your experience?*\n9', 'verbatim': False}}, {'type': 'section', 'block_id': 'rvi', 'text': {'type': 'mrkdwn', 'text': '*city*\nNew York', 'verbatim': False}}, {'type': 'section', 'block_id': 'q=L+', 'text': {'type': 'mrkdwn', 'text': '*order_id*\n123456', 'verbatim': False}}]}
]

Answer 2

Like @mozway said, it is simply some parentheses missing.

For such a large if condition, I would personnally go further and create a function:

def my_filter(elem):
    if not (elem.get('client_msg_id') 
      or (elem.get('type') == 'message' and elem.get('subtype') == 'channel_join') 
      or (elem.get('type') == 'message' and elem.get('subtype') == 'channel_leave') 
      or (elem.get('type') == 'message' and elem.get('reply_users_count') == 2)):
      return True
    return False

filtered_messages = [elem for elem in messages_all if my_filter(elem)]

Edit: delete extra boolean variable

Answer 3

Given the length of the resulting listcomp I would write something like this instead:

def filterdict(d):
    subtypes = {"channel_join", "channel_leave"}
    return any(
        test(d)
        for test in (
            lambda d: d["type"] == "message" and d.get("subtype") in subtypes,
            lambda d: d["type"] == "message" and d.get("reply_user_count") == 2,
            lambda d: d.get("client_msg_id"),
        )
    )


msgs = [x for x in messages_all if not filterdict(x)]

In this form:

• we have a filter fn which returns False for an interesting msg, so we can use it natively with itertools.filterfalse
• the conditions are clearly set out
• the use of lambdas and all ensures encapsulation of tests---a mislaid parenthesis is not going to cause the kind of problem which motivated the question
• we've wrapped two functionally idential tests in a test for membership, which is clearer and easier to read.

Whether one likes this kind of thing is going to be a matter of taste in the end.

Answer 4

I found out the get method is much slower than checking if a key is in the dictionary, so if you have big data it would be faster to go with check for existing key in dictionary:

filtered_messages = [elem for elem in messages_all
                     if "client_msg_id" not in elem
                     and not ("type" in elem
                              and not ('subtype' in elem
                                       and not (elem['subtype'] in ['channel_join', 'channel_leave']
                                                or ('reply_users_count' in elem
                                                    and elem['reply_users_count'] == 2))))]