如何按 substring 模式对列表进行排序到 dicts

Question

I am trying to sort a list of values based similar substrings. I would like to group this in in a dict of dicts of lists with a key being the similar substring and the value a list of those grouped value.

For example (the actual list has 24k entries):

test_list = [ 'Doghouse Amsterdam', 'Doghouse Antwerp', 'Doghouse Vienna', 
        'House by KatSkill', 'Garden by KatSkill', 'Meadow by KatSkill']

to:

resultdict = { 
'Doghouse' : ['Doghouse Amsterdam', 'Doghouse Antwerp', 'Doghouse Vienna'],
'by KatSkill' : [ 'House by KatSkill', 'Garden by KatSkill', 'Meadow by KatSkill' ]
}

I tried the following but that doesn't work at all.

from itertools import groupby 
test_list = [ 'Doghouse Amsterdam', 'Doghouse Antwerp', 'Doghouse Vienna', 
            'House by KatSkill', 'Garden by KatSkill', 'Meadow by KatSkill']


res = [list(i) for j, i in groupby(test_list, 
                          lambda a: a.partition('_')[0])]

Answer 1

mylist = [ 'Doghouse Amsterdam', 'Doghouse Antwerp', 'Doghouse Vienna', 
            'House by KatSkill', 'Garden by KatSkill', 'Meadow by KatSkill']
test = ['Doghouse', 'by KatSkill']

Use dict and list comprehension:

res = { i: [j for j in mylist if i in j] for i in test}

Or set up your dict and use a loop with list comprehension

resultdict = {}
for i in test:
     resultdict[i] = [j for j in mylist if i in j]

Answer 2

Initially, find all the sub strings separated by " " that appear in another string of the input list. In the process, build a dictionary containing all the corresponding sub strings as keys and the input strings as values. This returns a dictionary having only single sub strings as keys. Using the example this returns:

{'by': ['Garden by KatSkill', 'Meadow by KatSkill', 'House by KatSkill'], 'KatSkill': ['Garden by KatSkill', 'Meadow by KatSkill', 'House by KatSkill'], 'Doghouse': ['Doghouse Antwerp', 'Doghouse Vienna', 'Doghouse Amsterdam']}

To obtain the intended outcome, compaction is required. For compaction, it is beneficial to exploit the fact that each dictionary key is also a part of the dictionary's string lists. So iterate over the dictionary values and split the strings into sub strings again. Then iterate over the substrings in order of the substring list and determine the sub string list ranges that contain dictionary keys. Add the corresponding ranges to a new dict. For 24k entries this may take a while. See the souce code down below:

mylist = [ 'Doghouse Amsterdam', 'Doghouse Antwerp', 'Doghouse Vienna', 
        'House by KatSkill', 'Garden by KatSkill', 'Meadow by KatSkill']

def findSimilarSubstrings(list):
    res_dict = {}
    for string in list:
        substrings = string.split(" ")
        for otherstring in list:
            # Prevent check with the same string
            if otherstring == string:
                continue
            for substring in substrings:
                if substring in otherstring:
                   if not(substring in res_dict):
                       res_dict[substring] = []
                   # Prevent duplicates
                   if not(otherstring in res_dict[substring]):
                       res_dict[substring].append(otherstring)
    return res_dict

def findOverlappingLists(dict):
    res_dict = {}
    for list in dict.values():
        for string in list:
            substrings = string.split(" ")
            lastIndex = 0
            lastKeyInDict = False
            substring = ""
            numsubstrings = len(substrings)
            for i in range(len(substrings)):
               substring = substrings[i]
               if substring in dict:
                    if not(lastKeyInDict):
                        lastIndex = i
                        lastKeyInDict = True
               elif lastKeyInDict:
                   commonstring = " ".join(substrings[lastIndex:i])
                   # Add key string to res_dict
                   if not(commonstring in res_dict):
                      res_dict[commonstring] = []
                   # Prevent duplicates
                   if not(string in res_dict[commonstring]):
                      res_dict[commonstring].append(string)
                   lastKeyInDict = False
            # Handle last substring
            if lastKeyInDict:
                commonstring = " ".join(substrings[lastIndex:numsubstrings])
                if not(commonstring in res_dict):
                    res_dict[commonstring] = []
                if not(string in res_dict[commonstring]):
                    res_dict[commonstring].append(string)
    return res_dict

# Initially find all the substrings (seperated by " ") returning:
# {'by': ['Garden by KatSkill', 'Meadow by KatSkill', 'House by KatSkill'],
#  'KatSkill': ['Garden by KatSkill', 'Meadow by KatSkill', 'House by KatSkill'],
#  'Doghouse': ['Doghouse Antwerp', 'Doghouse Vienna', 'Doghouse Amsterdam']}
similiarStrings = findSimilarSubstrings(mylist)
# Perform a compaction on similiarStrings.values() by lookup in the dictionary's key set
resultdict = findOverlappingLists(similiarStrings)

Answer 3

Here's a perhaps simpler/faster implementation

from collections import Counter
from itertools import groupby
import pprint

# Strategy:
# 1.  Find common words in strings in list
# 2.  Group strings which have the same common words together

def find_common_words(lst):
  " finds strings with common words "
  cnt = Counter()
  for s in lst:
    cnt.update(s.split(" "))

  # return words which appear in more than one string
  words = set([k for k, v in cnt.items() if v > 1])
  return words
  
def grouping_key(s, words):
  " Key function for grouping strings with common words in the same sequence"
  k = []
  for i in s.split():
    if i in words:
      k.append(i)
  return ' '.join(k)

def calc_groupings(lst):
  " Generate the string groups based upon common words "
  common_words = find_common_words(lst)

  # Group strings with common words
  g = groupby(lst, lambda x: grouping_key(x, common_words))

  # Result
  return {k: list(v) for k, v in g}

t = ['Doghouse Amsterdam', 'Doghouse Antwerp', 'Doghouse Vienna', 
        'House by KatSkill', 'Garden by KatSkill', 'Meadow by KatSkill']

pp = pprint.PrettyPrinter(indent=4)
pp.pprint(calc_groupings(t))

Output

{   'Doghouse': ['Doghouse Amsterdam', 'Doghouse Antwerp', 'Doghouse Vienna'],
'by KatSkill': [   'House by KatSkill',
                   'Garden by KatSkill',
                   'Meadow by KatSkill']}