如何从python中的连接字符串中提取有意义和频繁的单词？

Question

I have a list of concatenated string as given below which I wish to split into meaningful and frequent words. Code which I have created is giving me all sorts of un-frequent words as well.

con_words = ["stainlesssteel", "screwhammerwing", "goldplated", "bearingball", "inchcountry"]

Expected output:

{"stainlesssteel": ["stainless", "steel"], 
"screwhammerwing": ["screw", "hammer", "wing"], 
"goldplated": ["gold", "plated"], 
"bearingball": ["bearing", "ball"], 
"inchcountry": ["inch", "country"]}

My code

from nltk.corpus import words
from nltk.corpus import stopwords

#list of all words from english dictionary
words = words.words()

#list of all english stopwords
stops = list(set(stopwords.words('english')))

alphabets = [chr(x) for x in range(ord('a'), ord('z') + 1)]
cleaned_word_list = list(set(words)|set(stops))
cleaned_word_list = list(set(cleaned_word_list)|set(alphabets))
cleaned_word_dict = dict((i, 0) for i in cleaned_word_list)

def extract_words(x):
    res = []
    subs = [x[i:j+1] for i in range(len(x)) for j in range(i,len(x))if (i - (j+1)) < -1]
    for sub in subs:
        try:
            l = cleaned_word_dict[sub]
            res.append(str(sub))
        except:
            pass
    
    return sorted(res, key = len, reverse=True)

common_words_dict = dict((i, extract_words(str(i))[:5]) for i in con_words)

Output:

{'stainlesssteel': ['stainless', 'stain', 'steel', 'tain', 'less'],
 'screwhammerwing': ['hammer', 'screw', 'ammer', 'crew', 'wham'],
 'goldplated': ['plated', 'plate', 'lated', 'gold', 'plat'],
 'bearingball': ['bearing', 'earing', 'bear', 'ring', 'ball'],
 'inchcountry': ['country', 'count', 'inch', 'try', 'in']}

Is there any other way of doing this?

Please help me in understand how to get the expected output.

Answer 1

I think the simplest way to get your expected output (although it might use a lot of memory) would be to run the extract_words() function on the first output, but remove words that are contained in other answers. This would prevent the fragmented words ('stainless'-'stain' ; 'hammer'-'ammer') and would still allow for multiple full words. I will update my answer with code once I make something that works.