有没有更快的方法来查找列表中的重复模式？

Question

Python novice here. I have a problem in which I want to find all of the repeated patterns within a list (it is, specifically in my case, a list of integers). So, for example, given the list [2,1,4,3,12,8,3,3,4,16,2,9,9,8,3,3,4,1,4,3,4,8,3,3,4] and a min pattern length of 3 the algorithm would find that [8,3,3,4] occurs thrice and [1,4,3] occurs twice (nice also to have the index of all occurrences).

I have some code that works, if a little clumsily, but the lists that I want eventually to use the code on may be very large. I'm not really sure how to work out the operational complexity of my code, but I know that it definitely gets very slow when I am using large lists.

My question is, are there any better algorithms anyone knows for doing this, and/or am I doing this in a very inefficient way? Thanks for any help you can give me.

Here is the code:

# Searches list to determine how many times small list is included in big list
def contains(small, big):
    counter = 0
    # initiating list of indexes. N.B. indexlist gives LAST index of sequence, not first
    indexlist = []
    for i in range(len(big)-len(small)+1):
        for j in range(len(small)):
            if big[i+j] != small[j]:
                break
        else:
            counter += 1
            indexlist.append(i+j)
    if counter > 0:
        return counter, indexlist
    return False

def findrepeats(sequence, n_letters):
    fulldict = {}
    # Iterating through all the short-sequences of n letters in the list
    for i in range(0, len(sequence) - n_letters):
        shortliststr = ""
        shortlist = sequence[i:i + n_letters]
        for number in shortlist:
            shortliststr = shortliststr + "." + str(number)
        # If short-sequence is found in full sequence more than once (i.e. itself), add to dict
        if contains(shortlist, sequence)[0] > 1 and len(shortlist) == n_letters:
            fulldict[shortliststr] = contains(shortlist, sequence)
    return fulldict

def findallrepeats(sequence, min_letters, max_letters):
    fulldict = {}
    # Iterating through all possible n_letters in findrepeats() between given range
    for i in range(min_letters, max_letters):
        newdict = findrepeats(sequence, i)
        fulldict.update(newdict)
    return fulldict

Answer 1

With overlapping

You can use a sliding window of size n = 3 which iterates your sequence and count the number of occurence of this window.

Using more_itertools .

For instance:

import collections
import more_itertools

sequence = [
    2, 1, 4, 3, 12, 8, 3, 3, 4, 16, 2, 9, 9,
    8, 3, 3, 4, 1, 4, 3, 4, 8, 3, 3, 4,
]
size = 3
windows = [
    tuple(window)
    for window in more_itertools.windowed(sequence, size)
]
counter = collections.Counter(windows)
for window, count in counter.items():
    if count > 1:
        print(window, count)

You get:

(1, 4, 3) 2
(8, 3, 3) 3
(3, 3, 4) 3