是否有快速算法来删除字符串中重复的子串？

Question

There is a string like it

dxabcabcyyyydxycxcxz

and I want to merge it into

dxabcydxycxz

Other examples: ddxddx -> dxdx , abbab -> abab.

The rule is that :

if (adjacent and same): merge

# Such as 'abc',they are same and , so I will delete one of them .
# Although 'dx' is same as 'dx',they are nonadjacent,so I do not delete any of them
# If one character has been deleted, we don't delete any sub-string include it 

I did it in my code in python,but it's slow when did in a long string.

# original string
mystr = "dxabcabcyyyydxycxcxz"
str_len = len(mystr)
vis = [1] *str_len #Use a list to mark which char is deleted

# enumerate the size of sub-str
for i in range(1,str_len):
    # enumerate the begin of the sub-str
    for j in range(0, str_len):
        offset = 2 #the size of sub-str + 1
        current_sub_str = mystr[j:j+i]
        s_begin = j+i*(offset-1)
        s_end = j+(i*offset)
        # delete all of the same char
        while((j+(i*offset) <= str_len) and current_sub_str == mystr[s_begin:s_end]
              and 0  not in vis[s_begin:s_end] and 0  not in vis[j:j+i]):
            vis[s_begin:s_end] = [0] * (s_end - s_begin) #if I deleted it ,mark it as 0
            offset += 1
            s_begin = j + i * (offset - 1)
            s_end = j + (i * offset)

res = []
for i in range(0,str_len):
    if(vis[i]!=0): res.append(mystr[i])

print "".join(res)

Is there any faster way to solve it?

update April 29, 2017

Sorry, it seems to like a XY problem.On the other hand,it maybe not. there is the content

I was coding for a web spider and got many 'tag-path's like those

ul/li/a
ul/li/div/div/div/a/span
ul/li/div/div/div/a/span 
ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a
ul/li/ul/li/a

As you see,there are some 'tag-path' did a same way,so I wanted to collapse them to find is there any other 'tag-path's have the same structure. After collapsing, I get the 'tag-path' like this.

ul/li/a
ul/li/div/div/div/a/span
ul/li/a
ul/li/ul/li/a
ul/li/a
ul/li/ul/li/a
ul/li/a
ul/li/ul/li/a

This is only my idea and I didn't know whether it is suitable to do in this way.(After trying, I chose another way to do it.

However there is an interesting question like a ACM question.

So I simplify one 'tag-path' to a character and ask for help.Because I didn't do a fast way by myself. Actually, the question has many corner cases that I don't mind and thank all for helping me complete it.

Thanks all.

Answer 1

Behold the power of regex:

>>> import re

>>> re.sub(r"(.+?)\1+", r"\1", "dxabcabcyyyydxycxcxz")
'dxabcydxycxz'

>>> re.sub(r"(.+?)\1+", r"\1", "ddxddx")
'dxdx'

>>> re.sub(r"(.+?)\1+", r"\1", "abbab")
'abab'

This looks for a sequence of 1 or more arbitrary characters (.+?) (as a non-greedy match, so that it tries shorter sequences first), followed by 1 or more repetitions of the matched sequence \\1+ , and replaces it all with just the matched sequence \\1 .

Answer 2

This can be a start:

for i in range(len(string)):
    for j in range(i + 1, len(string)):
        while string[i:j] == string[j:j + j - i]:
            string = string[:j] + string[j + j - i:]

The result on the examples provided:

abbab  -> abab
ddxddx -> dxdx
abcabcabc -> abc
dxabcabcyyyydxycxcxz -> dxabcydxycxz

Answer 3

This is a great question/series of responses!

Here's an implementation using a generator and string slicing:

import math
def dedupe(string, step=1):
    index = 0
    prior = ''
    while index < len(string):
        letter = string[index]
        window = index + step
        comparison = string[index:window]
        if comparison != prior:
            yield letter
            prior += letter
            index += 1
        else:
            index += step
        if len(prior) > (step):
            prior = prior[1:] # remove first character


def collapse(string):
    step = 1
    while step < math.sqrt(len(string)):
        generator = dedupe(string, step=step)
        string = ''.join(generator)
        step +=1
    return string

Edit: changed the step search to use the square root of the length to improve search times:

• %timeit collapse('dxabcabcyyyydxycxcxz') 10000 loops, best of 3: 24.7 µs per loop
• %timeit collapse(randomword(100) 1000 loops, best of 3: 384 µs per loop
• %timeit collapse("a" * 100) 10000 loops, best of 3: 27.1 µs per loop
• %timeit collapse(randomword(50) * 2) 1000 loops, best of 3: 382 µs per loop

Answer 4

One line:

def remove_repeats(iterable):
    return [e for (i, e) in enumerate(iterable) if i == 0 or e != iterable[i - 1]]

It works with any iterable data, returns list.

>>> print remove_repeats("aaabbc")
['a', 'b', 'c']

>>> s = '''
... ul/li/a
... ul/li/div/div/div/a/span
... ul/li/div/div/div/a/span
... ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... ul/li/ul/li/a
... '''

>>> print remove_repeats(s.split())
['ul/li/a', 'ul/li/div/div/div/a/span', 'ul/li/a', 'ul/li/ul/li/a', 'ul/li/a', '
ul/li/ul/li/a', 'ul/li/a', 'ul/li/ul/li/a']

Join if you need a string:

>>> print "".join(remove_repeats('111222333'))
123

>>> print "\n".join(remove_repeats(s.split()))
ul/li/a
ul/li/div/div/div/a/span
ul/li/a
ul/li/ul/li/a
ul/li/a
ul/li/ul/li/a
ul/li/a
ul/li/ul/li/a

Answer 5

from collections import OrderedDict
mystr = "dxabcabcyyyydxycxcxz"
index=0;indexs = [];count = OrderedDict()
while count!=None:
    count = {}
    for index in range(0,len(mystr)):
        flag = True
        for index1 in range(0,index+1)[::-1]:
            if(mystr.startswith(mystr[index1:index+1], index+1)):
                if count.get(str(index1),0)<(index+1-index1):
                    count.update({str(index1) : index+1-index1})
    for key in count:
        mystr = mystr[:int(key)]+mystr[int(key)+count[key]:]
    if count=={}:
        count=None
print "Answer:", mystr

Answer 6

One linear approach

import itertools
_str = 'dxabcabcyyyydxycxcxz'
print ''.join(ch for ch, _ in itertools.groupby(_str))

result:

dxabcabcyyyydxycxcxz -> dxabcabcydxycxcxz