高级多重搜索替换

Question

Problem: I want to batch replace patterns in a file in an advanced way, so I cannot use any standard search and replace tools:

Let's assume there is file 1:

B
B
A
  B
B
B
A
  B
B
A
  B

And I want to replace B by something else. But only each B, which comes after a A.

Here is File 2, which holds the "rules", how to search and replace:

A;B;C1
A;B;C2
A;B;C3

The ";" should be the divider. Can be anything else. Script should search for and A. Then continue to search for B. And replace that B by C1. Afterwards continue to the next occurence of A. Search for the next B and replace this B by C2. And so on. When the script has replaced B by C3, it should stop, because there is no further rule.

Final file should look like:

B
B
A
  C1
B
B
A
  C2
B
A
  C3

I want to use python for it, but it is not mandatory, if there is an easier way.

Answer 1

You could implement something similar using regular expressions. re.finditer returns starting/ending position of match and re.sub accepts parameter how many substitutions should be made. You can start from this:

import re

data = '''B
B
A
  B
B
B
A
  B
B
A
  B'''

rules = [
    (r'A.*?(B)', r'C1'),
    (r'A.*?(B)', r'C2'),
    (r'A.*?(B)', r'C3'),
]

startpos = 0
while rules:
    rule = rules.pop(0)
    for g in re.finditer(rule[0], data[startpos:], flags=re.DOTALL):
        data = data[:startpos + g.start(1)] + re.sub(g.group(1), rule[1], data[startpos + g.start(1):], count=1)
        startpos += g.start(1)
        break

print(data)

Prints:

B
B
A
  C1
B
B
A
  C2
B
A
  C3

Answer 2

I started writing a regex based solution, but @Andrej got there first! So I present you a more "naive" approach that does not use regex.

#!/usr/bin/env python3
import sys


def read_rules(fpath="/tmp/test.rules", sep=";"):
    rules = []
    with open(fpath) as f:
        for line in f:
            rules.append(line.strip().split(sep))

    return rules


def parse_data(rules, fpath="/tmp/test.data"):
    cur_rule = rules[0]
    rule_idx = 0
    data = []
    state = None

    with open(fpath) as f:
        for line in f:
            line = line.strip('\n')
            if not cur_rule:
                data.append(line)
                continue

            # We match start
            if cur_rule[0] in line and not state:
                # End matches in the same line and start < end
                # This case is not in your data
                if (
                    cur_rule[1] in line
                    and line.index(cur_rule[0]) < line.index(cur_rule[1])
                ):
                    new_line = line.replace(cur_rule[1], cur_rule[2], 1)
                    data.append(new_line)
                    rule_idx += 1

                    # We reached the end of rules
                    if len(rules) == rule_idx:
                        cur_rule = None
                    else:
                        cur_rule = rules[rule_idx]
                else:
                    # Set state to looking for end
                    state = 1
                    data.append(line)

                continue

            # Now, if here we are looking for end...
            if state == 1:
                # Nope... not found... move on
                if cur_rule[1] not in line:
                    data.append(line)
                    continue

                # replace
                data.append(
                    line.replace(cur_rule[1], cur_rule[2], 1)
                )

                # Reset state
                state = None

                rule_idx += 1

                # We reached the end of rules
                if len(rules) == rule_idx:
                    cur_rule = None
                else:
                    cur_rule = rules[rule_idx]
                continue

            # Here, no line matched
            data.append(line)


    return data


def main():
    rules = read_rules()
    print(rules)
    data = parse_data(rules)
    print("\n".join(data))


if __name__ == "__main__":
    sys.exit(main())

Explanation:

• This is a line-by-line algorithm which makes it efficient for large datasets
• It is "state" based: We either look for "start" (first character) or "end" (second character to match)
• If start is found:
  • If we have end in the same line, perform replacement and advance to the next rule
  • If we do not have end in the same line, change state and move to the next line
• If we are in state=1 (looking for "end") and we find it in the current line, perform replacement and move to the next rule
• At any point we advance rule, if we reached the end of rules, set cur_rule to None. All lines past that point are just copied from the input to the output without any processing

Pros:

• This should be faster for huge input. Output can also be optimized to be "on-the-fly" and not stored in memory
• Easier to follow I think

Cons:

• It does not handle all cases, that's why I called it "naive". One example is if you have 2 matches in the same line, or if you match "end" and "start" in the same line (in this order - end first). It can be adjusted for such cases if necessary but it might get complex and a regex solution becomes more attractive

Output (note I added one extra match to check that it stops when rules finish):

B
B
A
  C1
B
B
A
  C2
B
A
  C3
A
  B