在C＃中使用Regex匹配两个字符串

Question

I have two files, each contains a number on each line. Let's name them File A and File B. Each file contains roughly 2 million lines, and I'm trying to match those numbers using Regex.

The tricky part is that, each number in B can be a number in A + "00" or can simply be equal to A.

B = A OR B = A + "00"

This is how I'm identifying my Regex:

Regex re = new Regex("[0-9](0)*");

My code stores the smaller file into a HashSet, then I build my Regex and do the matching:

HashSet<string> matchVal = new HashSet<string>();

            foreach (var mdeSubid in File.ReadLines("FileA.txt"))
            {
                matchVal.Add(mdeSubid);
            }

            Regex re = new Regex("[0-9](0)*");
            StreamWriter sw = new StreamWriter("output.txt");

            foreach (var rx in File.ReadLines("FileB.txt"))
            {
                var matches = re.Matches(rx);
                foreach (Match m in matches)
                {
                    if (matchVal.Contains(m))
                    {
                        sw.WriteLine(m);   
                    }
                }
            }
            Console.WriteLine("DONE!");
            Console.ReadLine();

When I'm running my script I'm not getting any match even though I'm sure 100% that there are bunch of them.

Real Example

File A:

846535465

846536589

8465631

File B:

84653546500

846536589

846563100

846563102

In the example, B should match A in all cases except the last one,

846535465 == 84653546500
846536589 == 846536589 
8465631 == 846563100
8465631 != 846563101

Answer 1

The issue is your regex. What you're asking for is a single number followed by as many zeroes as possible.

[0-9] - any number

(0) - a matching group containing the number zero as many times as possible *

I believe what you really want is to match only the first part of the sequence, excluding the trailing zeroes, against a predefined list of numbers. I also believe you should make sure this is at the start of the line only, just in case you get a match elsewhere on the line causing a false positive.

Therefore, your regex should be: ^(\\d+)(00)?

^ - asset position at the start of the string

(\\d+) - first capture group, match any number one or more times

(00)? - second capture group, match '00' zero or one times

Finally, you need to alter your code so that rather than checking simply for a match, you are checking that Group(1) is a match (the sequence minus the trailing zeroes)

EDIT

So I tried this regex out on www.regex101.com and it's not quite right. I'm asking for the first capture group to match as many digits as possible, which by definition already encompasses the second capture group. I'd suggest using the first part of the regex ^\\d+ and doing the trailing zero check in code. Although that'll potentially double the time taken checking all those matches, I'm struggling to think of a more efficient method right now...

Answer 2

This is far from ideal, but I gave it a shot. It requires reading all the lines of FileA first which isn't ideal if there are millions of lines, but I can't figure out if there is a way around that, that wouldn't slow down the process tremendously.

Anyway, here's the commented code, which simply prints the matches to the console.

//Read all the lines from FileA
var FileAContents = File.ReadAllLines(PathFileA);

//Read FileB one line at a time, and compare with FileA to search for matches.
string line;
StreamReader file = new StreamReader(PathFileB);
while ((line = file.ReadLine()) != null)
{
    //Start by finding all the possible matches, where the line from FileB starts with something that can be found in FileA.
    List<string> possibleMatches = FileAContents.Where(m => line.StartsWith(m)).ToList();
    foreach(string pm in possibleMatches)
    {
        //If the lines are completely equal, you've found a match.
        if(pm.Equals(line))
        {
            Console.WriteLine("Match found, FileA: {0}, FileB: {1}", pm, line);
        }
        else if(line.EndsWith("00"))
        {
            //Remove the "00", then check if you've found a match.
            string tempLine = line.Substring(0, line.Length - 2);
            if(pm.Equals(tempLine))
            {
                Console.WriteLine("Match found, FileA: {0}, FileB: {1}", pm, line);
            }
        } 
    }
}

Answer 3

The problem you have here is that you're matching against a very large number of values. It is possible that you may run low on memory. Here are two possible choices of ways to go.

The first is the simplest, but may use too much memory:

var words = new HashSet<string>(
    File
        .ReadLines("FileA.txt")
        .SelectMany(x => new[] { x, x + "00" }));

var query = from word in File.ReadLines("FileB.txt")
            where words.Contains(word)
            select word;

File.WriteAllLines("output.txt", query);

The second uses a structure called a "trie" which can be far more efficient with the storage of the numbers from the first file. It depends on how random the numbers are.

void Main()
{
    var words = File.ReadLines("FileA.txt").SelectMany(x => new[] { x, x + "00" });

    var trie = new Trie(words);

    var query = from word in File.ReadLines("FileB.txt")
                where trie.Contains(word)
                select word;

    File.WriteAllLines("output.txt", query);
}

public class Trie : Dictionary<char, Trie>
{
    public Trie() : base(0) { }
    public Trie(IEnumerable<string> words) : base(0)
    {
        foreach (var word in words)
        {
            this.Add(word);
        }
    }

    public void Add(string word)
    {
        if (String.IsNullOrEmpty(word))
        {
            this[char.MinValue] = null;
        }
        else
        {
            Trie t = null;
            if (this.ContainsKey(word[0]))
            {
                t = this[word[0]];
            }
            else
            {
                t = new Trie();
                this[word[0]] = t;
            }
            t.Add(word.Substring(1));
        }
    }

    public bool Contains(string prefix)
    {
        return this.ContainsInternal(prefix + char.MinValue);
    }

    private bool ContainsInternal(string prefix)
    {
        if (!string.IsNullOrEmpty(prefix) && this.ContainsKey(prefix[0]))
        {
            return prefix.Length == 1 || this[prefix[0]].ContainsInternal(prefix.Substring(1));
        }
        return false;
    }
}

Both versions work in my tests.

Answer 4

Please try the below code (it's using System.Linq ):

var fileA = new string[] {"846535465", "846536589", "8465631"};
var fileB = new string[] {"84653546500", "846536589", "846563100", "846563102"};

foreach (var bText in fileB){
    var aText = fileA.FirstOrDefault(a => Regex.Match(bText, ("^(" + a + ")(00)?$")).Success);
    if(!String.IsNullOrEmpty(aText)){
        Console.WriteLine("B: " + bText + " - A: " + aText);
    }       
}

In this case, you check if the line in file B matches any value from file A, either with or without 00 .

In the above example, fileA and fileB arrays are your file contents, I just simplified them to be arrays, so you can test it quickly.

EDIT: This example may not be the best when it comes to performance, but you can get the idea from here, to build the Regex dynamically.