Replacing strings in a string array efficiently

Question

I have a string array full of guids. I am trying to replace certain guids with different guids. My approach is below;

var newArray = this.to.Select(s => s.Replace("e77f75b7-2373-dc11-8f13-0019bb2ca0a0", "1fe8f3f6-fe17-e811-80d8-00155d5ce473")
    .Replace("fbd0c892-2373-dc11-8f13-0019bb2ca0a0", "1fe8f3f6-fe17-e811-80d8-00155d5ce473")
    .Replace("76cd4297-1e31-dc11-95d8-0019bb2ca0a0", "eb892fb0-fe17-e811-80d8-00155d5ce473")
    .Replace("cd42bb68-2073-dc11-8f13-0019bb2ca0a0", "dc6077e2-fe17-e811-80d8-00155d5ce473")
    .Replace("96b97150-cd45-e111-a3d5-00155d10010f", "1fe8f3f6-fe17-e811-80d8-00155d5ce473")
    ).ToArray();

I have a few fields I am doing this for and it is leading to an OutOfMemoryException. Is it because the Replace() method is creating a new array every time? Is there a more efficient way to do this with an array of strings? This method is running for tens of thousands of records and so I think this is the issue. When I comment these lines out then i do not get the exception.

EDIT: The data in the 'to' variable is a short string in each case, but this is run for thousands of records. So 'to' might look like this for one record;

"systemuser|76cd4297-1e31-dc11-95d8-0019bb2ca0a0;contact|96b97150-cd45-e111-a3d5-00155d10010f"

It might have any of the guids I want to replace in it, so even though it might only have one guid in for that record, I need to run the full set of replaces() just in case it has any of them in it.

Any pointers would be great! Thanks.

Answer 1

I would use a replacement dictionary - its easier to maintain and easier to understand (I think) so its easier all the way:

Boilerplate and create demo data / replace dict:

using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;

internal class Program
{
    static void Main(string[] args)
    {
        // c#7 inline func
        string[] CreateDemoData(Dictionary<string, string> replDict)
        {
            // c#7 inline func
            string FilText(string s) => $"Some text| that also incudes; {s} and more.";

            return Enumerable
                .Range(1, 5)
                .Select(i => FilText(Guid.NewGuid().ToString()))
                .Concat(replDict.Keys.Select(k => FilText(k)))
                .OrderBy(t => Guid.NewGuid().GetHashCode())
                .ToArray();
        }

        // replacement dict
        var d = new Dictionary<string, string>
        {
            ["e77f75b7-2373-dc11-8f13-0019bb2ca0a0"] = "e77f75b7-replaced",
            ["fbd0c892-2373-dc11-8f13-0019bb2ca0a0"] = "fbd0c892-replaced",
            ["76cd4297-1e31-dc11-95d8-0019bb2ca0a0"] = "76cd4297-replaced",
            ["cd42bb68-2073-dc11-8f13-0019bb2ca0a0"] = "cd42bb68-replaced",
            ["96b97150-cd45-e111-a3d5-00155d10010f"] = "96b97150-replaced",
        };

        var arr = CreateDemoData(d);

Code that creates the actual replaced array:

        // c#7 inline func
        string Replace(string a, Dictionary<string, string> dic)
        {
            foreach (var key in dic.Keys.Where(k => a.Contains(k)))
                a = a.Replace(key, dic[key]);

            return a;
        }

        // select value from dict in key in dict else leave unmodified            
        var b = arr.Select(a => Replace(a, d));
        // if you have really that much data (20k guids of ~50byte length
        // is not really much imho) you can use the same approach for in
        // place replacement - just foreach over your array.

Output code:

        Console.WriteLine("\nBefore:");
        foreach (var s in arr)
            Console.WriteLine(s);

        Console.WriteLine("\nAfter:");
        foreach (var s in b)
            Console.WriteLine(s);

        Console.ReadLine(); 
    }
}

Output:

Before:
Some text| that also incudes; a5ceefd8-1388-47cd-b69e-55b6ddbbc133 and more.
Some text| that also incudes; 76cd4297-1e31-dc11-95d8-0019bb2ca0a0 and more.
Some text| that also incudes; 3311a8c5-015e-4260-af80-86b20b277234 and more.
Some text| that also incudes; ed10c79c-dad6-4c88-865c-4d7624945d66 and more.
Some text| that also incudes; 96b97150-cd45-e111-a3d5-00155d10010f and more.
Some text| that also incudes; 0226d9b1-c5f0-41fb-9294-bc9297e8afd9 and more.
Some text| that also incudes; e77f75b7-2373-dc11-8f13-0019bb2ca0a0 and more.
Some text| that also incudes; a04d1e34-e7bc-4bbc-ae0e-12ec846a353c and more.
Some text| that also incudes; cd42bb68-2073-dc11-8f13-0019bb2ca0a0 and more.
Some text| that also incudes; fbd0c892-2373-dc11-8f13-0019bb2ca0a0 and more.

Output:

After:
Some text| that also incudes; a5ceefd8-1388-47cd-b69e-55b6ddbbc133 and more.
Some text| that also incudes; 76cd4297-replaced and more.
Some text| that also incudes; 3311a8c5-015e-4260-af80-86b20b277234 and more.
Some text| that also incudes; ed10c79c-dad6-4c88-865c-4d7624945d66 and more.
Some text| that also incudes; 96b97150-replaced and more.
Some text| that also incudes; 0226d9b1-c5f0-41fb-9294-bc9297e8afd9 and more.
Some text| that also incudes; e77f75b7-replaced and more.
Some text| that also incudes; a04d1e34-e7bc-4bbc-ae0e-12ec846a353c and more.
Some text| that also incudes; cd42bb68-replaced and more.
Some text| that also incudes; fbd0c892-replaced and more.

Answer 2

I'd do this in a single sweep by using a regex to extract fields, then a dictionary of replacements to apply the change, then to reconstitute the string:

IDictionary<string, string> replacements = new Dictionary<string, string>
{
    {"76cd4297-1e31-dc11-95d8-0019bb2ca0a0","something else"},
    //etc
};
var newData = data
    //.AsParallel() //for speed
    .Select(d => Regex.Match(d, @"^(?<f1>[^\|]*)\|(?<f2>[^;]*);(?<f3>[^\|]*)\|(?<f4>.*)$"))
    .Where(m => m.Success)
    .Select(m => new
    {
        field1 = m.Groups["f1"].Value,
        field2 = m.Groups["f2"].Value,
        field3 = m.Groups["f3"].Value,
        field4 = m.Groups["f4"].Value
    })
    .Select(x => new
    {
        x.field1,
        field2 = replacements.TryGetValue(x.field2, out string r2) ? r2 : x.field2,
        x.field3,
        field4 = replacements.TryGetValue(x.field4, out string r4) ? r4 : x.field4
    })
    .Select(x => $"{x.field1}|{x.field2};{x.field3}|{x.field4}")
    .ToArray();

Answer 3

Have you tested with StringBuilder?

StringBuilder sb = new StringBuilder(string.Join(",", this.to));

      string tempStr = sb
            .Replace("e77f75b7-2373-dc11-8f13-0019bb2ca0a0", "1fe8f3f6-fe17-e811-80d8-00155d5ce473")
            .Replace("fbd0c892-2373-dc11-8f13-0019bb2ca0a0", "1fe8f3f6-fe17-e811-80d8-00155d5ce473")
            .Replace("76cd4297-1e31-dc11-95d8-0019bb2ca0a0", "eb892fb0-fe17-e811-80d8-00155d5ce473")
            .Replace("cd42bb68-2073-dc11-8f13-0019bb2ca0a0", "dc6077e2-fe17-e811-80d8-00155d5ce473")
            .Replace("96b97150-cd45-e111-a3d5-00155d10010f", "1fe8f3f6-fe17-e811-80d8-00155d5ce473")
            .ToString();

      var newArray = tempStr.Split(',');