Create multiple lists of unique entries from master list in c#

Question

I need to process an outbound SMS queue and create batches of messages. The queued list might contain multiple messages to the same person. Batches do not allow this, so I need to run through the main outbound queue and create as many batches as necessary to ensure they contain unique entries. Example:

Outbound queue = (1,2,3,3,4,5,6,7,7,7,8,8,8,8,9)

results in...

 batch 1 = (1,2,3,4,5,6,7,8,9)
    batch 2 = (3,7,8)
    batch 3 = (7,8)
batch 4 = (8)

I can easily check for duplicates but I'm looking for a slick way to generate the additional batches.

Thanks!

Answer 1

Have a look at this approach using Enumerable.ToLookup and other LINQ methods:

var queues = new int[] { 1, 2, 3, 3, 4, 5, 6, 7, 7, 8, 8, 8, 8, 9 };
var lookup = queues.ToLookup(i => i);
int maxCount = lookup.Max(g => g.Count());
List<List<int>> allbatches = Enumerable.Range(1, maxCount)
    .Select(count => lookup.Where(x => x.Count() >= count).Select(x => x.Key).ToList())
    .ToList();

Result is a list which contains four other List<int> :

foreach (List<int> list in allbatches)
    Console.WriteLine(string.Join(",", list));

1, 2, 3, 4, 5, 6, 7, 8, 9
3, 7, 8
8
8

Answer 2

Depending on the specific data structures used, the Linq GroupBy extension method could be used (provided that the queue implements IEnumerable<T> for some type T ) for grouping by the same user; afterwards, the groups can be iterated separately.

Answer 3

A naive approach would be to walk over the input, creating and filling the batches as you go:

private static List<List<int>> CreateUniqueBatches(List<int> source)
{
    var batches = new List<List<int>>();

    int currentBatch = 0;

    foreach (var i in source)
    {
        // Find the index for the batch that can contain the number `i`
        while (currentBatch < batches.Count && batches[currentBatch].Contains(i))
        {
            currentBatch++;
        }

        if (currentBatch == batches.Count)
        {
            batches.Add(new List<int>());
        }

        batches[currentBatch].Add(i);
        currentBatch = 0;
    }

    return batches;
}

Output:

1, 2, 3, 4, 5, 6, 7, 8, 9
3, 7, 8
8
8

I'm sure this can be shortened or written in a functional way. I've tried using GroupBy, Distinct and Except, but couldn't figure it out that quickly.