从对象集合创建批次

Question

I have a collection of objects (more than 500 in count) I would like to batch it with some logic, like a batch of 50's and I should get 10 sets.(10*50=500)

I am using the logic below:

 public class CustomEngineReader :IEnumerable<List<EngineToken>>
{
    StreamReader sr;
    int _batchSize = 1;

    public CustomFileReader(List<EngineToken> tokens, int batchSize)
    {
        if (batchSize > 0)
        {
            _batchSize = batchSize;
        }



    }


    public IEnumerator<List<string>> GetEnumerator()
    {

        string input = string.Empty;

        foreach(var item in EngineTokens)
        {
            int i = 0;
            List<string> batch = new List<string>();

            while (i < _batchSize && item !=null)
            {
                batch.Add(item );
                i++;
            }

            if (batch.Count != 0)
            {
                yield return batch;
            }
        }

    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

I am using the above code as below

CustomEngineReader reader = new CustomEngineReader (this.tokencollection, 50);
foreach(List<EngineToken> items in reader)
{
       //in each iteration we get batch specified objects
       foreach(EngineToken item in items)
       {
          //Process
       }    
}

It's not working.

Answer 1

Having a while loop inside of your foreach means that you're adding the same item over and over again until you hit your batch size.

You also need to create your batch object outside of the foreach , so that you can add multiple items from the loop into it.

Personally I prefer to write this as an extension method, rather than a separate class, to follow in line with the LINQ style of programming. It also can be trivially made generic, greatly improving its usefulness. My implementation of Batch is:

public static IEnumerable<IEnumerable<T>> Batch<T>(
    this IEnumerable<T> source, int batchSize)
{
    List<T> buffer = new List<T>(batchSize);

    foreach (T item in source)
    {
        buffer.Add(item);

        if (buffer.Count >= batchSize)
        {
            yield return buffer;
            buffer = new List<T>(batchSize);
        }
    }
    if (buffer.Count > 0)
    {
        yield return buffer;
    }
}

Answer 2

You are adding same item multiple times in a loop. Try:

public IEnumerator<List<EngineToken>> GetEnumerator()
{

    string input = string.Empty;

    int i=0;
    List<string> batch = new List<string>();
    foreach(var item in EngineTokens)
    {
        batch.Add(item);
        i++;
        if(i==_batchSize)
        {
             yield return batch;
             batch = new List<string>();
             i = 0;
        }
    }
    if (batch.Count != 0)
    {
        yield return batch;
    }

}

Answer 3

Here is a simpler implementation, non-thread safe. For thread safety you will need to add locks as necessary.

public IEnumerator<List<EngineToken> GetEnumerator()
{    
        var currentBatch = EngineTokens.Take(_batchSize);
        EngineTokens = EngineTokens.Skip(_batchSize).ToList();
        return currentBatch;

}