TPL數據流，從BroadcastBlock到BatchBlocks

Question

我在將BroadcastBlock(s)連接到BatchBlocks 。 場景是源是BroadcastBlocks ，接收者是BatchBlocks 。

在下面的簡化代碼中，僅執行一個補充動作塊。 我什至將每個BatchBlock為1來說明問題。

將Greedy設置為“ true”將使2個ActionBlocks執行，但這不是我想要的，因為即使尚未完成，它也會導致BatchBlock繼續進行。 有任何想法嗎？

class Program
{
    static void Main(string[] args)
    {
        // My possible sources are BroadcastBlocks. Could be more
        var source1 = new BroadcastBlock<int>(z => z);

        // batch 1
        // can be many potential sources, one for now
        // I want all sources to arrive first before proceeding
        var batch1 = new BatchBlock<int>(1, new GroupingDataflowBlockOptions() { Greedy = false }); 
        var batch1Action = new ActionBlock<int[]>(arr =>
        {
            // this does not run sometimes
            Console.WriteLine("Received from batch 1 block!");
            foreach (var item in arr)
            {
                Console.WriteLine("Received {0}", item);
            }
        });

        batch1.LinkTo(batch1Action, new DataflowLinkOptions() { PropagateCompletion = true });

        // batch 2
        // can be many potential sources, one for now
        // I want all sources to arrive first before proceeding
        var batch2 = new BatchBlock<int>(1, new GroupingDataflowBlockOptions() { Greedy = false  });
        var batch2Action = new ActionBlock<int[]>(arr =>
        {
            // this does not run sometimes
            Console.WriteLine("Received from batch 2 block!");
            foreach (var item in arr)
            {
                Console.WriteLine("Received {0}", item);
            }
        });
        batch2.LinkTo(batch2Action, new DataflowLinkOptions() { PropagateCompletion = true });

        // connect source(s)
        source1.LinkTo(batch1, new DataflowLinkOptions() { PropagateCompletion = true });
        source1.LinkTo(batch2, new DataflowLinkOptions() { PropagateCompletion = true });

        // fire
        source1.SendAsync(3);

        Task.WaitAll(new Task[] { batch1Action.Completion, batch2Action.Completion }); ;

        Console.ReadLine();
    }
}

Answer 1

您完全沒有理解Greedy標志的作用。 如果它等於true ，則即使沒有足夠的數據量來收集到批處理中，您的批處理塊也會收集數據。 通過設置Greedy = false ，對TPL Dataflow說： I will post to batch blocks, not you ，因此批處理塊可能會或可能不會決定從廣播塊獲取消息。

而且，您確實可以通過調用Task.WaitAll(new Task[] { batch1Action.Completion, batch2Action.Completion });阻塞線程Task.WaitAll(new Task[] { batch1Action.Completion, batch2Action.Completion }); ，因為它會阻止您的每個主線程和線程Completion任務。 這可能導致死鎖，因為線程在能夠通過管道發布消息之前被阻塞。 另外，您不會調用source1.Complete() ，因此此WaitAll調用將永遠不會返回 。

您真正需要的是將Greedy設置為true （默認設置），將批處理大小設置為所需的值（例如2 ），調用Complete()方法，並且不要在管道中使用線程阻塞方法 。 這樣，您的批處理塊將從廣播中獲取所有數據，但是其他塊在獲得批處理的所有數據之前將不會獲得任何數據：

var source1 = new BroadcastBlock<int>(z => z);
var options = new DataflowLinkOptions { PropagateCompletion = true };

// this block wouldn't execute, as it doesn't get the data with greedy execution
var batch1 = new BatchBlock<int>(2, new GroupingDataflowBlockOptions { Greedy = false });
var batch1Action = new ActionBlock<int[]>(arr =>
{
    Console.WriteLine("Received from batch 1 block!");
    foreach (var item in arr)
    {
        Console.WriteLine("Received {0}", item);
    }

});
batch1.LinkTo(batch1Action, options);

// this batch is freedy, so it will execute always
var batch2 = new BatchBlock<int>(2);
var batch2Action = new ActionBlock<int[]>(arr =>
{
    Console.WriteLine("Received from batch 2 block!");
    foreach (var item in arr)
    {
        Console.WriteLine("Received {0}", item);
    }
});
batch2.LinkTo(batch2Action, options);

// connect source(s)
source1.LinkTo(batch1, options);
source1.LinkTo(batch2, options);

// fire
source1.SendAsync(3);
// simulate some over work
Thread.Sleep(3000);
// complete batch, now the ActionBlock2 will execute
source1.SendAsync(3);

// if you need to wait for completion, call this method
source1.Complete();
// note that WhenAll isn't blocking task
var allTasks = Task.WhenAll(batch1Action.Completion, batch2Action.Completion);
// non-blocking wait
await allTasks;
// blocking wait
allTasks.Wait();

Console.ReadLine();