无限并行运行多次相同的ActionBlock

Question

I am learning TPL Dataflow, I have seen it's power through some friends of mine and I am running into an issue with my implementation.

What I want/need is to send messages as fast as I can. I am doing some prototyping in LinqPad and this is what I have thus far:

// Holds all the messages for my loadMessage ActionBlock to grab its data from
var bufferBlock = new BufferBlock<string>();

// Sends message to where it needs to go as fast as it can.
var loadMessage = new ActionBlock<string>(msg => 
{
    msg.Dump();
},  
new ExecutionDataflowBlockOptions
{
    MaxDegreeOfParallelism = DataflowBlockOptions.Unbounded
});

// Links the blocks together
bufferBlock.LinkTo(loadMessage);

// Loads the Buffer
for (int i = 0; i < 10; i++)
{
    bufferBlock.Post("This is a message");
}

//Calls completion to stop threads
bufferBlock.Complete();
loadMessage.Complete(); 

The problem is that the loadMessageBlock isn't dumping the message in the example above. I have been looking for some insight with little luck. I think I am missing something fundamental for TPL. My understanding is that the BufferBlock holds information to be processed by other blocks, and the ActionBlocked (Which is linked to the BufferBlock) should grab the data off of the buffer and do what it needs to do. After the For loop that puts the information on the buffer stops completion is called to stop the threads.

In the implementation I have, I have a Parallel.For that runs the code inside my loadMessage just fine. I just can't implement TPL to do what I want and my understanding is that TPL will be faster than Parallel.For.

Am I way off here in how I think this is suppose to work? Am I mis-using TPL? I am going to continue to research for an answer, any pointers will be highly appreciated. Thanks!

Answer 1

First, a note about terminology: TPL (short for Task Parallel Library) is not the same as TPL Dataflow, that's just a subset. TPL as a whole includes things like Parallel.For() and the Task type.

Now, the problem with your code is that you're completing the loadMessage block too soon. After you call Complete() , the block won't accept any more messages, so the messages you posted to bufferBlock will never reach loadMessage .

What you need is to complete loadMessage only after bufferBlock sends all its messages to it. Which is exactly what PropagateCompletion does:

bufferBlock.LinkTo(
    loadMessage, new DataflowLinkOptions { PropagateCompletion = true });

// post your data to bufferBlock here

bufferBlock.Complete();
await loadMessage.Completion;

Also, in this specific case, the bufferBlock is not needed at all, you could have just posted the messages directly to loadMessage .

my understanding is that TPL will be faster than Parallel.For

I don't see why should be faster in general. Under normal circumstances, their performance should be comparable. So you should use the one that fits your problem better, instead of choosing one because “it's faster”. And if you really care about performance that much, write the code both ways and measure which one is better.

Answer 2

1. What I want/need is to send messages as fast as I can:

   To achieve that, you need to post/ receive data concurrently to and from the buffer block. Below is the snippet:

    var bufferBlock = new BufferBlock<string>(); // Write to and read from the message block concurrently. var post01 = Task.Run(() => { // Loads the Buffer for (int i = 0; i < 10; i++) { bufferBlock.Post(string.Format("This is a message {0}",i)); } }); var receive = Task.Run(() => { for (int i = 0; i < 10; i++) { var message = bufferBlock.Receive(); message.Dump(); } }); Task.WaitAll(post01, receive); 

   More on this you can see at MSDN link

2. my understanding is that TPL will be faster than Parallel.For.

   This not correct, because they use the same underlying structure. They belong to the same name space System.Threading.Tasks