[英]How can a TPL Dataflow block downstream get data produced by a source?
I'm processing images using TPL Dataflow. 我正在使用TPL Dataflow处理图像。 I receive a processing request, read an image from a stream, apply several transformations, then write the resulting image to another stream: 我收到处理请求,从流中读取图像,应用几个转换,然后将生成的图像写入另一个流:
Request -> Stream -> Image -> Image ... -> Stream
For that I use the blocks: 为此,我使用块:
BufferBlock<Request>
TransformBlock<Request,Stream>
TransformBlock<Stream,Image>
TransformBlock<Image,Image>
TransformBlock<Image,Image>
...
writerBlock = new ActionBlock<Image>
The problem is the initial Request
is what contains some data necessary to create the resulting Stream
along with some additional info I need at that point. 问题是初始Request
是包含创建结果Stream
所需的一些数据以及此时我需要的一些其他信息。 Do I have to pass the original Request
(or some other context object) down the line to the writerBlock
across all the other blocks like this: 我是否必须将原始Request
(或其他一些上下文对象) writerBlock
到所有其他块的writerBlock
,如下所示:
TransformBlock<Request,Tuple<Request,Stream>>
TransformBlock<Tuple<Request,Stream>,Tuple<Request,Image>>
TransformBlock<Tuple<Request,Image>,Tuple<Request,Image>>
...
(which is ugly), or is there a way to link the first block to the last one (or, generalizing, to the ones that need the additional data)? (这很难看),还是有办法将第一个块链接到最后一个块(或者,推广到需要附加数据的块)?
Yes, you pretty much need to do what you described, passing the additional data from every block to the next one. 是的,你几乎需要做你所描述的,将每个块中的附加数据传递给下一个块。
But using a couple of helper methods, you can make this much simpler: 但是使用几个辅助方法,你可以使这更简单:
public static IPropagatorBlock<TInput, Tuple<TOutput, TInput>>
CreateExtendedSource<TInput, TOutput>(Func<TInput, TOutput> transform)
{
return new TransformBlock<TInput, Tuple<TOutput, TInput>>(
input => Tuple.Create(transform(input), input));
}
public static IPropagatorBlock<Tuple<TInput, TExtension>, Tuple<TOutput, TExtension>>
CreateExtendedTransform<TInput, TOutput, TExtension>(Func<TInput, TOutput> transform)
{
return new TransformBlock<Tuple<TInput, TExtension>, Tuple<TOutput, TExtension>>(
tuple => Tuple.Create(transform(tuple.Item1), tuple.Item2));
}
The signatures look daunting, but they are actually not that bad. 签名看起来令人生畏,但实际上并没有那么糟糕。
Also, you might want to add overloads that pass options to the created block, or overloads that take async delegates. 此外,您可能希望添加将选项传递给创建的块的重载,或者重载以获取异步委托的重载。
For example, if you wanted to perform some operations on a number using separate blocks, while passing the original number along the way, you could do something like: 例如,如果您想使用单独的块对数字执行某些操作,同时沿途传递原始数字,您可以执行以下操作:
var source = new BufferBlock<int>();
var divided = CreateExtendedSource<int, double>(i => i / 2.0);
var formatted = CreateExtendedTransform<double, string, int>(d => d.ToString("0.0"));
var writer = new ActionBlock<Tuple<string, int>>(tuple => Console.WriteLine(tuple));
source.LinkTo(divided);
divided.LinkTo(formatted);
formatted.LinkTo(writer);
for (int i = 0; i < 10; i++)
source.Post(i);
As you can see, your lambdas (except for the last one) deal only with the “current” value ( int
, double
or string
, depending on the stage of the pipeline), the “original” value (always int
) is passed automatically. 如您所见,您的lambda(除了最后一个)只处理“当前”值( int
, double
或string
,具体取决于管道的阶段),“原始”值(总是int
)自动传递。 At any moment, you can use block created using the normal constructor to access both values (like the final ActionBlock
in the example). 在任何时候,您都可以使用使用普通构造函数创建的块来访问这两个值(例如示例中的最终ActionBlock
)。
(That BufferBlock
isn't actually necessary, but I added it to more closely match your design.) (那个BufferBlock
实际上并不是必需的,但我添加它以更BufferBlock
您的设计。)
I may be going over my head since I am only starting to play with TPL Dataflow. 因为我刚刚开始使用TPL Dataflow,所以我可能会过头了。 But I believe you can accomplish that using a BroadcastBlock
as an intermediary between your source and your first target. 但我相信你可以使用BroadcastBlock
作为源和你的第一个目标之间的媒介来实现这一点。
BroadcastBlock
can offer the message to many targets, so you use it to offer to your target, and also to a JoinBlock
, at the end that will merge the result with the original message. BroadcastBlock
可以提供消息给许多目标,所以你用它来提供给您的目标, 同时也为JoinBlock
,在将结果与原始邮件合并结束。
source -> Broadcast ->-----------------------------------------> JoinBlock <source, result>
-> Transformation1 -> Transformation 'n' ->
For example: 例如:
var source = new BufferBlock<int>();
var transformation = new TransformBlock<int, int>(i => i * 100);
var broadCast = new BroadcastBlock<int>(null);
source.LinkTo(broadCast);
broadCast.LinkTo(transformation);
var jb = new JoinBlock<int, int>();
broadCast.LinkTo(jb.Target1);
transformation.LinkTo(jb.Target2);
jb.LinkTo(new ActionBlock<Tuple<int, int>>(
c => Console.WriteLine("Source:{0}, Target Result: {1}", c.Item1, c.Item2)));
source.Post(1);
source.Post(2);
source.Complete();
yields... 收益率...
Source:1, Target Result: 100 资料来源:1,目标结果:100
Source:2, Target Result: 200 资料来源:2,目标结果:200
I am just not too sure about how it would behave in an asynchronous environment. 我不太确定它在异步环境中的表现。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.