简体   繁体   English

为什么我的 TPL 数据流操作块没有并行执行?

[英]Why is my TPL Dataflow Actionblock not executing in parallell?

I am trying to execute some web requests in parallell but for some reason it all runs sync one after the other and I cannot figure out why我正在尝试并行执行一些 web 请求,但由于某种原因,它一个接一个地运行同步,我不知道为什么

The code is fairly simple as follows代码相当简单,如下

 ConcurrentBag<string> documentsToImportInBatch = new ConcurrentBag<string>();

            var actionBlocHapiTest = new ActionBlock<Tuple<string, string>>(
                async request =>
            {
                var uriArticleTest = $"https://xxxx.com/omni-product"
                    + $"/api/v1/brands/0/articles/{request.Item1}?"
                    + $"channelIds=1&season={request.Item2}";
                var response = await staticHttpClient.GetAsync(uriArticleTest);
                var jsonString = await response.Content.ReadAsStringAsync();

                Article article = new Article
                {
                    Brand = brand,
                    UpdatedAt = DateTime.Now.ToString("s"),
                    Source = source,
                    ItemLevel = Enums.ItemLevel.Article,
                    DataType = Enums.DataType.DetailedItemInformation,
                    DetailedData = JObject.Parse(jsonString)
                };
                documentsToImportInBatch.Add(
                    Newtonsoft.Json.JsonConvert.SerializeObject(article));

            }, new ExecutionDataflowBlockOptions
            {
                MaxDegreeOfParallelism = 10
            });

            foreach (var itemTuple in articleAndSeasonList)
            {
                actionBlocHapiTest.Post(new Tuple<string, string>(
                    itemTuple.Item1, itemTuple.Item2));
            }
            actionBlocHapiTest.Complete();
            await actionBlocHapiTest.Completion;

I can see from performance monitoring that it only executing 1 request or so per second where I expected it to be a lot more.我可以从性能监控中看到它每秒只执行 1 个左右的请求,而我预计它会更多。

Why is it not filling the block and emptying it in parallell with 10 parallell requests?为什么它没有填充块并用 10 个并行请求并行清空它?

figured out the issue, before my action block I had a web request paging through creating the list that the action block used it would submit one by one to the action block stead of building the list and then submit all to the action block so in fact the block itself did process everything it had it's just that it only got one entry at the time, once I fixed that and sent the full list to the block it worked as expected and processed in parallel as expected.解决了这个问题,在我的操作块之前,我有一个 web 请求通过创建操作块使用的列表进行分页,它将一一提交给操作块,而不是构建列表,然后将所有内容提交给操作块,所以事实上该块本身确实处理了它所拥有的所有内容,只是它当时只有一个条目,一旦我修复了它并将完整列表发送到它按预期工作并按预期并行处理的块。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 等待ActionBlock <T> - TPL DataFlow - Awaiting ActionBlock<T> - TPL DataFlow 如何将 TPL Dataflow TransformBlock 或 ActionBlock 放在单独的文件中? - How to put a TPL Dataflow TranformBlock or ActionBlock in a separate file? TPL数据流转换块发布到批处理块后跟动作块 - TPL Dataflow Transform block post to batch block followed by actionblock 直接指定动作或使用TPL数据流ActionBlock指定任务生成器有什么区别? - What is the difference between specifying Action directly or Task-Generator with TPL dataflow ActionBlock? TPL数据流。 无法将BatchBlock附加到ActionBlock。 无法从用法中推断出类型实参? - TPL Dataflow. Cannot attach BatchBlock to ActionBlock. Type argument cannot be inferred from usage? TPL 数据流:为什么会出现以下阻塞? - TPL Dataflow: Why is the following blocking? TPL.Dataflow - 在ActionBlock中发生无法处理的异常时阻止挂起<T> - TPL.Dataflow - preventing hangs when unhanded exception occurs in ActionBlock<T> TPL 数据流:避免在每次调用其委托时重复运行 using 块(例如写入 StreamWriter)的 ActionBlock - TPL Dataflow: ActionBlock that avoids repeatedly running a using-block (such as for writing to a StreamWriter) on every invocation of its delegate 取消数据流 ActionBlock 中的特定任务 - Cancelling specific task in a dataflow ActionBlock TPL DataFlow缠住我的头 - Wrapping my head around TPL DataFlow
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM