简体繁体 English

限制AWS Data Pipeline上的并发执行

[英]Limit concurrent executions on AWS Data Pipeline

原文 2018-02-09 11:29:06 4 1 amazon-web-services/ amazon-data-pipeline

Is there a way to limit concurrent execution on an AWS Data Pipeline? 有没有办法限制AWS Data Pipeline上的并发执行？ We need to limit simultaneous executions to 1. 我们需要将同时执行限制为1。

Something similar to what Oozie has with the <concurrency> property? 类似于Oozie的<concurrency>属性吗？

From the oozie docs: 来自oozie docs：

concurrency: The maximum number of actions for this job that can be running at the same time. 并发：可以同时运行此作业的最大操作数。 This value allows to materialize and submit multiple instances of the coordinator app, and allows operations to catchup on delayed processing. 此值允许实现并提交协调器应用程序的多个实例，并允许操作赶上延迟的处理。 The default value is 1 . 默认值为1。

1 个解决方案

You can use maxActiveInstances field under EC2Resource / EmrCluster to achieve this. 您可以使用EC2Resource / EmrCluster下的maxActiveInstances字段来实现此目的。

References - 参考-

https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-emrcluster.html https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-emrcluster.html

https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-ec2resource.html https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-ec2resource.html

我可以限制 AWS Lambda 的并发调用吗？ - Can I limit concurrent invocations of an AWS Lambda?

AWS Cognito 中每个用户是否有并发会话限制？ - Is there a concurrent sessions limit per user in AWS Cognito?

我如何使用 boto3 在 AWS lambda 中获得最大并发执行数、最小值和平均值 - How do i get the maximum number of concurrent executions,minimum and average in AWS lambda using boto3

AWS数据管道 - AWS Data Pipeline

AWS Data Pipeline中的ShellCommandActivity - ShellCommandActivity in AWS Data Pipeline

AWS数据管道错误 - AWS Data Pipeline Error

如何以秒为单位监控 Lambda 并发执行（或找到更好的解决方案来限制 Lambda 并发执行）？ - How Could I Monitor Lambda Concurrent Executions on a Second-by-Second Basis (or Find a Better Solution to Limit Lambda ConcurrentExecutions)?

如何达到 AWS Lambda 并发执行限制？ - How can the AWS Lambda concurrent execution limit be reached?

AWS Lambda > 达到并发限制时会发生什么 - AWS Lambda > what happens when reach concurrent limit

AWS DynamoDB 中一张表的并发 put 请求的限制是多少 - what is the limit of concurrent put requests for one table in AWS DynamoDB

暂无

暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 我可以限制 AWS Lambda 的并发调用吗？ - Can I limit concurrent invocations of an AWS Lambda? AWS Cognito 中每个用户是否有并发会话限制？ - Is there a concurrent sessions limit per user in AWS Cognito? 我如何使用 boto3 在 AWS lambda 中获得最大并发执行数、最小值和平均值 - How do i get the maximum number of concurrent executions,minimum and average in AWS lambda using boto3 AWS数据管道 - AWS Data Pipeline AWS Data Pipeline中的ShellCommandActivity - ShellCommandActivity in AWS Data Pipeline AWS数据管道错误 - AWS Data Pipeline Error 如何以秒为单位监控 Lambda 并发执行（或找到更好的解决方案来限制 Lambda 并发执行）？ - How Could I Monitor Lambda Concurrent Executions on a Second-by-Second Basis (or Find a Better Solution to Limit Lambda ConcurrentExecutions)? 如何达到 AWS Lambda 并发执行限制？ - How can the AWS Lambda concurrent execution limit be reached? AWS Lambda > 达到并发限制时会发生什么 - AWS Lambda > what happens when reach concurrent limit AWS DynamoDB 中一张表的并发 put 请求的限制是多少 - what is the limit of concurrent put requests for one table in AWS DynamoDB

相关标签

粤ICP备18138465号 © 2020-2024 STACKOOM.COM