[英]AWS Batch: mount an efs volume into a container using cloud formation
我有一个带有作业定义的 AWS Batch 计算环境。
我使用 Cloud Formation 创建了所有这些。
现在我想向这个作业定义添加一个 EFS 卷(名称:EFS-000,文件系统 ID:fs-9999999)和一个装载点。
我读
在第一个链接中,我们有一个任务定义示例(AWS ECS 而不是 AWS Batch 概念)
{
"containerDefinitions": [
{
"memory": 128,
"portMappings": [
{
"hostPort": 80,
"containerPort": 80,
"protocol": "tcp"
}
],
"essential": true,
"mountPoints": [
{
"containerPath": "/usr/share/nginx/html",
"sourceVolume": "efs-html"
}
],
"name": "nginx",
"image": "nginx"
}
],
"volumes": [
{
"name": "efs-html",
"efsVolumeConfiguration": {
"fileSystemId": "fs-1324abcd",
"transitEncryption": "ENABLED"
}
}
],
"family": "efs-tutorial"
}
似乎很容易将正确的代码添加到我的 Cloud Formation 配方中(我选择了 yaml 语法)。 在我的 ContainerDefinition 中,我添加了...
Volumes:
- Name: SRV
EfsVolumeConfiguration:
FileSystemId: fs-9999999
TransitEncryption: ENABLED
但是当我运行 Cloud Formation 配方时,我得到......
The following resource(s) failed to update: [ContentJob1].
Property validation failure: [Encountered unsupported properties in {/ContainerProperties/Volumes/0}: [EfsVolumeConfiguration]]
如果 EfsVolumeConfiguration 不是有效的属性...
如何使用 Cloud Formation 将 EFS 卷添加到 AWS Batch 作业定义?
在 CloudFormation 支持 EFS 之前,我一直使用这个解决方案。 但它仍然有用。
我创建一个启动模板
LaunchTemplate:
Type: AWS::EC2::LaunchTemplate
Properties:
LaunchTemplateName: !Join ['', [!Ref ServiceName, '-EC2-', LaunchTemplate]]
LaunchTemplateData:
UserData:
Fn::Base64: !Sub |
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="==MYBOUNDARY=="
--==MYBOUNDARY==
Content-Type: text/cloud-config; charset="us-ascii"
runcmd:
- mkdir /mnt/efs-misc
- mkdir /mnt/efs-jobs
- echo "${EfsJobs}:/ /mnt/efs-jobs nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=300,retrans=2,noresvport 0 0" >> /etc/fstab
- echo "${EfsMisc}:/ /mnt/efs-misc nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=300,retrans=2,noresvport 0 0" >> /etc/fstab
- mount -a
--==MYBOUNDARY==--
InstanceMarketOptions:
MarketType: spot
SpotOptions:
MaxPrice: 0.096
然后我创建了一个使用此模板的计算环境。
BatchCompute:
Type: 'AWS::Batch::ComputeEnvironment'
DependsOn: LaunchTemplate
Properties:
ComputeEnvironmentName: !Join ['', [!Ref ServiceName, '-EC2']]
ComputeResources:
AllocationStrategy: SPOT_CAPACITY_OPTIMIZED
DesiredvCpus: 0
Ec2KeyPair: !Join ['-', [MyComputeEnv, !Ref EnvironmentLow]]
InstanceRole: !GetAtt BatchInstanceProfile.Arn
InstanceTypes:
- c4.large
- c5.large
LaunchTemplate:
LaunchTemplateId: !Ref LaunchTemplate
Version: $Latest
MaxvCpus: 256
MinvCpus: 0
DesiredvCpus: 0
SecurityGroupIds:
- !Ref DefaultSecurityGroup
Subnets: !Split [',', !Join [',', [!Ref SubnetA, !Ref SubnetB, !Ref SubnetC ]]]
Type: SPOT
SpotIamFleetRole: SpotFleetTagServiceRole
ServiceRole: !Ref BatchServiceRole
State: ENABLED
Type: Managed
最后,我使用 AWS::Batch::JobDefinition 的 ContainerProperties/[Volumes|MountPoints] 添加我的 EFS 卷。
ContentJob0:
Type: 'AWS::Batch::JobDefinition'
Properties:
Type: Container
ContainerProperties:
Command:
- ls
Environment:
- Name: AAA
Value: AAA
ExecutionRoleArn: !GetAtt BatchJobRole.Arn
Image: !Join ['', [!Ref 'AWS::AccountId','.dkr.ecr.', !Ref 'AWS::Region', '.amazonaws.com/', !Ref Image ]]
LogConfiguration:
LogDriver: awslogs
JobRoleArn: !Ref BatchContainerRole
Memory: 2048
Vcpus: 2
Volumes:
- Name: MISC
Host:
SourcePath: '/mnt/efs-0'
- Name: JOBS
Host:
SourcePath: '/mnt/efs-1'
MountPoints:
- SourceVolume: MISC
ContainerPath: '/mnt/efs-0'
- SourceVolume: JOBS
ContainerPath: '/mnt/efs-1'
JobDefinitionName: !Join ['-', ['MyJob', !Ref ServiceName, 'EC2', '0']]
PlatformCapabilities:
- EC2
PropagateTags: true
RetryStrategy:
Attempts: 3
Timeout:
AttemptDurationSeconds: 300
由于我需要将 SPOT 实例和其他东西的价格设置到 LaunchTemplate 中,这对我来说是一个很好的解决方案。
InstanceMarketOptions:
MarketType: spot
SpotOptions:
MaxPrice: 0.096
我有一个带有作业定义的 AWS Batch 计算环境。
我使用 Cloud Formation 创建了所有这些。
现在我想在此作业定义中添加一个 EFS 卷(名称:EFS-000,文件系统 ID:fs-9999999)和一个 MountPoint。
我读
在第一个链接中,我们有一个任务定义示例(AWS ECS 而不是 AWS Batch 概念)
{
"containerDefinitions": [
{
"memory": 128,
"portMappings": [
{
"hostPort": 80,
"containerPort": 80,
"protocol": "tcp"
}
],
"essential": true,
"mountPoints": [
{
"containerPath": "/usr/share/nginx/html",
"sourceVolume": "efs-html"
}
],
"name": "nginx",
"image": "nginx"
}
],
"volumes": [
{
"name": "efs-html",
"efsVolumeConfiguration": {
"fileSystemId": "fs-1324abcd",
"transitEncryption": "ENABLED"
}
}
],
"family": "efs-tutorial"
}
似乎很容易将正确的代码添加到我的 Cloud Formation 配方中(我选择了 yaml 语法)。 在我的 ContainerDefinition 中,我添加了...
Volumes:
- Name: SRV
EfsVolumeConfiguration:
FileSystemId: fs-9999999
TransitEncryption: ENABLED
但是当我运行 Cloud Formation 配方时,我得到....
The following resource(s) failed to update: [ContentJob1].
Property validation failure: [Encountered unsupported properties in {/ContainerProperties/Volumes/0}: [EfsVolumeConfiguration]]
如果 EfsVolumeConfiguration 不是有效属性...
如何使用 Cloud Formation 将 EFS 卷添加到 AWS 批处理作业定义?
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.