Enable autoscaling feature for AWS Glue Jobs using AWS CloudFormation

Question

Very recently AWS announced an autoscaling feature for Glue ETL and streaming jobs. I've seen today that the region I'm currently working in is now supporting this feature.

I can confirm that this is working using the console, as described in the documentation .

I was wondering if this could already be enabled using CloudFormation. In the CloudFormation documentation I don't see any settings that could enable this feature.

I'm guessing that this will only be available once the feature is out of preview mode, but I'm not 100% sure. Does anyone know more about this? Or can someone confirm my presumption?

Answer 1

Looks like you can just pass the right argument to the job to enable autoscaling, without having to wait on any resource updates.

Here is an example to enable bookmarks from the documentation, you should be able to pass "--enable-auto-scaling": "true" instead.

DefaultArguments: "--job-bookmark-option": "job-bookmark-enable"