Dockerized Jenkins server on AWS

Question

I have a dockerized Jenkins build server set up like below and I want move it to AWS.

I have some questions on how to do it, Thanks.

1. Is ECS the right choice to deploy dockerized Jenkins agents?

2. Is Fargate launch type of ECS support Windows containers?

3. I know ECS can dynamically provision EC2 instances, can ECS provision like below?

   a. If there is no job to build, there is no ECS2 instance running in the cluster.

   b. If a build job started, ECS dynamically launch a EC2 instance to run the dockerized agents to handle it.

   c. After build job is finished, ECS cluster will automatically stop or terminate the running EC2 instance.

==================================================================

Jenkins master:
Runs as a Linux container hosted on a UBUNTU virtual machine.

Jenkins Agents:

Linux Agent:
Runs as a Linux container hosted on the same UBUNTU virtual machine as master.

Windows Agents:
Runs as a windows container hosted on a Windows server 2019. 

Answer 1

Well, I have some tips for you:

• Yes, ECS can dynamically provision EC2 Instances using autoscaling, but only when the threshold of a metric is reached in cloudwatch and an alarm is thrown out and autoscaling start to works. Start a task in ECS with a jenkins master server, and then start 1 or 2 agents when you go to execute a job is not a good tactic neither practical idea, who going to wake up these tasks?

• If you want to use a jenkins docker inside an EC2 instance and you have a master node running and you want to keep stopped your unused agents and start it only if is needed by a job maybe in your Jenkinsfile you can call a lambda function to start your agent, here and example in a Jenkinsfile:

   stage('Start Infrastructure') { steps { sh ''' #:/bin/bash aws lambda invoke --function-name Wake_Up_Jenkins_Agent --invocation-type Event --log-type Tail --payload '{"node","NodeJS-Java-Agent":"action"."start"}' logsfile.txt ''' } }

And later another stage to stop your agent, but your master node need to be online because it is the key and main component either called from the repository or your CI/CD process. Also you need to implement the lambda with a logical procedure to start or stop the instance.

• In my experience run Jenkins directly in EC2 is a better choice that run it in ECS or Fargate.