如何从在自动缩放组后面运行并连接到 jms 队列的 aws ecs docker 容器中排出消息

Question

we are using aws ecs task to run our docker containers. our applications are connecting to TIBCO queues and using AlpakkaJMSListeners to read messages from queues.

We want to enable autoscaling for these containers at ec2 instance level. We have application LB with Target Group defined for some of our services which have http end points.

I know that if we use Target Group with http health check enabled and autoscaling on instances then ecs task will drain requests before stopping container.

Question is how to achieve the same behavior (ie draining all messages before container is stopped) when our containers are connected to JMS Queues with autoscaling?

Answer 1

we found a workaround for this one, by default ECS task when triggers TERM signal it waits for 30 sec to let current thread finish their task.

ref - https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_StopTask.html ,
By adding a SIGTERM handler in the listener we can stop consumption of msgs and process inflight messages.

add Terminal signal handling in jms listener like below

Signal.handle(new Signal("TERM"), signal -> {
        log.info("Received termination signal" +signal.getName());
        if ("TERM".equals(signal.getName())) {
            try {
                stopStream();
                log.info("Stopped stream");
            } catch (Exception e) {
                log.error("Exception while handling signal: "+e);
            }
        }
    });

note: in order to receive signal make sure your application is running as a separate java process;