How to get all running containers in Yarn ApplicationMaster code?

Question

I'm developing a long-running service using Yarn framework. The ApplicationMaster code just allocates and starts some containers and let them running forever. AM also reports the status of every running containers periodically. AM knows every container it allocated and started by explicitly storing them into a in-memory map.

Now the question is: in case of AM restart, ie a new appattempt is made. How does the new AM know all of the running containers that the old AM allocated? The new AM needs this because it needs to report the status of them.

The AMRMClient clearly doesn't have this interface for AM to get the container list of its application.

Answer 1

AM (one per Job) is a container which has been initialized after the RM (one per cluster) reserves the Memory and Vcores for that Job. Are you talking of AM failure and it needs to be restarted? If so then the new AM will start the new attempt with new containers (it will lose the connection to the old containers) and the old ones will be freed after some timeout due to NM (NodeManager) heartbeat to RM (which happens periodically).

Regarding the code, I am not sure how it is implemented.