A Problem on getting docker stats by Python

Question

I tried to use Python to get the docker stats, by using Python's docker module. the code is:

import docker
cli = docker.from_env()

for container in cli.containers.list():
    stream = container.stats()
    print(next(stream))

I run 6 docker containers, but when I run the code, It needs a few second to get all containers' stats, so is there have some good methods to get the stats immediately?

Answer 1

Docker stats inherently takes a little while, a large part of this is waiting for the next value to come through the stream

$ time docker stats 1339f13154aa --no-stream
CONTAINER ID   NAME                CPU %     MEM USAGE / LIMIT     MEM %     NET I/O      BLOCK I/O   PIDS
...
real    0m1.556s
user    0m0.020s
sys 0m0.015s

You could reduce the time it takes to execute by running the commands in parralell, as opposed to one at a time.

To achieve this, you could use the wonderful threading or multiprocessing library.

Digital Ocean provides a good tutorial on how to accomplish this with a ThreadPoolExecutor :

import requests
import concurrent.futures

def get_wiki_page_existence(wiki_page_url, timeout=10):
    response = requests.get(url=wiki_page_url, timeout=timeout)

    page_status = "unknown"
    if response.status_code == 200:
        page_status = "exists"
    elif response.status_code == 404:
        page_status = "does not exist"

    return wiki_page_url + " - " + page_status

wiki_page_urls = [
    "https://en.wikipedia.org/wiki/Ocean",
    "https://en.wikipedia.org/wiki/Island",
    "https://en.wikipedia.org/wiki/this_page_does_not_exist",
    "https://en.wikipedia.org/wiki/Shark",
]
with concurrent.futures.ThreadPoolExecutor() as executor:
    futures = []
    for url in wiki_page_urls:
        futures.append(executor.submit(get_wiki_page_existence, wiki_page_url=url))
    for future in concurrent.futures.as_completed(futures):
        print(future.result())

Answer 2

This is what I use to get the stats directly for each running docker (still takes like 1 second per container, so I don't think it can be helped). Besides this if it helps https://docker-py.readthedocs.io/en/stable/containers.html documentation for the arguments. Hope it helps.

import docker
client = docker.from_env()
containers = client.containers.list()
for x in containers:
    print(x.stats(decode=None, stream=False))

As the TheQueenIsDead suggested, it might require threading if you want to get it faster.