Docker Python API - Tagging Containers

Question

I'm using Docker with AWS ECR repo. One of the steps they instruct you to do is to run "docker tag" to tag a built image with a tag that includes a "fully-ish qualified" location of where the image is going to be stored in ECR.

I was working on migrating a script I had to Python API (instead of doing shell calls to the docker client). I'm unable to find the equivalent of "docker tag" in the API docs at https://docker-py.readthedocs.io/en/stable/images.html .

Can somebody point me in the right direction?

Answer 1

For those of you using ECR/ECS in AWS, here is an example of how you go about this.

Amazon provides instructions like this in ECR to push your images:

aws ecr get-login --no-include-email --region us-west-2
docker build -t myproj .
docker tag calclab:latest XXXXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/myproj:latest
docker push XXXXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/myproj:latest

Here is the rough equivalent using the Docker Python API and Boto (AWS's Python library). It includes tagging the image twice in ECR, so that I can track each image's version number while tracking what the latest is (so my ECS Task can, by default, always grab the most current image)

import docker
import boto3

def ecrDemo(version_number):

    # The ECR Repository URI
    repo = XXXXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/myproj
    # The name of the [profile] stored in .aws/credentials
    profile = "sandbox"
    # The region your ECR repo is in
    region = "us-west-2"
    # How you want to tag your project locally
    local_tag = "myproj"

    #Set up a session
    session = boto3.Session(profile_name=profile, region_name=region)
    ecr = session.client('ecr')

    docker_api = docker.APIClient()

    print "Building image " + local_tag
    for line in docker_api.build(path='.', tag=local_tag, stream=True, \
        dockerfile='./Dockerfile.myproj'):
        process_docker_api_line(line)

    # Make auth call and parse out results
    auth = ecr.get_authorization_token()
    token = auth["authorizationData"][0]["authorizationToken"]
    username, password = b64decode(token).split(':')
    endpoint = auth["authorizationData"][0]["proxyEndpoint"]

    # print "Make authentication call"
    # docker_api.login(username=user, password=password, \
    #             registry=endpoint, reauth=True)
    auth_config_payload = {'username': username, 'password': password}



    version_tag = repo + ':latest'
    latest_tag = repo + ':' + version_number

    print "Tagging version " + version_tag
    if docker_api.tag(local_tag, version_tag) is False:
        raise RuntimeError("Tag appeared to fail: " + version_tag)

    print "Tagging latest " + latest_tag
    if docker_api.tag(local_tag, latest_tag) is False:
        raise RuntimeError("Tag appeared to fail: " + tag_latest)

    print "Pushing to repo " + version_tag
    for line in docker_api.push(version_tag, stream=True, auth_config=auth_config_payload):
        self.process_docker_api_line(line)

    print "Pushing to repo " + latest_tag
    for line in docker_api.push(latest_tag, stream=True, auth_config=auth_config_payload):
        self.process_docker_api_line(line)

    print "Removing taged deployment images"
    # You will still have the local_tag image if you need to troubleshoot
    docker_api.remove_image(version_tag, force=True)
    docker_api.remove_image(latest_tag, force=True)

def process_docker_api_line(payload):
    """ Process the output from API stream, throw an Exception if there is an error """
    # Sometimes Docker sends to "{}\n" blocks together...
    for segment in payload.split('\n'):
        line = segment.strip()
        if line:
            try:
                line_payload = json.loads(line)
            except ValueError as ex:
                print "Could not decipher payload from API: " + ex.message
            if line_payload:
                if "errorDetail" in line_payload:
                    error = line_payload["errorDetail"]
                    sys.stderr.write(error["message"])
                    raise RuntimeError("Error on build - code " + `error["code"]`)
                elif "stream" in line_payload:
                    sys.stdout.write(line_payload["stream"])

Answer 2

These are the steps you can use to build and tag an image.

import docker
tag = 'latest' # or whatever you want
client = docker.from_env()
# identifier of the image on your system
dockername = "%s:%s" % (<name of the image on your system>, <tag>)
# the target identifier
target = "%s:%d/%s" % (<registry address>, <registry_port>, <id or name of the image>)

# the url is usually unix://var/run/docker.sock' but depends on your environment
cli = docker.APIClient(base_url="<the daemon\'s url or socket>")
# build the image
cli.build(path=..., tag=dockername, pull=..., buildargs=...)
# tag it
image = client.images.get(dockername)
image.tag(target, tag=tag)

Answer 3

This answer helped me a great deal, thank you!

When I was developing my flow, I was confused by the terminology on the docker-py page, which I think would benefit by more examples.

I was not sure during development if I was correctly building, or if I was having issues with authentication or authorization.

I needed to carefully monitor my build results using the Docker CLI, and capture and analyze the output from the different build, tag and push functions.

Another caveat with getting output from these functions is a warning that is clearly stated for the docker-py pull() function but not the others: if you request that the function provide a generator for output from the operation you must consume that generator. I was able to get my flow working with a debug level of verbosity.

Unfortunately, when I toggled off the verbosity in my code and did not consume the generators for build() and push() ( tag() only has a boolean result), my flow only appeared to work: it was not throwing errors, but it also was not building or pushing the code! It is better to either not turn on the streaming output if you're not in debug mode, or leave it on and use deque() to consume the output without processing it.

To summarize the differences in how tags are used:

• build() takes a 'local tag', which is just the name of the build, eg 'myproj'
• tag() applies a 'version tag' to a 'local tag' you just built with build() , the version tag will include the registry and a version label, eg, myregistry.mydomain.com/myname/myproj:latest
• push() will push the image in a 'version tag' to the registry designated in the version tag. So in this case, the image you tagged as myregistry.mydomain.com/myname/myproj:latest will be pushed to the registry myregistry.mydomain.com .