[英]How to measure Docker build steps duration?
Is it possible to configure Docker to output timing for the build of a Dockerfile?是否可以将 Docker 配置为输出构建 Dockerfile 的时间?
We run a medium sized development team and would like to collect statistics on the average build times for our developers' development containers.我们运行着一个中等规模的开发团队,并希望收集有关开发人员开发容器的平均构建时间的统计数据。
Ideally, it should measure the duration of individual steps.理想情况下,它应该测量单个步骤的持续时间。
BuildKit , which was experimental in 18.06 and generally available in 18.09, has this functionality built in. To configure the dockerd daemon with experimental mode, you can setup the daemon.json: BuildKit在 18.06 中是实验性的,并且在 18.09 中普遍可用,它内置了这个功能。要使用实验模式配置 dockerd 守护进程,你可以设置 daemon.json:
$ cat /etc/docker/daemon.json
{
"experimental": true
}
Then you can enable BuildKit from the client side with an environment variable:然后,您可以使用环境变量从客户端启用 BuildKit:
$ export DOCKER_BUILDKIT=1
$ docker build -t java-test:latest .
[+] Building 421.6s (13/13) FINISHED
=> local://context (.dockerignore) 1.6s
=> => transferring context: 56B 0.3s
=> local://dockerfile (Dockerfile) 2.0s
=> => transferring dockerfile: 895B 0.4s
=> CACHED docker-image://docker.io/tonistiigi/copy:v0.1.3@sha256:e57a3b4d6240f55bac26b655d2cfb751f8b9412d6f 0.1s
=> docker-image://docker.io/library/openjdk:8-jdk-alpine 1.0s
=> => resolve docker.io/library/openjdk:8-jdk-alpine 0.0s
=> local://context 1.7s
=> => transferring context: 6.20kB 0.4s
=> docker-image://docker.io/library/openjdk:8-jre-alpine 1.3s
=> => resolve docker.io/library/openjdk:8-jre-alpine 0.0s
=> /bin/sh -c apk add --no-cache maven 61.0s
=> copy /src-0/pom.xml java/pom.xml 1.3s
=> /bin/sh -c mvn dependency:go-offline 339.4s
=> copy /src-0 java 0.9s
=> /bin/sh -c mvn package -Dmaven.test.skip=true 10.2s
=> copy /src-0/gs-spring-boot-docker-0.1.0.jar java/app.jar 0.8s
=> exporting to image 1.2s
=> => exporting layers 1.0s
=> => writing image sha256:d57028743ca10bb4d0527a294d5c83dd941aeb1033d4fe08949a135677846179 0.1s
=> => naming to docker.io/library/java-test:latest 0.1s
There's also an option to disable the tty console output which generates output more suitable for scripting with each section having a start, stop, and duration:还有一个选项可以禁用 tty 控制台输出,它生成的输出更适合脚本编写,每个部分都有开始、停止和持续时间:
$ docker build -t java-test:latest --progress plain .
#1 local://dockerfile (Dockerfile)
#1 digest: sha256:da721b637ea85add6e26070a48520675cefc2bed947c626f392be9890236d11b
#1 name: "local://dockerfile (Dockerfile)"
#1 started: 2018-09-05 19:30:53.899809093 +0000 UTC
#1 completed: 2018-09-05 19:30:53.899903348 +0000 UTC
#1 duration: 94.255µs
#1 started: 2018-09-05 19:30:53.900069076 +0000 UTC
#1 transferring dockerfile: 38B done
#2 ...
#2 local://context (.dockerignore)
#2 digest: sha256:cbf55954659905f4d7bd2fc3e5e52d566055eecd94fd7503565315022d834c21
#2 name: "local://context (.dockerignore)"
#2 started: 2018-09-05 19:30:53.899624016 +0000 UTC
#2 completed: 2018-09-05 19:30:53.899695455 +0000 UTC
#2 duration: 71.439µs
#2 started: 2018-09-05 19:30:53.899839335 +0000 UTC
#2 completed: 2018-09-05 19:30:54.359527504 +0000 UTC
#2 duration: 459.688169ms
#2 transferring context: 34B done
#1 local://dockerfile (Dockerfile)
#1 completed: 2018-09-05 19:30:54.592304408 +0000 UTC
#1 duration: 692.235332ms
#3 docker-image://docker.io/tonistiigi/copy:v0.1.3@sha256:e57a3b4d6240f55ba...
#3 digest: sha256:39386c91e9f27ee70b2eefdee12fc8a029bf5edac621b91eb5f3e6001d41dd4f
#3 name: "docker-image://docker.io/tonistiigi/copy:v0.1.3@sha256:e57a3b4d6240f55bac26b655d2cfb751f8b9412d6f7bb1f787e946391fb4b21b"
#3 started: 2018-09-05 19:30:54.731749377 +0000 UTC
#3 completed: 2018-09-05 19:30:54.732013326 +0000 UTC
#3 duration: 263.949µs
#5 docker-image://docker.io/library/openjdk:8-jdk-alpine
#5 digest: sha256:d680c6a82813d080081fbc3c024d21ddfa7ff995981cc7b4bfafe55edf80a319
#5 name: "docker-image://docker.io/library/openjdk:8-jdk-alpine"
#5 started: 2018-09-05 19:30:54.731483638 +0000 UTC
#5 completed: 2018-09-05 19:30:54.732480345 +0000 UTC
#5 duration: 996.707µs
#4 docker-image://docker.io/library/openjdk:8-jre-alpine
#4 digest: sha256:9ed31df4e6731a1718ea93bfa77354ad1ea2d1625c1cb16e2087d16d0b84bd00
#4 name: "docker-image://docker.io/library/openjdk:8-jre-alpine"
#4 started: 2018-09-05 19:30:54.73176516 +0000 UTC
#4 completed: 2018-09-05 19:30:54.732603067 +0000 UTC
#4 duration: 837.907µs
#7 local://context
#7 digest: sha256:efe765161a29e2bf7a41439cd2e6656fcf6fa6bc97da825ac9b5a0d8adecf1ac
#7 name: "local://context"
#7 started: 2018-09-05 19:30:54.73178732 +0000 UTC
#7 completed: 2018-09-05 19:30:54.731880943 +0000 UTC
#7 duration: 93.623µs
#7 started: 2018-09-05 19:30:54.792740019 +0000 UTC
#7 transferring context: 473B done
#7 completed: 2018-09-05 19:30:55.059008345 +0000 UTC
#7 duration: 266.268326ms
#9 /bin/sh -c mvn dependency:go-offline
#9 digest: sha256:2197672cd7a44d93e0dba40aa00d7ef41f8680226d91f469d1c925646bdc8d6d
#9 name: "/bin/sh -c mvn dependency:go-offline"
#9 started: 2018-09-05 19:30:55.203449147 +0000 UTC
#9 completed: 2018-09-05 19:30:55.203449147 +0000 UTC
#9 duration: 0s
#9 cached: true
#10 copy /src-0 java
#10 digest: sha256:36cf252c34be098731bd8c5fb3f273f9c1437a5f74a65a3555d71150c2092fa7
#10 name: "copy /src-0 java"
#10 started: 2018-09-05 19:30:55.203449147 +0000 UTC
#10 completed: 2018-09-05 19:30:55.203449147 +0000 UTC
#10 duration: 0s
#10 cached: true
#11 /bin/sh -c mvn package -Dmaven.test.skip=true
#11 digest: sha256:390464b1fdc7a4c833b3476033d95b7714e22bcbfd018469e97b04781cb41532
#11 name: "/bin/sh -c mvn package -Dmaven.test.skip=true"
#11 started: 2018-09-05 19:30:55.203449147 +0000 UTC
#11 completed: 2018-09-05 19:30:55.203449147 +0000 UTC
#11 duration: 0s
#11 cached: true
#12 copy /src-0/gs-spring-boot-docker-0.1.0.jar java/app.jar
#12 digest: sha256:a7d60191a720f80de72a77ebe0d4bd1b0fd55d44e623661e80916b7fd1952076
#12 name: "copy /src-0/gs-spring-boot-docker-0.1.0.jar java/app.jar"
#12 started: 2018-09-05 19:30:55.203449147 +0000 UTC
#12 completed: 2018-09-05 19:30:55.203555216 +0000 UTC
#12 duration: 106.069µs
#12 cached: true
#6 /bin/sh -c apk add --no-cache maven
#6 digest: sha256:db505db5e418f195c7bad3a710ad40bec3d91d47ff11a6f464b3ae37af744e7d
#6 name: "/bin/sh -c apk add --no-cache maven"
#6 started: 2018-09-05 19:30:55.203449147 +0000 UTC
#6 completed: 2018-09-05 19:30:55.203449147 +0000 UTC
#6 duration: 0s
#6 cached: true
#8 copy /src-0/pom.xml java/pom.xml
#8 digest: sha256:f032d4ff111c6ab0efef1a4e37d2467fffe43f48a529b8d56291ec81f96296ab
#8 name: "copy /src-0/pom.xml java/pom.xml"
#8 started: 2018-09-05 19:30:55.203449147 +0000 UTC
#8 completed: 2018-09-05 19:30:55.203449147 +0000 UTC
#8 duration: 0s
#8 cached: true
#13 exporting to image
#13 digest: sha256:d536dc2895c30fbde898bb4635581350a87c21f3695913ba21850a73d31422d9
#13 name: "exporting to image"
#13 started: 2018-09-05 19:30:55.203674127 +0000 UTC
#13 exporting layers done
#13 writing image sha256:d57028743ca10bb4d0527a294d5c83dd941aeb1033d4fe08949a135677846179 0.1s done
#13 naming to docker.io/library/java-test:latest
#13 completed: 2018-09-05 19:30:55.341300051 +0000 UTC
#13 duration: 137.625924ms
#13 naming to docker.io/library/java-test:latest 0.0s done
time docker build .
docker build . | while read line ; do echo "$(date)| $line"; done;
Wed 5 Sep 2018 19:12:22 BST| Sending build context to Docker daemon 27.65kB
Wed 5 Sep 2018 19:12:22 BST| Step 1/19 : FROM centos:centos7
Wed 5 Sep 2018 19:12:22 BST| ---> 49f7960eb7e4
...
You can improve the results by only outputting the "Step ?/? :" lines, like so:您可以通过仅输出“Step ?/? :”行来改善结果,如下所示:
docker build . | grep "^Step" | while read line ; do echo "$(date +%s)| $line"; done;
1536171476| Step 1/19 : FROM centos:centos7
1536171476| Step 2/19 : ENV TERM xterm
1536171476| Step 3/19 : RUN *** omitted ***
1536171476| Step 4/19 : RUN *** omitted ***
1536171476| Step 5/19 : COPY *** omitted ***
1536171476| Step 6/19 : RUN *** omitted ***
1536171476| Step 7/19 : COPY *** omitted ***
You can make this into a script if you wanted to run in a CI/CD pipeline, or add to your developer tools.如果您想在 CI/CD 管道中运行或添加到您的开发人员工具中,您可以将其制作成脚本。
#!/bin/bash
# script: time-docker-build.sh
#
# All command line arguments are passed to docker build command.
#
# usage: ./time-docker-build.sh
#
DATE_FORMAT="+%s"
(
# Output START line
echo "$(date $DATE_FORMAT) | - 0 - START"
docker build $* . | \
grep "^Step" | \
while read line ;
do
# Output build output prefixed with date
echo "$(date $DATE_FORMAT) | $line";
done;
# Output END line
echo "$(date $DATE_FORMAT) | - -1 - END"
) | (
# Generate JSON array output.
# - START is step: 0
# - END is step: -1
echo "["
FIRST_RUN=true
while read line ;
do
[[ -z "$FIRST_RUN" ]] && echo "," # if not first line, print ','
lineArray=($line)
time="${lineArray[0]}" # step is 0th
step="${lineArray[3]}" # step is 2nd
cmd="${lineArray[@]:5}" # cmd is everything after 5th
stepNum=${step/\/*/}
escapedCmd="${cmd//\"/\\\"}" # escape all double quotes '"'
echo " {"
echo " \"time\": $time,"
echo " \"step\": $stepNum,"
echo " \"cmd\": \"$escapedCmd\""
echo -n " }"
unset FIRST_RUN
done
echo
echo "]"
)
bash-3.2$ ./time-docker-build.sh
[
{
"time": 1536174052,
"step": 0,
"cmd": "START"
},
{
"time": 1536174052,
"step": 1,
"cmd": "FROM centos:centos7"
},
{
"time": 1536174052,
"step": 2,
"cmd": "ENV TERM xterm"
},
Script is available as a gist here:脚本可作为要点在这里:
https://gist.github.com/philpoore/05eca572f3aadf70f529c470ac679147 https://gist.github.com/philpoore/05eca572f3aadf70f529c470ac679147
I had the same issue, there is a command line utility called ts taking care of that issue (part of moreutils ):我遇到了同样的问题,有一个名为 ts 的命令行实用程序可以解决这个问题( moreutils 的一部分):
$ docker build . | ts
Aug 09 09:09:56 Sending build context to Docker daemon 21.2MB
Aug 09 09:09:56 Step 1/22 : FROM node:alpine as apibuild
Aug 09 09:09:56 ---> 5a519d1e3a24
[...]
Aug 09 09:12:56 Successfully built 80bf576659e9
You could use the tool time
to measure the build times.您可以使用工具
time
来衡量构建时间。 Eg例如
time docker build .
For individual build steps it's getting more difficult.对于单独的构建步骤,它变得越来越困难。 You could add a
RUN date
command after each step, but this would add another layer to the image.您可以在每一步之后添加一个
RUN date
命令,但这会为图像添加另一个图层。 So it's getting a bit messy.所以有点乱。
I was wondering that too.我也想知道。 The only solution I came up with for individual steps is to put
date +'%F %T'
to the end of each step.我为各个步骤提出的唯一解决方案是将
date +'%F %T'
放在每个步骤的末尾。
RUN set -x && apt-get update && date +'%F %T'
RUN apt-get install -y vim && date +'Time: %F %T'
Output:输出:
Get:1 http://security.debian.org stretch/updates InRelease [63.0 kB]
...
Time: 2018-03-08 00:42:41
In Bash, you could use echo $SECONDS
which gives time since the Bash session start;在 Bash 中,您可以使用
echo $SECONDS
它给出自 Bash 会话开始以来的时间; or time ( ... )
(a subshell), but the Docker build environment runs in sh
, not bash
.或
time ( ... )
(子shell),但 Docker 构建环境在sh
运行,而不是bash
。
You can use also docker buildx
to find time taken for each layers during building a docker image.您还可以使用
docker buildx
来查找docker buildx
映像期间每个层所花费的时间。 Buildx builds using the BuildKit engine and does not require DOCKER_BUILDKIT=1
environment variable to start the builds. Buildx 使用 BuildKit 引擎构建,不需要
DOCKER_BUILDKIT=1
环境变量来启动构建。
docker buildx docker build --tag "ubuntu" -f Dockerfile
Here ubuntu is the tag and -f
path to the Dockerfile这里 ubuntu 是 Dockerfile 的标签和
-f
路径
Link to the doc docker buildx链接到文档docker buildx
Many non native solutions here!这里有许多非本地解决方案!
As of November 2021, the correct native solution is to use the buildx .截至 2021 年 11 月,正确的本机解决方案是使用buildx 。 It is not experimental anymore.
它不再是实验性的。 As per docs
根据文档
Using buildx as a docker CLI plugin requires using Docker 19.03 or newer.
使用 buildx 作为 docker CLI 插件需要使用 Docker 19.03 或更新版本。 A limited set of functionality works with older versions of Docker when invoking the binary directly.
在直接调用二进制文件时,一组有限的功能适用于旧版本的 Docker。
Install it using the following command.使用以下命令安装它。 [ ref ]
[ 参考]
docker buildx install
From here on, you should be getting the new buildx cli interface.从这里开始,您应该会获得新的 buildx cli 界面。
If you are using recent version of docker desktop, you might already using buildx, I guess.如果您使用的是最新版本的 docker 桌面,我猜您可能已经在使用 buildx。
Image source: https://frightanic.com/computers/docker-buildx-the-best-thing-since-sliced-bread/图片来源: https : //frightanic.com/computers/docker-buildx-the-best-thing-since-sliced-bread/
Hope this helped.希望这有帮助。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.