[英]How to run Kafka from Jenkins Pipeline using Groovy and Kubernetes plugin?
I couldn't find such a specific command around the internet so I kindly ask for your help with this one :) 我在互联网上找不到这样的特定命令,所以请您帮忙:)
Context 上下文
I have defined a podTemplate
with a few containers, by using the containerTemplate
methods: 我已经通过使用
containerTemplate
方法定义了一个包含几个容器的podTemplate
:
ubuntu:trusty
(14.04 LTS) ubuntu:trusty
(14.04 LTS) postgres:9.6
wurstmeister/kafka:latest
wurstmeister/kafka:latest
Doing some Groovy coding in Pipeline, I install several dependencies into my ubuntu:trusty
container, such as latest Git, Golang 1.9, etc., and I also checkout my project from Github. 在Pipeline中进行一些Groovy编码,我在
ubuntu:trusty
trusty容器中安装了一些依赖项,例如最新的Git,Golang 1.9等,并且还从Github签出了我的项目。
After all that dependencies are dealt with, I manage to compile, run migrations (which means Postgres is up and running and my app is connected to it), and spin up my app just fine until it complains that Kafka is not running because it couldn't connect to any broker. 处理完所有依赖项后,我设法进行编译,运行迁移(这意味着Postgres已启动并正在运行,并且我的应用已连接到它),然后旋转我的应用就可以了,直到它抱怨Kafka不能运行,因为它不能不连接任何经纪人。
Debugging sessions 调试会话
After some debug sessions I have ps aux
'ed each and every container to make sure all the services I needed were running in their respective containers, such as: 经过一些调试会话后,我对每个容器进行了
ps aux
,以确保所需的所有服务都在各自的容器中运行,例如:
container(postgres) {
sh 'ps aux' # Show Postgres, as expected
}
container(linux) {
sh 'ps aux | grep post' # Does not show Postgres, as expected
sh 'ps aux | grep kafka' # Does not show Kafka, as expected
}
container(kafka) {
sh 'ps aux' # Does NOT show any Kafka running
}
I have also exported KAFKA_ADVERTISED_HOST_NAME
var to 127.0.0.1
as explained in the image docs, without success, with the following code: 如图像文档中所述,我还使用以下代码将
KAFKA_ADVERTISED_HOST_NAME
var导出到127.0.0.1
,但未成功完成:
containerTemplate(
name: kafka,
image: 'wurstmeister/kafka:latest',
ttyEnabled: true,
command: 'cat',
envVars: [
envVar(key: 'KAFKA_ADVERTISED_HOST_NAME', value: '127.0.0.1'),
envVar(key: 'KAFKA_AUTO_CREATE_TOPICS_ENABLE', value: 'true'),
]
)
Questions 问题
This image documentation details https://hub.docker.com/r/wurstmeister/kafka/ is explicit about starting a Kafka cluster with docker-compose up -d
该映像文档详细信息https://hub.docker.com/r/wurstmeister/kafka/明确涉及使用
docker-compose up -d
启动Kafka集群
1) How do I actually do that with this Kubernetes plugin + Docker + Groovy + Pipeline combo in Jenkins? 1)我实际上如何使用Jenkins中的Kubernetes插件+ Docker + Groovy + Pipeline组合来做到这一点?
2) Do I actually need to do that? 2)我真的需要这样做吗? Postgres image docs ( https://hub.docker.com/_/postgres/ ) also mentions about running the instance with
docker run
, but I didn't need to do that at all, which makes me think that containerTemplate
is probably doing it automatically. Postgres图像文档( https://hub.docker.com/_/postgres/ )也提到了使用
docker run
运行实例,但是我根本不需要这样做,这使我认为containerTemplate
可能正在做它会自动。 So why is it not doing this for the Kafka container? 那么为什么不对Kafka容器这样做呢?
Thanks! 谢谢!
So... problem is with this image, and way how kubernetes works with them. 所以...问题在于该图像以及kubernetes如何与它们一起工作。 Kafka does not start because you override dockers
CMD
with command:'cat'
which causes start-kafka.sh
to never run. Kafka无法启动,因为您使用
command:'cat'
覆盖了dockers CMD
,这导致start-kafka.sh
永远不会运行。 Because of above I suggest using different image. 由于上述原因,我建议使用其他图像。 Below template worked for me.
下面的模板为我工作。
containerTemplate(
name: 'kafka',
image: 'quay.io/jamftest/now-kafka-all-in-one:1.1.0.B',
resourceRequestMemory: '500Mi',
ttyEnabled: true,
ports: [
portMapping(name: 'zookeeper', containerPort: 2181, hostPort: 2181),
portMapping(name: 'kafka', containerPort: 9092, hostPort: 9092)
],
command: 'supervisord -n',
envVars: [
containerEnvVar(key: 'ADVERTISED_HOST', value: 'localhost')
]
),
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.