[英]JcaPEMKeyConverter is provided by BouncyCastle, an optional dependency. To use support for EC Keys you must explicitly add dependency to classpath
I have a simple Flink streaming app.我有一个简单的 Flink 流媒体应用。 It runs well in a cluster created by
start-cluster.sh
command.它在由
start-cluster.sh
命令创建的集群中运行良好。
Now based on the Flink tutorial , I hope to deploy it in application mode natively in a Kubernetes cluster created by k3d on macOS.现在基于Flink 教程,希望在 macOS 上由k3d创建的 Kubernetes 集群中以应用模式原生部署。
First, I created a cluster by k3d cluster create dev
.首先,我通过
k3d cluster create dev
创建了一个集群。
Here is my Dockerfile :这是我的Dockerfile :
FROM flink
RUN mkdir -p $FLINK_HOME/usrlib
COPY target/streaming-0.1.jar $FLINK_HOME/usrlib/streaming-0.1.jar
I built and pushed it to Docker Hub.我构建并将其推送到 Docker Hub。
My cluster name is k3d-dev
, so I ran我的集群名称是
k3d-dev
,所以我跑了
flink run-application \
--target kubernetes-application \
-Dkubernetes.cluster-id=k3d-dev \
-Dkubernetes.container.image=hongbomiao/my-flink-xxx:latest \
local:///opt/flink/usrlib/streaming-0.1.jar
However, I got error:但是,我得到了错误:
The program finished with the following exception:
io.fabric8.kubernetes.client.KubernetesClientException: JcaPEMKeyConverter is provided by BouncyCastle, an optional dependency. To use support for EC Keys you must explicitly add this dependency to classpath.
at io.fabric8.kubernetes.client.internal.CertUtils.handleECKey(CertUtils.java:161)
at io.fabric8.kubernetes.client.internal.CertUtils.loadKey(CertUtils.java:131)
at io.fabric8.kubernetes.client.internal.CertUtils.createKeyStore(CertUtils.java:111)
at io.fabric8.kubernetes.client.internal.CertUtils.createKeyStore(CertUtils.java:243)
at io.fabric8.kubernetes.client.internal.SSLUtils.keyManagers(SSLUtils.java:128)
at io.fabric8.kubernetes.client.internal.SSLUtils.keyManagers(SSLUtils.java:122)
at io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:82)
at io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:62)
at io.fabric8.kubernetes.client.BaseClient.<init>(BaseClient.java:51)
at io.fabric8.kubernetes.client.DefaultKubernetesClient.<init>(DefaultKubernetesClient.java:105)
at org.apache.flink.kubernetes.kubeclient.FlinkKubeClientFactory.fromConfiguration(FlinkKubeClientFactory.java:102)
at org.apache.flink.kubernetes.KubernetesClusterClientFactory.createClusterDescriptor(KubernetesClusterClientFactory.java:61)
at org.apache.flink.kubernetes.KubernetesClusterClientFactory.createClusterDescriptor(KubernetesClusterClientFactory.java:39)
at org.apache.flink.client.deployment.application.cli.ApplicationClusterDeployer.run(ApplicationClusterDeployer.java:63)
at org.apache.flink.client.cli.CliFrontend.runApplication(CliFrontend.java:213)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1057)
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
After reading看完之后
I added我添加了
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpkix-jdk15on</artifactId>
<version>1.69</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
<version>1.69</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-ext-jdk15on</artifactId>
<version>1.69</version>
</dependency>
to my pom.xml file.到我的pom.xml文件。 I built and pushed to Docker Hub again.
我再次构建并推送到 Docker Hub。
When I ran the Flink command above, I still got same error.当我运行上面的 Flink 命令时,我仍然遇到同样的错误。 Any idea?
任何想法? Thanks!
谢谢!
UPDATE 1:更新 1:
Besides above pom.xml change, I manually downloaded those 3 jars and changed my Dockerfile to除了上面的pom.xml更改之外,我手动下载了这 3 个 jar 并将我的 Dockerfile 更改为
FROM flink
COPY lib/* $FLINK_HOME/lib
RUN mkdir -p $FLINK_HOME/usrlib
COPY target/streaming-0.1.jar $FLINK_HOME/usrlib/streaming-0.1.jar
and tried again, but still same error.并再次尝试,但仍然是同样的错误。
I can confirm the 3 jar files bcpkix-jdk15on-1.69.jar
, bcprov-ext-jdk15on-1.69.jar
, bcprov-jdk15on-1.69.jar
are in the docker image:我可以确认 3 个 jar 文件
bcpkix-jdk15on-1.69.jar
、 bcprov-ext-jdk15on-1.69.jar
、 bcprov-jdk15on-1.69.jar
在 docker 映像中:
➜ docker run -it 6c48af48db55c334003a307d1ef7a5fc5181f389613284b66b5cb97588b9708d sh
$ cd lib && ls
bcpkix-jdk15on-1.69.jar flink-dist_2.12-1.13.2.jar flink-table_2.12-1.13.2.jar log4j-slf4j-impl-2.12.1.jar
bcprov-ext-jdk15on-1.69.jar flink-json-1.13.2.jar log4j-1.2-api-2.12.1.jar
bcprov-jdk15on-1.69.jar flink-shaded-zookeeper-3.4.14.jar log4j-api-2.12.1.jar
flink-csv-1.13.2.jar flink-table-blink_2.12-1.13.2.jar log4j-core-2.12.1.jar
$ cd ../usrlib && ls
streaming-0.1.jar
UPDATE 2:更新 2:
I tried to start the session mode by我试图通过以下方式启动会话模式
/usr/local/Cellar/apache-flink/1.13.1/libexec/bin/kubernetes-session.sh
but still got same error.但仍然有同样的错误。 So now I can confirm when I was using application mode before, the issue is not related my Docker image.
所以现在我可以确认我之前使用应用程序模式时,问题与我的 Docker 映像无关。
I have those jars are located at ~/.m2
on my machine:我有这些罐子位于我的机器上的
~/.m2
:
Did I miss any other jars?我错过了其他罐子吗?
Also, I found the error only happens to the cluster created by k3d/k3s, but not minikube.另外,我发现错误只发生在由 k3d/k3s 创建的集群上,而不是 minikube。
After checking the code of检查代码后
The first script is pointing to the second script, and the second script has第一个脚本指向第二个脚本,第二个脚本有
# ...
CC_CLASSPATH=`manglePathList $(constructFlinkClassPath):$INTERNAL_HADOOP_CLASSPATHS`
# ...
"$JAVA_RUN" $JVM_ARGS -classpath "$CC_CLASSPATH" $log_setting org.apache.flink.kubernetes.cli.KubernetesSessionCli "$@"
I added echo $CC_CLASSPATH
, and printed out the classpath.我添加了
echo $CC_CLASSPATH
,并打印了类路径。
In my case, it is at /usr/local/Cellar/apache-flink/1.13.1/libexec/lib
.就我而言,它位于
/usr/local/Cellar/apache-flink/1.13.1/libexec/lib
。
After I put bcprov-jdk15on-1.69.jar
and bcpkix-jdk15on-1.69.jar
in the folder above, Flink can be deployed to k3s (k3d) now in both session and application modes.我把
bcprov-jdk15on-1.69.jar
和bcpkix-jdk15on-1.69.jar
上面的文件夹后,现在 Flink 可以在 session 和 application 模式下部署到 k3s (k3d) 上。
Download bcprov-jdk15on
and bcpkix-jdk15on
jar files at在以下位置下载
bcprov-jdk15on
和bcpkix-jdk15on
jar 文件
Then move to the folder然后移动到文件夹
/usr/local/Cellar/apache-flink/{version}/libexec/lib
Then you are good to go.那你就可以走了。
I stumbled upon this problem today on my m1 mac using docker and k3d to create a k3s cluster.我今天在我的 m1 mac 上使用 docker 和 k3d 创建 k3s 集群时偶然发现了这个问题。
I solved it by actually doing what the error message says: adding bouncy castle to the class path.我通过实际执行错误消息中的操作来解决它:将充气城堡添加到类路径中。
But you need to add bc to the plugin's class path in the pom.xml
, like so:但是您需要在
pom.xml
中将 bc 添加到插件的类路径中,如下所示:
...
<plugin>
<groupId>${quarkus.platform.group-id}</groupId>
<artifactId>quarkus-maven-plugin</artifactId>
<version>${quarkus.platform.version}</version>
<extensions>true</extensions>
<executions>
<execution>
<goals>
<goal>build</goal>
<goal>generate-code</goal>
<goal>generate-code-tests</goal>
</goals>
</execution>
</executions>
<!-- add bouncy castle to class path, so that elliptic curve keys work -->
<dependencies>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpkix-jdk15on</artifactId>
<version>1.69</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
<version>1.69</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-ext-jdk15on</artifactId>
<version>1.69</version>
</dependency>
</dependencies>
</plugin>
...
I don't know, it seems to me, that k3d always generates EC keys, as I had this exact problem on a totally different machine (win10/wsl2 + k3d) also.在我看来,我不知道 k3d 总是生成 EC 密钥,因为我在完全不同的机器(win10/wsl2 + k3d)上也遇到了这个确切的问题。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.