[英]Dockerfile for Spark/Java application to execute via Spark Operator
I am trying to run spark/java application on kubernetese (via minikube) using spark-operator.我正在尝试使用 spark-operator 在 kubernetese(通过 minikube)上运行 spark/java 应用程序。 I am getting a bit confused on what should I place in the Dockerfile so that it could be built in the image format and execute via spark-operator?我对应该在 Dockerfile 中放置什么内容感到有些困惑,以便它可以以图像格式构建并通过 spark-operator 执行?
Sample spark-operator.yaml:示例 spark-operator.yaml:
apiVersion: sparkoperator.k8s.io/v1beta2
kind: SparkApplication
metadata:
name: my-spark-app
namespace: default
spec:
type: Java
mode: cluster
image: docker/repo/my-spark-app-image
mainApplicationFile: local:///opt/app/my-spark-app.jar
As mentioned above, the spark operator yaml only requires the jar and the image location.如上所述,火花算子 yaml 只需要 jar 和图像位置。 So, do I need to mention just below in my Dockerfile?那么,我是否需要在我的 Dockerfile 中提及以下内容? Is there any sample Dockerfile available which I can refer?是否有任何样品 Dockerfile 可供我参考?
Dockerfile: Dockerfile:
FROM openjdk11:alpine-jre
COPY target/*.jar /opt/app/csp_auxdb_refresh.jar
COPY src/main/resources/* opt/app
In the Dockerfile
you have provided, nor Spark, nor other dependencies are installed.在您提供的Dockerfile
中,也没有安装 Spark,也没有安装其他依赖项。 To quickly get started, use gcr.io/spark-operator/spark:v3.1.1
as the base for your image, ie change the FROM
statement to FROM gcr.io/spark-operator/spark:v3.1.1
and build again.要快速开始,请使用gcr.io/spark-operator/spark:v3.1.1
作为映像的基础,即将FROM
语句更改为FROM gcr.io/spark-operator/spark:v3.1.1
并再次构建。
There is a great guide on how to get started with the spark-operator
in their Github repo ( here ).在他们的 Github 存储库( 这里)中有一个关于如何开始使用spark-operator
的很好的指南。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.