简体   繁体   English

如何将气流工人的体积安装到气流kubernetes吊舱操作员上?

[英]How to mount volume of airflow worker to airflow kubernetes pod operator?

I am trying to using the kubernetes pod operator in airflow, and there is a directory that I wish to share with kubernetes pod on my airflow worker, is there is a way to mount airflow worker's directory to kubernetes pod? 我正在尝试在气流中使用kubernetes pod运算符,并且我想与我的气流工作者上的kubernetes pod共享一个目录,有没有办法将气流工作者的目录安装到kubernetes pod?

I tried with the code below, and the volumn seems not mounted successfully. 我尝试使用下面的代码,但似乎未成功装入该卷。

import datetime
import unittest
from unittest import TestCase
from airflow.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.kubernetes.volume import Volume
from airflow.kubernetes.volume_mount import VolumeMount


class TestMailAlarm(TestCase):
    def setUp(self):
        self.namespace = "test-namespace"
        self.image = "ubuntu:16.04"
        self.name = "default"

        self.cluster_context = "default"

        self.dag_id = "test_dag"
        self.task_id = "root_test_dag"
        self.execution_date = datetime.datetime.now()

        self.context = {"dag_id": self.dag_id,
                        "task_id": self.task_id,
                        "execution_date": self.execution_date}

        self.cmds = ["sleep"]
        self.arguments = ["100"]

        self.volume_mount = VolumeMount('test',
                                        mount_path='/tmp',
                                        sub_path=None,
                                        read_only=False)

        volume_config = {
            'persistentVolumeClaim':
                {
                    'claimName': 'test'
                }
        }
        self.volume = Volume(name='test', configs=volume_config)

        self.operator = KubernetesPodOperator(
            namespace=self.namespace, image=self.image, name=self.name,
            cmds=self.cmds,
            arguments=self.arguments,
            startup_timeout_seconds=600,
            is_delete_operator_pod=True,
            # the operator could run successfully but the directory /tmp is not mounted to kubernetes operator
            volume=[self.volume],
            volume_mount=[self.volume_mount],
            **self.context)

    def test_execute(self):
        self.operator.execute(self.context)

The example in the docs seems pretty similar to your code, only the parameters are plurals volume_mounts and volumes . 文档中的示例似乎与您的代码非常相似,只是参数是复数volume_mountsvolumes For your code it would look like this: 对于您的代码,它看起来像这样:

self.operator = KubernetesPodOperator(
            namespace=self.namespace, image=self.image, name=self.name,
            cmds=self.cmds,
            arguments=self.arguments,
            startup_timeout_seconds=600,
            is_delete_operator_pod=True,
            # the operator could run successfully but the directory /tmp is not mounted to kubernetes operator
            volumes=[self.volume],
            volume_mounts=[self.volume_mount],
            **self.context)

暂无
暂无

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 如何从使用 KubernetesPodOperator 触发它的 Airflow 主机将卷挂载到运行 docker 容器的 Kubernetes Pod - How to mount a volume to a Kubernetes Pod running a docker container from the Airflow host that triggers it using the KubernetesPodOperator 无法从气流吊舱中提取xcom-Kubernetes Pod Operator - Failed to extract xcom from airflow pod - Kubernetes Pod Operator Airflow - Kubernetes Executor:如何只挂载与 run_id 相对应的 Persistent Volume Claim 目录 - Airflow - Kubernetes Executor : How to only mount only a directory of a Persistent Volume Claim that correspond to the run_id Airflow Kubernetes pod for worker 无法启动任务 - Airflow Kubernetes pod for worker can't start task 如何从Kubernetes集群中的Airflow工作程序在另一个命名空间中的另一个窗格中运行命令 - How to run a command in another pod in another namespace from an Airflow worker in a Kubernetes cluster Airflow + Kubernetes 工作数据库连接 - Airflow + Kubernetes worker database connection 如何使用来自 Apache Airflow 的 Docker Operator 的卷 - how to use volume with Docker Operator from Apache Airflow 如何优化此 airflow 操作员代码以在 celery 工作程序上使用最少的 RAM? - How to optimize this airflow operator code to use minimal RAM on the celery worker? 如何在气流中重用电子邮件运算符? - How to reuse email operator in airflow? 如何将文件夹/文件挂载到 KubernetesPodOperator Airflow - How to mount folder/file to KubernetesPodOperator Airflow
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM