[英]notebook to execute Databricks job
Is there an api or other way to programmatically run a Databricks job.是否有 api 或其他方式以编程方式运行 Databricks 作业。 Ideally, we would like to call a Databricks job from a notebook.
理想情况下,我们希望从笔记本中调用 Databricks 作业。 Following just gives currently running job id but that's not very useful:
以下只是给出当前正在运行的作业 ID,但这不是很有用:
dbutils.notebook.entry_point.getDbutils().notebook().getContext().currentRunId().toString()
To run a databricks job, you can use Jobs API
.要运行数据块作业,您可以使用
Jobs API
。 I have a databricks job called for_repro
which I ran using the 2 ways provided below from databricks notebook.我有一个名为
for_repro
的 databricks 作业,我使用 databricks notebook 下面提供的 2 种方式运行该作业。
Using requests
library:使用
requests
库:
Settings -> User settings
.Settings -> User settings
创建访问令牌。 Under Access token
tab, click generate token.Access token
选项卡下,单击生成令牌。import requests
import json
my_json = {"job_id": <your_job-id>}
auth = {"Authorization": "Bearer <your_access-token>"}
response = requests.post('https://<databricks-instance>/api/2.0/jobs/run-now', json = my_json, headers=auth).json()
print(response)
<databricks-instance>
value from the above code can be extracted from your workspace URL.<databricks-instance>
值可以从您的工作区 URL 中提取。 Using %sh magic command script:使用 %sh 魔法命令脚本:
%sh
curl --netrc --request POST --header "Authorization: Bearer <access_token>" \
https://<databricks-instance>/api/2.0/jobs/run-now \
--data '{"job_id": <your job id>}'
Refer to this Microsoft documentation to know all other operations that can be achieved using Jobs API
.请参阅此 Microsoft 文档以了解使用
Jobs API
可以实现的所有其他操作。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.