[英]How can I reference the path of a notebook in Databricks/what is %run doing?
I'm familiar with the %run magic commands in Databricks, but where do the notebooks actually live?我熟悉 Databricks 中的 %run 魔法命令,但是笔记本实际上在哪里?
The rootdir using %sh pwd appears to be /databricks/driver.使用 %sh pwd 的 rootdir 似乎是 /databricks/driver。 Making Python look for the notebook path (eg, subprocess.call([the/notebook/path]) fails, since it looks for the notebook path in this rootdir and obviously comes up empty. The notebooks must live somewhere else, but where? What path is %run invoking to find the notebook?让 Python 查找笔记本路径(例如, subprocess.call([the/notebook/path]) 失败,因为它在此 rootdir 中查找笔记本路径并且显然是空的。笔记本必须位于其他地方,但在哪里? %run 调用什么路径来查找笔记本?
For reference, I am trying to implement pytest within Databricks, and pytest.main() should allow me to run the tests within the notebook itself in lieu of the typical command line approach... if it could just find out where the notebook lives.作为参考,我正在尝试在 Databricks 中实现 pytest,而 pytest.main() 应该允许我在笔记本本身内运行测试,而不是典型的命令行方法……如果它可以找出笔记本所在的位置.
I was looking for this same thing and after some digging into dbutils
, this is the solution I came up with: 我正在寻找同样的东西,经过一些挖掘dbutils
,这是我提出的解决方案:
import json
context = json.loads(dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson())
print(context['extraContext']['aclPathOfAclRoot']) # Base path
print(context['extraContext']['notebook_path']) # Notebook path
You can get the current Notebook Path according to https://docs.azuredatabricks.net/user-guide/faq/get-notebook-path.html by using Scala. 您可以使用Scala根据https://docs.azuredatabricks.net/user-guide/faq/get-notebook-path.html获取当前的Notebook Path。 For Python you need the workaround via Scala and a Widget as described in the same FAQ: 对于Python,您需要通过Scala和Widget进行解决方法,如同一个FAQ中所述:
%scala
dbutils.widgets.text("notebook", dbutils.notebook.getContext().notebookPath.get)
And read it in python: 并在python中阅读:
%python
dbutils.widgets.get("notebook")
Based on @Martin's answer, but without using a widget (which creates a UI element that you might not want):基于@Martin 的回答,但不使用小部件(这会创建您可能不想要的 UI 元素):
%scala
spark.conf.set("nb.path", dbutils.notebook.getContext().notebookPath.get)
And then:进而:
%python
path = spark.conf.get("nb.path")
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.