How to use DataBricks dbutils jar outside notebook?

Question

DataBricks dbutils library needs to be used in eclipse or any other IDE. Methods like dbutils.secrets.get are not available from SecretUtil API outside notebook. In this scenario we can use com.databricks jar

Answer 1

This is the Maven Repo for DataBricks dbutils library

<dependency>
<groupId>com.databricks</groupId>
<artifactId>dbutils-api_2.11</artifactId>
<version>0.0.3</version>

Once you add this import this in your code

import com.databricks.dbutils_v1.DBUtilsHolder.dbutils

This is the dbutils what we use in DataBricks notebook. You can use it like

dbutils.secrets.get(scope, name)

Answer 2

1.If running your java app through IDE ---

a. download the required jar using databricks-connect.

b.add the downloaded jars.

c.now code will be ---

com.databricks.service.SecretUtils$ secretClient = com.databricks.service.DBUtils.secrets();
System.out.println(secretClient.get("<secret-scope>", "<secret-key>"));

now when you run this it will in your console it will give you a command that you need to run on any of the cluster in your databricks account to get the token. once you get the token you need to write the code as:

com.databricks.service.SecretUtils$ secretClient = com.databricks.service.DBUtils.secrets();
secretClient.setToken("<token>");
System.out.println(secretClient.get("<secret-scope>", "<secret-key>"));

2.Now if you are going to deploy the jar on databricks then use the below code: a.add the latest maven dependency for dbutils-api can be found here:

https://docs.databricks.com/dev-tools/databricks-utils.html#databricks-utilities-api-library

then simply run the following code:

com.databricks.dbutils_v1.SecretUtils hostedSecretClient = com.databricks.dbutils_v1.DBUtilsHolder.dbutils().secrets();
System.out.println(hostedSecretClient.get("<secret-scope>", "<secret-key>"));

Hope this will work!!!