I want to convert a JSON string in variable to PySpark DataFrame on Databricks. I have a payload coming from API. It is a list of JSON objects hold o ...

When I'm trying to select something from one specific table in SQL Editor, I'm getting an error "Failure to initialize configuration". The query is si ...

In python I am trying to create and write to the table TBL in the database DB in Databricks. But I get an exception: A schema mismatch detected when w ...

I am trying to access a remote S3 bucket using Databricks. From what I have understood, this is what my code should look like, I seem to be gettin ...

So how we can fetch the secrets or keys for Azure Data Factory , logic apps, Azure Synapse and Azure DataBricks if we disable the public access for ke ...

Do we have a solution in python to list job runs that are created using Azure VM managed identity in Azure Databricks. Appreciate the help! I am get ...

I'm working on the Deployment of the Purview ADB Lineage Solution Accelerator. In step 3 of Install OpenLineage on Your Databricks Cluster section, th ...

I tried to start the Databricks SQL Warehouse cluster today, but received the following message: Clusters are failing to launch. Cluster launch wi ...

I have dataframe in pandas :- purchase_df. I want to convert it to sql table so I can perform sql query in pandas. I tried this method It throw an ...

So, here is my problem. I have a pyspark job stored in dbfs as I don't have access to databricks repo due to org policy and I cannot create a new clus ...

Question: Does Unity Catalog in Azure Databricks have the feature of classifying assets? If so, can someone please provide links to online documentati ...

Today our DBT workflow in databricks failed. The workflow runs as: Any suggestions what could be wrong or how to fix it? Version reported in Datab ...

enter image description here Trying to compile sql query in python. Which i used to do very frequently in python. But never faced this error in my pa ...

I've a dataframe with CreateDate column with this format: and I want to convert that format to date/timestamp, so the excepted output will be: I ...

I am trying to read from Azure Datalake and write to Bigquery . Refered https://docs.databricks.com/external-data/bigquery.html I have created 2 serv ...

Context I am using Databricks to connect to an Oracle Database and fetch data daily. We are using the following example code in PySpark to authentica ...

I have a DataFrame with a single column which is a struct type and contains an array. I'm trying to convert the CustomFields array column in 3 thre ...

we are trying to create an online feature store using cosmosdb following this documentation: https://learn.microsoft.com/en-us/azure/databricks/machin ...

Unity Catalog is the Azure Databricks data governance solution for the Lakehouse. Whereas, Microsoft Purview provides a unified data governance soluti ...

Can we use Synapse serverless pool (Built-in) to query a delta file's previous version? I am keen to a SQL statement similar to what we do in Databri ...