我如何在PySpark笔记本中读取Azure Synapse中的Lake数据库

Question

Hi I created a Database in Azure Synapse Studio and I can see the database and table in there, Now I have created a Notebook where I have added the required libraries but I am unable to read the table by below code. Can anyone fix what wrong am i doing here?

My database name is Utilities_66_Demo . It gives me error as

AnalysisException: Path does not exist: abfss://users@stcdmsynapsedev01.dfs.core.windows.net/Utilities_66_Demo.parquet

From where should I take the path? I tried to follow the MS article . Where Do I read path? if I click on edit Database, i get this

%%pyspark
df = spark.read.load('abfss://users@stcdmsynapsedev01.dfs.core.windows.net/Utilities_66_Demo.parquet', format='parquet')
display(df.limit(10))

Trying to access the created Lake Database Table: Selected Azure Synapse Analytics:

I select my workspace and in dropdown there is no table shown:

I select Edit and put my Db name and Table name and it says Invalid details.

Now I select Azure Dedicated Synapse Pool from Linked Service ,

I get no option to select in SQL Pool or Table, and without SQL Pool I am unable to create a Linked service just by inserting Table name:

Answer 1

Here are some official MS samples you can go through:

Access data on Azure Data Lake Storage Gen2 (ADLS Gen2) with Synapse Spark

And..

Access Synapse SQL table from Synapse Spark

Further, refer: Bring data to a notebook

Answer 2

You can directly go to your ADLS and right click the parquet file and select properties. There, you will be able to find the ABFSS path which is in the format: abfss://<container_name>@<storage_account_name>.dfs.core.windows.net/<path