Read CSV file from Azure Blob Storage with out knowing the csv file name in python

Question

In azure Blob storage i have CSV files. I need to read those CSV files into dataframe. Csv file name vary every time. So i need to read csv from from azure blobstorage container folder. Folder name is constant but csv file name vary.

Answer 1

Here is how you can read csv files to dataframes

from azure.storage.blob import BlockBlobService
import pandas as pd
from io import StringIO

STORAGEACCOUNTNAME= "<YOUR_STORAGE_ACCOUNTNAME>"
STORAGEACCOUNTKEY= "<YOUR_STORAGE_ACCOUNT_KEY>"
CONTAINERNAME= "<YOUR_CONTAINER_NAME>"
BLOBNAME= "<BLOB_NAME>"

blob_service=BlockBlobService(account_name=STORAGEACCOUNTNAME,account_key=STORAGEACCOUNTKEY)

blobstring = blob_service.get_blob_to_text(CONTAINERNAME,BLOBNAME).content
df = pd.read_csv(StringIO(blobstring))
print(df)

RESULTS:

REFERENCES: Explore data in Azure Blob storage with the pandas Python package

Answer 2

To get this resolved. You may want to consider either having the CSV files with generic names so as to call them generically. But since you mentioned the CSV file name changes. I'd suggest saving only that CSV in the container then call it using the code below:

file_loc = "wasbs://<continer name>@<storage account name>.blob.core.windows.net/*.csv
df = pd.read_csv(file_loc)