Using Azure Function APP in Data Factory to run Python script

Question

I am merging to CSV files from a blob storage and uploading it to a Data Lake storage(Gen 2). The code works in PyCharm and VS Code, but I would like to run it in an Azure Data Factory pipeline using a function app. I get this error if I try to run it in a pipeline: "Operation on target Azure Function1 failed: Call to provided Azure function 'name' failed with status-'Unauthorized' and message - 'Invoking Azure function failed with HttpStatusCode - Unauthorized.'."

import azure.functions as func
import pandas as pd
import logging
from azure.storage.blob import BlobServiceClient
from azure.storage.filedatalake import DataLakeServiceClient

def main(req: func.HttpRequest) -> func.HttpResponse:
    logging.info('Python HTTP trigger function processed a request.')

    STORAGEACCOUNTURL= 'https://storage.blob.core.windows.net/'
    STORAGEACCOUNTKEY= '****'
    LOCALFILENAME= ['file1.csv', 'file2.csv']
    CONTAINERNAME= 'inputblob'

    file1 = pd.DataFrame()
    file2 = pd.DataFrame(])
    #download from blob

    blob_service_client_instance = BlobServiceClient(account_url=STORAGEACCOUNTURL, credential=STORAGEACCOUNTKEY)

    for i in LOCALFILENAME:
        with open(i, "wb") as my_blobs:
            blob_client_instance = blob_service_client_instance.get_blob_client(container=CONTAINERNAME, blob=i, snapshot=None)
            blob_data = blob_client_instance.download_blob()
            blob_data.readinto(my_blobs)
            if i == 'file1.csv':
                file1 = pd.read_csv(i)
            if i == 'file2.csv':
                file2 = pd.read_csv(i)

    # load

    # join the 2 dataframes into the final dataframe
    summary = pd.merge(left=file1, right=file2, on='key', how='inner')
        
    summary.to_csv(path_or_buf=r'path\summary.csv', index=False, encoding='utf-8')

    global service_client
            
    service_client = DataLakeServiceClient(account_url="https://storage.dfs.core.windows.net/", credential='****')
        
    file_system_client = service_client.get_file_system_client(file_system="outputdatalake")

    directory_client = file_system_client.get_directory_client("functionapp") 

    file_client = directory_client.create_file("merged.csv")
            
    local_file = open(r"path\summary.csv",'rb') 

    file_contents = local_file.read()

    file_client.upload_data(file_contents, overwrite=True) 

    return func.HttpResponse("This HTTP triggered function executed successfully.")

Answer 1

I tried to reproduce with python based http trigger and after first deployement faced the below error

Call to provided Azure function 'HttpTriggerT' failed with status-'Unauthorized' and message - 'Invoking Azure function failed with HttpStatusCode - Unauthorized.'.

Note: Try to refresh and restart the Function App service after deployment or changes to it. This is know to solve few transient issues.

Ideally you can use function key (for particular single function) or master/host key (for all functions in the function app service) to allow access. Managed identity provides secure access to the entire function app