How to recursively upload folder to Azure blob storage with Python

Question

I can upload single file to Azure blob storage with Python. But for a folder with multiple folders containing data, is there a way I can try to upload the whole folder with same directory to Azure?

Say I have

FOLDERA
------SUBFOLDERa
----------filea.txt
----------fileb.txt
------SUBFOLDERb
------SUBFOLDERc

I want to put this FOLDERA as above structure to Azure. Any hints?

Answer 1

There is nothing built in, but you can easily write that functionality in your code (see os.walk ).

Another option is to use the subprocess module to call into the azcopy command line tool.

Answer 2

@Krumelur is almost right, but here I want to give a working code example, as well as explain some folders are not be able to upload to azure blob storage.

1.Code example:

from azure.storage.blob import BlockBlobService,PublicAccess
import os

def run_sample():
    account_name = "your_account_name"
    account_key ="your_account_key"
    block_blob_service = BlockBlobService(account_name, account_key)
    container_name ='test1'

    path_remove = "F:\\"
    local_path = "F:\\folderA"

    for r,d,f in os.walk(local_path):        
        if f:
            for file in f:
                file_path_on_azure = os.path.join(r,file).replace(path_remove,"")
                file_path_on_local = os.path.join(r,file)
                block_blob_service.create_blob_from_path(container_name,file_path_on_azure,file_path_on_local)            

# Main method.
if __name__ == '__main__':
    run_sample()

2.You should remember that any empty folder can not be created / uploaded to azure blob storage, since there is no real "folder" in azure blob storage. The folder or directory is just a part of the blob name. So without a real blob file like test.txt inside a folder, there is no way to create/upload an empty folder. So in your folder structure, the empty folder SUBFOLDERb and SUBFOLDERc are not be able to upload to azure blob storage.

The test result is as below, all the non-empty folders are uploaded to blob storage in azure: