How to create a folder inside container in Azure Data Lake Storage Gen2 with the help of 'azure-storage' Package

Question

I am using Azure Data Lake Gen2 as my storage. I need to create different folder structure before uploading the files to relevant folders.

I am using "Azure-Storage" javascript library. But I am not able to figure out how to create a folder inside a container through this library.

Below is the code to connect to container. I am able to connect to container and upload a file into container itself.

var azure = require('azure-storage'); //connecting to container
var blobService = azure.createBlobService("DefaultEndpointsProtocol=<>;EndpointSuffix=core.windows.net");
blobService.createContainerIfNotExists("pbitestdl2",{publicAccessLevel: 'blob'},(error, result, response) => {
    if (!error) {
        console.log('connected');
    }
});

Answer 1

As you may know that, the official SDK of ADLS Gen2 is not available now.

Since you're using blob storage sdk for ADLS Gen2, here is something you should know:

In azure blob storage, there is one thing you need to know: there is no "folder" in blob storage, the "folder" actually is part of the blob name. You cannot create an empty folder inside blob storage.

When you're using blob storage SDK for ADLS Gen2, you cannot directly create a folder, you should create a blob file whose name include the folder name, like myfolder/my.txt .

So when using blob storage SDK for ADLS Gen2, you should use the following code:

var azure = require('azure-storage'); //connecting to container
var blobService = azure.createBlobService("DefaultEndpointsProtocol=xxx;EndpointSuffix=core.windows.net");
blobService.createContainerIfNotExists("pbitestdl2",{publicAccessLevel: 'blob'},(error, result, response) => {
    if (!error) {
        console.log('connected');
    }
});

blobService.createBlockBlobFromLocalFile('pbitestdl2', 'myfolder/my.txt', 'f:\\testfile.txt', function(error, result, response) {
    if (!error) {
      // file uploaded
      console.log('ok, uploaded');
    }
  });

then you can see the folder is created in Azure Data Lake Gen2 storage, screenshot as below:

Another way , you can use ADLS Gen2 Path - Create rest api to directly create a folder, but you need to do a lot of work to build authentication token for the rest api.

Answer 2

This sample worked for me, specifically: filesystem_client.create_directory(dir_name)