如何从 C# 核心中的 azure blob 存储读取所有文件

Question

I want to read files from an azure blob storage (the files inside the folder), the blob storage contains many folders. I want to read my folder 'blobstorage',it contains many JSON files performing.read to each file and some manipulations. I tried many code that did not work:

 CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
            CloudBlobContainer container = blobClient.GetContainerReference($"blobstorage");

The above code uses 'Microsoft.WindowsAzure.Storage' nuget package. This code is not working as expected. In many questions and answers found in stack overflow I found that most of them are outdated and does not work. Note: if any nuget mention that also bcs they are many packages

Answer 1

I found the solution in this post and worked perfectly for me. You just have to read it as a normal stream after the download.

BlobServiceClient blobServiceClient = new BlobServiceClient("connectionString");
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient("containerName");
BlobClient blobClient = containerClient.GetBlobClient("blobName.csv");
if (await blobClient.ExistsAsync())
{
  var response = await blobClient.DownloadAsync();
  using (var streamReader= new StreamReader(response.Value.Content))
  {
    while (!streamReader.EndOfStream)
    {
      var line = await streamReader.ReadLineAsync();
      Console.WriteLine(line);
    }
  }
}

Answer 2

I don't see any option to list all blob using Microsoft.WindowsAzure.Storage package. If you can use Azure.Storage.Blobs package then try below code.

using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using System;

namespace ConsoleApp2
{
    class Program
    {
        static string connectionString = "DefaultEndpointsProtocol=https;AccountName=storage******c9709;AccountKey=v**************************************;EndpointSuffix=core.windows.net";
        static string container = "azure-webjobs-hosts";
        static void Main(string[] args)
        {
            // Get a reference to a container named "sample-container" and then create it
            BlobContainerClient blobContainerClient = new BlobContainerClient(connectionString, container);
            blobContainerClient.CreateIfNotExists();
            Console.WriteLine("Listing blobs...");
            // List all blobs in the container
            var blobs = blobContainerClient.GetBlobs();
            foreach (BlobItem blobItem in blobs)
            {
                Console.WriteLine("\t" + blobItem.Name);
            }            
            Console.Read();
        }
    }
}

Output

You can also download the content of blob, Check this link

Answer 3

You can find example code in the SDK github repo here for c#: https://github.com/Azure/azure-sdk-for-net/tree/Azure.Storage.Blobs_12.8.0/sdk/storage/Azure.Storage.Blobs/

You can use the following command to add the package to your dotNet Core project.

dotnet add package Azure.Storage.Blobs

Based on the examples there, you can enumerate the blobs and then read the one you're looking for.

Answer 4

If you have mass data to download and are looking for efficiency, you probably don't want to download them 1 by 1 on a single thread. Use multiple threads and async.

Here's a good read on the topic:

https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-scalable-app-download-files?tabs=dotnet

downloading 1000 files:

• with single-thread: 30seconds download time
• with multi-thread: 4seconds download time

Answer 5

Try Below Code Out:

var connectionString = "your connection string";

        CloudStorageAccount storageacc = CloudStorageAccount.Parse(connectionString);
        //Create Reference to Azure Blob
        CloudBlobClient blobClient = storageacc.CreateCloudBlobClient();
        CloudBlobContainer container = blobClient.GetContainerReference("containerName");
        var blobs = container.GetDirectoryReference("FolderName").GetDirectoryReference("FolderName").ListBlobs().OfType<CloudBlockBlob>().ToList();
        Console.WriteLine("Total files found in directory: " + blobs.Count.ToString());
        var tempBlobList = blobs.Where(b => b.Name.Contains("fileName")).ToList();
        var response = await tempBlobList[0].DownloadTextAsync();