Read CSV From Azure Data lake storage Gen 1 in c# .net API

Question

我们需要读取一个大约2 GB的 CSV 文件，该文件存储在Azure 数据湖存储 Gen1中。目的就像我们必须在用户请求时以高性能的网格格式 (UI) 呈现数据。 我们使用.Net Core 2.1 (c#)来做同样的事情 API。

        var creds = new ClientCredential(applicationId, clientSecret);
        var clientCreds = ApplicationTokenProvider.LoginSilentAsync(tenantId, creds).GetAwaiter().GetResult();

        // Create ADLS client object
        AdlsClient client = AdlsClient.CreateClient(adlsAccountFQDN, clientCreds);

        string fileName = "/cchbc/sources/MVP/Data.csv";

        using (var readStream = new StreamReader(client.GetReadStream(fileName)))
        {

            while ((line = readStream.ReadLine()) != null)
            {
                content = content + line;
            }
        }

我已经尝试了上面的代码，但失败并出现错误GETFILESTATUS failed with HttpStatus:Forbidden RemoteException: AccessControlException GETFILESTATUS failed with error 0x83090aa2 (Forbidden. ACL 验证失败。资源不存在或用户无权执行请求的操作。 )

任何建议都会非常有益。在此先感谢

Answer 1

如果要使用服务主体访问存储在 Azure 数据湖 gen 1 中的文件，我们需要为服务主体配置ACL 。 ACL 具有三个权限读取（读取文件的内容）写入（写入或 append 到文件）和执行（遍历文件夹的子项）。

例如我访问文件 /test/test.csv

1. 如下配置ACL

Opreation    Object        /        test/      test.csv
Read         tets.csv      --X      --X        R-- 

2. 代码

string appId = "service principal appId";
            string appSecret = "service principal appSecret";
            string domain = "service principal domain";

            var serviceSettings = ActiveDirectoryServiceSettings.Azure;
            serviceSettings.TokenAudience = new Uri(@"https://datalake.azure.net/");
            
            var creds = await ApplicationTokenProvider.LoginSilentAsync(domain, appId, appSecret, serviceSettings);

            string accountName = "testadls02";
            AdlsClient client = AdlsClient.CreateClient($"{accountName}.azuredatalakestore.net", creds);
            string fileName = "/test/test.csv";
            string line = null;
            using (var readStream = new StreamReader(client.GetReadStream(fileName)))
            {
                while ((line =  await readStream.ReadLineAsync()) != null) {

                    Console.WriteLine(line);
                }
               
            }

更多详情，请参考这里