简体繁体 English

HDInsight Spark群集-无法连接到Azure Data Lake Store

[英]HDInsight Spark cluster - can't connect to Azure Data Lake Store

原文 2016-05-18 11:32:00 3 3 azure

So I have created an HDInsight Spark Cluster. 因此，我创建了一个HDInsight Spark群集。 I want it to access Azure Data Lake Store. 我希望它访问Azure Data Lake Store。

To create the HDInsight Spark cluster I followed the instructions at: https://azure.microsoft.com/en-gb/documentation/articles/data-lake-store-hdinsight-hadoop-use-portal however there was no option in the Azure Portal to configure the AAD or add a Service Principle. 要创建HDInsight Spark群集，我按照以下说明进行操作： https ://azure.microsoft.com/zh-cn/documentation/articles/data-lake-store-hdinsight-hadoop-use-portal，但是在Azure门户以配置AAD或添加服务原则。

So my cluster was created using Azure Blob Storage only. 因此，我的群集仅使用Azure Blob存储创建。 Now I want to extend it to access Azure Data Lake Store. 现在，我想扩展它以访问Azure Data Lake Store。 However the "Cluster AAD Identity" dialog states "Service Principal: DISABLED" and all fields in the dialog are greyed our and disabled. 但是，“群集AAD身份”对话框状态为“服务主体：已禁用”，对话框中的所有字段均为灰色且已禁用。 I can't see any way to extend the storage to point to ADL. 我看不到任何方法来扩展存储以指向ADL。

Any help would be appreciated! 任何帮助，将不胜感激！ Thanks :-) 谢谢：-）

3 个解决方案

您可以使用Data Factory将数据从Blob移动到ADLS，但不能从Spark集群直接访问ADLS。

Please create an Azure Hdinsight cluster with ServicePrincipal. 请使用ServicePrincipal创建一个Azure Hdinsight群集。 ServicePrincipal should have access to your data lake storage account. ServicePrincipal应该有权访问您的数据湖存储帐户。 You can configure your cluster to use Data lake storage but that is very complicated. 您可以将集群配置为使用Data Lake存储，但这非常复杂。 And in fact there is no documentation for that. 实际上，没有任何文档。 So recommended way to create is with ServicePrincipal. 因此，推荐的创建方法是使用ServicePrincipal。

Which type of cluster did you create? 您创建了哪种类型的集群？ In our Linux cluster all the option listed in the guide you linked are available. 在我们的Linux集群中，您链接的指南中列出的所有选项均可用。