How to create Azure HDInsight cluster with Hive metastore using Powershell?

Question

I was trying to create HDInsight cluster with Hive metastore. Cluster was created successfully but without Hive metastore. Also any error was not returned. I used Powershell script below.

$resourceGroupName = "ResourceGroup"
$storageAccountName = "myStorage"
$containerName="myContainer"

$clusterName = "myCluster"
$location = "West Europe"
$clusterNodes = 2
$clusterType = "Hadoop"
$OSType = "Windows"
$hdVersion = "3.2"

$hadoopUserName = "user"
$hadoopUserPassword = "password"
$hadoopUserPW = ConvertTo-SecureString -String $hadoopUserPassword -AsPlainText -Force
$clusterCreds = New-Object System.Management.Automation.PSCredential($hadoopUserName,$hadoopUserPW)

$sqlDatabaseServerName = "mydbserver"
$metaStoreDBName = "HiveMetaStoreDB"
$sqlDatabaseLogin = "myDBLogin"
$sqlDatabasePassword = "myDBPassword"
$sqlDatabaseSecurePassword = ConvertTo-SecureString -String $sqlDatabasePassword -AsPlainText -Force
$sqlServerCred = New-Object System.Management.Automation.PSCredential ($sqlDatabaseLogin, $sqlDatabaseSecurePassword)


# Create a new HDInsight cluster

    $config = New-AzureRmHDInsightClusterConfig -ClusterType Hadoop `
        | Add-AzureRmHDInsightMetastore `
            -SqlAzureServerName "$sqlDatabaseServerName.database.windows.net" `
            -DatabaseName $metaStoreDBName `
            -Credential $sqlServerCred `
            -MetastoreType HiveMetastore 

$config.DefaultStorageAccountName = "$storageAccountName.blob.core.windows.net"
$config.DefaultStorageAccountKey = $storageAccountKey           

New-AzureRmHDInsightCluster `
        -config $config `
        -OSType $OSType `
        -clustername $clusterName `
        -HttpCredential $clusterCreds `
        -DefaultStorageContainer $containerName `
        -Location $location `
        -ResourceGroupName $resourceGroupName `
        -ClusterSizeInNodes $clusterNodes `
        -Version $hdVersion             

Could somebody tell me the reason why Hive metastore was not created? Or what should I do to create it?

Answer 1

The cluster creation process doesn't create the Azure Database for you. You must create the database before you can use it as the Hive metastore.