Fail to create in-demand hadoop cluster in Azure Data Factory; additionalProperties is not active

Question

It's my first time trying out the Azure data factory so I hope this is not a bad question to ask.

So I'm using the Azure portal trying to create an on-demand hadoop cluster as one of the linked service in Azure Data Factory following the steps in the tutorial .

But whenever I click create, the following error message pops up.

Failed to save HDinisghtLinkedService. Error: An additional property 'subnetName' has been specified but additionalProperties is not active.The relevant property is 'HDInsightOnDemandLinkedServiceTypeProperties'.The error occurred at the location 'body/properties/typeProperties' in the request.;An additional property 'virtualNetworkId' has been specified but additionalProperties is not active.The relevant property is 'HDInsightOnDemandLinkedServiceTypeProperties'.The error occurred at the location 'body/properties/typeProperties' in the request.

I couldn't understand why it requires the 'subnetName' and 'virtualNetworkId'. But I tried putting values under Advanced Properties -> Chose Vnet and Subnet -> From Azure subscription -> and put in the existing vitrual network ID and subnet name . But the problem still present and the same error message shows up.

Other background information:

For the tutorial I posted above, I did not use its powershell code. I have existing resource group and created a new storage account on the Azure portal. I also created a new app registration in Azure Active Directory and retrieve principal service application ID and authentication key following this link

Some parameters:

• Type: On-demand HDInsight
• Azure Storage Linked Service: the one listed in the connection
• Cluster size: 1 (for testing)
• Service principal id/service principal key: described above
• Version: 3.6 ...

Any thoughts or anything I might be doing wrong?

Answer 1

From the error message, it clearly states that “subnetName” is not active, which means it has not created at all.

Note: If you want to create on-demand cluster within your Vnet, then first create Vnet and Subnet and the pass the following values.

Advanced Properties are not mandatory to create a on-demand cluster.

Have you tried created on-demand cluster without passing the Vnet and Subnet?

Hope this helps. Do let us know if you any further queries.