AADToken: HTTP connection to https://login.microsoftonline.com/<tenantID>/oauth2/token failed for getting token from AzureAD
Question
I want to get access to Azure Data Lake Storage Gen2 from Azure Databricks Cluster - SCALA version, via mount point in filesystem.
I tried the following code where azure service principal credentials are designated as entry point to azure subscription (role --> storage blob data owner ON Data Lake containers).
val fileSystemName = "XXXXXXXXXX"
val storageAccountName = "XXXXXXXXXXXX"
val appID = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
val appSecret = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
val tenantID = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
val configs = Map(
"fs.azure.account.auth.type" -> "OAuth",
"fs.azure.account.oauth.provider.type" -> "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
"fs.azure.account.oauth2.client.id" -> "<appID>",
"fs.azure.account.oauth2.client.secret" -> "<appSecret>",
"fs.azure.account.oauth2.client.endpoint" -> "https://login.microsoftonline.com/<tenantID>/oauth2/token")
// Optionally, you can add <directory-name> to the source URI of your mount point.
dbutils.fs.mount(
source = "abfss://<fileSystemName>@<storageAccountName>.dfs.core.windows.net/",
mountPoint = "/mnt/raw-container",
extraConfigs = configs)
But get this error message :
shaded.databricks.v20180920_b33d810.org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator$HttpException: HTTP Error 400; url='https://login.microsoftonline.com/<tenantID>/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/<tenantID>/oauth2/token failed for getting token from AzureAD.; requestId=''; contentType='text/html; charset=us-ascii'; response '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>
'
at shaded.databricks.v20180920_b33d810.org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenSingleCall(AzureADAuthenticator.java:372)
at shaded.databricks.v20180920_b33d810.org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenCall(AzureADAuthenticator.java:270)
at shaded.databricks.v20180920_b33d810.org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenUsingClientCreds(AzureADAuthenticator.java:95)
at com.databricks.backend.daemon.dbutils.DBUtilsCore.verifyAzureOAuth(DBUtilsCore.scala:477)
at com.databricks.backend.daemon.dbutils.DBUtilsCore.verifyAzureFileSystem(DBUtilsCore.scala:488)
at com.databricks.backend.daemon.dbutils.DBUtilsCore.mount(DBUtilsCore.scala:446)
at com.databricks.dbutils_v1.impl.DbfsUtilsImpl.mount(DbfsUtilsImpl.scala:85)
at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(command-3300122855614219:18)
at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw$$iw$$iw$$iw$$iw.<init>(command-3300122855614219:67)
at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw$$iw$$iw$$iw.<init>(command-3300122855614219:69)
at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw$$iw$$iw.<init>(command-3300122855614219:71)
at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw$$iw.<init>(command-3300122855614219:73)
at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw.<init>(command-3300122855614219:75)
at linec1d28633ab4c4f2d9530ae7396d8282327.$read.<init>(command-3300122855614219:77)
at linec1d28633ab4c4f2d9530ae7396d8282327.$read$.<init>(command-3300122855614219:81)
at linec1d28633ab4c4f2d9530ae7396d8282327.$read$.<clinit>(command-3300122855614219)
at linec1d28633ab4c4f2d9530ae7396d8282327.$eval$.$print$lzycompute(<notebook>:7)
at linec1d28633ab4c4f2d9530ae7396d8282327.$eval$.$print(<notebook>:6)
at linec1d28633ab4c4f2d9530ae7396d8282327.$eval.$print(<notebook>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:745)
at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1021)
at scala.tools.nsc.interpreter.IMain.$anonfun$interpret$1(IMain.scala:574)
at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:41)
at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:37)
at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:41)
at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:573)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:600)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:570)
at com.databricks.backend.daemon.driver.DriverILoop.execute(DriverILoop.scala:215)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.$anonfun$repl$1(ScalaDriverLocal.scala:202)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExitInternal$.trapExit(DriverLocal.scala:714)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExit$.apply(DriverLocal.scala:667)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.repl(ScalaDriverLocal.scala:202)
at com.databricks.backend.daemon.driver.DriverLocal.$anonfun$execute$10(DriverLocal.scala:396)
at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext$1(UsageLogging.scala:238)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:233)
at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:230)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionContext(DriverLocal.scala:49)
at com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:275)
at com.databricks.logging.UsageLogging.withAttributionTags$(UsageLogging.scala:268)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionTags(DriverLocal.scala:49)
at com.databricks.backend.daemon.driver.DriverLocal.execute(DriverLocal.scala:373)
at com.databricks.backend.daemon.driver.DriverWrapper.$anonfun$tryExecutingCommand$1(DriverWrapper.scala:653)
at scala.util.Try$.apply(Try.scala:213)
at com.databricks.backend.daemon.driver.DriverWrapper.tryExecutingCommand(DriverWrapper.scala:645)
at com.databricks.backend.daemon.driver.DriverWrapper.getCommandOutputAndError(DriverWrapper.scala:486)
at com.databricks.backend.daemon.driver.DriverWrapper.executeCommand(DriverWrapper.scala:598)
at com.databricks.backend.daemon.driver.DriverWrapper.runInnerLoop(DriverWrapper.scala:391)
at com.databricks.backend.daemon.driver.DriverWrapper.runInner(DriverWrapper.scala:337)
at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:219)
at java.lang.Thread.run(Thread.java:748)
It seems there is an endpoint authentication issue.
If the question helped, up-vote it. Thanks in advance.
2 answers
solution1
0 ACCPTED 2020-08-26 18:33:31
I tried the problem differently and it works....
I provisioned an azure databricks PREMIUM version
Then, at Python cluster configuration page I checked "Enable credential passthrough for user-level data access" option under "Azure Data Lake Storage Credential Passthrough" at "Advanced Options" section.
So I only have to pass the moint point script in a cell and it works :
storageaccountName = "XXXXXXXXXXXX"
storagecontainerName = "XXXXXXXXXXXX"
configs = {
"fs.azure.account.auth.type": "CustomAccessToken",
"fs.azure.account.custom.token.provider.class": spark.conf.get("spark.databricks.passthrough.adls.gen2.tokenProviderClassName")
}
# Optionally, you can add <directory-name> to the source URI of your mount point.
dbutils.fs.mount(
source = "abfss://"+storagecontainerName+"@"+storageaccountName+".dfs.core.windows.net/",
mount_point = "/mnt/raw-container",
extra_configs = configs)
solution2
0 2021-11-21 00:40:24
Please be sure that you have well copied the tenant_id not the Object ID,
there are some thing called the tenant_id.
I did that mistakes
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
Claim type roles is missing in the access token when a HTTP POST call is made to https://login.microsoftonline.com/{{TenantID}}/oauth2/token
Not getting access_token by calling https://login.microsoftonline.com/{tenant}/oauth2/token
CORS issue while trying to access https://login.microsoftonline.com/common/oauth2/v2.0/token' to "acquireTokenOnBehalfOf" from localhost & app domains
CORS Exception : Failed to load https://login.microsoftonline.com/{TENANT_ID}/oauth2/authorize
Customize domain with my own domain in Azure AD b2b https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token
Hadoop client Azure Data Lake Gen2 Token Acess - AADToken: HTTP connection failed for getting token from AzureAD. Http response: 400 Bad Request
The OAuth2 authentication request process of Office 365 forwards the link to https://login.microsoftonline.com/login.srf
Is it possible to call login.microsoftonline.com from javascript to get Bearer Token without using node.js
Azure AD Cannot hit login.microsoftonline.com/{tenantId}
Issuer in access token from azure active directory is https://sts.windows.net when I'm expecting https://login.microsoftonline.com
Related Tags