无法使用azure-activedirectory-library-for-java库使用Java在Spark YARN群集中连接Azure SQL DB

Question

In one of our project, we are using Azure SQL DB, authentication mode is Access Token based. We are using Spark Java to connect. We are able to connect to DB when we run our spark job as standalone but when we are running in YARN mode, we are not able to connect. We are using the library “azure-activedirectory-library-for-java”

We are getting the error as: Login failed for user: ''. ClientConnectionId:

Exception: INFO Client: client token: N/A diagnostics: User class threw exception: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user ''. ClientConnectionId:2b288e9e-15cc-448a-8210-0d184d27a12b ApplicationMaster host: 172.25.3.7 ApplicationMaster RPC port: 0 queue: default start time: 1538583454649 final status: FAILED tracking URL:

References:

https://github.com/Azure/azure-sqldb-spark

https://docs.microsoft.com/en-us/azure/sql-database/sql-database-spark-connector

Thanks in advance.

Thanks & Regards,

-Venkat

Below is the exception stacktrace:

18/10/08 10:07:16 WARN SparkSession$Builder: Using an existing SparkSession; some configuration may not take effect. 18/10/08 10:07:16 ERROR ApplicationMaster: User class threw exception: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user ''. ClientConnectionId:8ecbc2ae-cede-4b1d-8eba-f362010f71e6 com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user ''. ClientConnectionId:8ecbc2ae-cede-4b1d-8eba-f362010f71e6 at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:217) at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:251) at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:81) at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:3077) at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:2360) at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$100(SQLServerConnection.java:43) at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:2346) at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:6276) at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1793) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1404) at com.microsoft.sqlserver.jdbc.SQLServerConnection.log in(SQLServerConnection.java:1068) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:904) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:451) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1014) at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:61) at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:52) at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:58) at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation.(JDBCRelation.scala:114) at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:52) at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:309) at org.apache.spark.sql.DataFrameReader.load(DataFrameRead er.scala:178) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:146) at org.apache.spark.sql.DataFrameReader.jdbc(DataFrameReader.scala:193) at com.microsoft.azure.sqldb.spark.connect.DataFrameReaderFunctions.sqlDB(DataFrameReaderFunctions.scala:44) at com.sew.data.SQLDBConnectionEngine$.main(SQLDBConnectionEngine.scala:71) at com.sew.data.SQLDBConnectionEngine.main(SQLDBConnectionEngine.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:646) 18/10/08 10:07:16 INFO ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user ''. ClientConnectionId:8ecbc2ae-cede-4b1d-8eba-f362010f71e6) 18/10/08 10:07:16 INFO SparkContext: Invoking stop() from shutdown hook 18/10/08 10:07:16 INFO AbstractConnector: Stopped Spark@683861d4{HTTP/1.1,[http/1.1]}{0.0.0.0:0}

Answer 1

The issue is Login failed for user ''. and I don't see a user name that the login is attempting to use?

18/10/08 10:07:16 ERROR ApplicationMaster: User class threw exception: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user ''. ClientConnectionId:8ecbc2ae-cede-4b1d-8eba-f362010f71e6 com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user ''. ClientConnectionId:8ecbc2ae-cede-4b1d-8eba-f362010f71e6 at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError

What user are you attempting to connect as? Has this identify been added as a 'contained database user' per the instructions for creating Azure Active Directory:

Create and populate an Azure AD

If the above set of steps have been complete properly, you then move on to:

Connecting using Access Token

Setup Requirement

If you are using the access token-based authentication mode, you need to download azure-activedirectory-library-for-java and its dependencies, and include them in the Java build path.

See Use Azure Active Directory Authentication for authentication with SQL Database to learn how to get access token to your Azure SQL database.