How to connect to an Oracle DB from a Python Azure Synapse notebook?

Question

I am trying to query an Oracle database from within an Azure Synapse notebook, preferably using Pyodbc but a pyspark solution would also be acceptable. The complexity here comes from, I believe, the low configurability of the spark pool - I believe the code is generally correct.

host = 'my_endpoint.com:[port here as plain numbers, e.g. 1111]/orcl'
database = 'my_db_name'
username = 'my_username'
password = 'my_password'

conn = pyodbc.connect( 'DRIVER={ODBC Driver 17 for SQL Server};'
'SERVER=' + host + ';'
'DATABASE=' + database + ';'
'UID=' + username + ';'
'PWD=' + password + ';')

Approaches I have tried:

Pyodbc - I can use the default driver available ({ODBC Driver 17 for SQL Server}) and I get login timeouts. I have tried both the normal URL of the server and the IP, and with all combinations of port, no port, comma port, colon port, and without the service name "orcl" appended. Code sample is above, but I believe the issue lies with the drivers.

Pyspark.read - With no JDBC driver specified, I get a "No suitable driver" error. I am able to add the OJDBC.jar to the workspace or to my file directory, but I was not able to figure out how to tell spark that it should be used.

cx_oracle - This is not permitted in my workspace.

If the solution requires setting environment variables or using spark-submit, please provide a link that explains how best to do that in Synapse. I would be happy with either a JDBC or an ODBC solution.

Answer 1

By adding the.jar here (ojdbc8-19.15.0.0.1.jar) to the Synapse workspace packages and then adding that package to the Apache spark pool packages, I was able to execute the following code:

host = 'my_host_url'
port = 1521
service_name = 'my_service_name'

jdbcUrl = f'jdbc:oracle:thin:@{host}:{port}:{service_name}'
sql = 'SELECT * FROM my_table'
user = 'my_username'
password = 'my_password'
jdbcDriver = 'oracle.jdbc.driver.OracleDriver'

jdbcDF = spark.read.format('jdbc') \
    .option('url', jdbcUrl) \
    .option('query', sql) \
    .option('user', user) \
    .option('password', password) \
    .option('driver', jdbcDriver) \
    .load()

display(jdbcDF)