Connecting to an Oracle database using SQLAlchemy

Question

I am able to successfully connect to a SQLite database and access a particular table using the set of commands below:

from sqlalchemy import create_engine, MetaData, Table, and_
from sqlalchemy.sql import select
from pandas import DataFrame 

db = create_engine('sqlite:///path\\database.db')
metadata = MetaData(db)
table = Table('table name', metadata, autoload=True)

I am able to fetch data from an Oracle database using cx_Oracle .

However, when I try to connect to an Oracle database in SQLAlchemy, I am getting the following error:

NoSuchTableError: <table name>

I have used the following commands:

db = create_engine('oracle://username:password@hostname:1521/instance name', echo='debug')
md = MetaData(bind=db)
t = Table('table name', md, autoload=True, schema='schema name')

When I use the following command:

t= Table('table name', md, autoload=True, oracle_resolve_synonyms=True)

I get the following error:

AssertionError: There are multiple tables visible to the schema, you must specify owner

Can you please explain where exactly I am going wrong?

Answer 1

You don't need to import cx_Oracle anymore. The newer version of the sqlalchemy module calls the function cx_Oracle.makedsn(). Have a look:

from sqlalchemy.engine import create_engine

DIALECT = 'oracle'
SQL_DRIVER = 'cx_oracle'
USERNAME = 'your_username' #enter your username
PASSWORD = 'your_password' #enter your password
HOST = 'subdomain.domain.tld' #enter the oracle db host url
PORT = 1521 # enter the oracle port number
SERVICE = 'your_oracle_service_name' # enter the oracle db service name
ENGINE_PATH_WIN_AUTH = DIALECT + '+' + SQL_DRIVER + '://' + USERNAME + ':' + PASSWORD +'@' + HOST + ':' + str(PORT) + '/?service_name=' + SERVICE

engine = create_engine(ENGINE_PATH_WIN_AUTH)


#test query
import pandas as pd
test_df = pd.read_sql_query('SELECT * FROM global_name', engine)

Answer 2

from sqlalchemy import create_engine
import cx_Oracle

host=hostname
port=port
sid='sid'
user='username'
password='password'
sid = cx_Oracle.makedsn(host, port, sid=sid)

cstr = 'oracle://{user}:{password}@{sid}'.format(
    user=user,
    password=password,
    sid=sid
)

engine =  create_engine(
    cstr,
    convert_unicode=False,
    pool_recycle=10,
    pool_size=50,
    echo=True
)

result = engine.execute('select * from TABLE')

for row in result:
    print row

This worked for me. A connection object can also be created like

conn = engine.connect()
conn.close()

which will enable to close the connection. This works even if you have a tunnel to your remote DB from your local port.

Answer 3

Assuming you have the Oracle client on your machine with a valid tnsnames.ora file, this works for me:

from sqlalchemy import create_engine
import pandas as pd 
engine = create_engine('oracle://myusername:mypassword@SID')
con = engine.connect()
outpt = con.execute("SELECT * FROM YOUR_TABLE")
df = pd.DataFrame(outpt.fetchall())
df.columns = outpt.keys()
print(df.head())
con.close() 

Answer 4

This works for me, when there is no tnsnames.ora file.

user = 'system'
pwd = 'oracle'
dsn = cx_Oracle.makedsn(
    '192.168.1.105', 49161,
    # service_name='your_service_name_if_any'
)
ora_engine = create_engine(f'oracle+cx_oracle://{user}:{pwd}@{dsn}', echo=True)
ora_engine.connect()

Answer 5

Here's a slighty polished version of @mkarun2's answer.

Prerequisites:

• Install packages:

  sudo apt install libaio1

• Download Oracle InstantClient library

• Make a symlink:

  ln -s libclntsh.so.21.1 libclntsh.so

• Run the app like this:

  export LD_LIBRARY_PATH=oracle-instantclient/instantclient_21_1 python main.py

Heres main.py :

import cx_Oracle
import sqlalchemy as sa


# Use your config
username = 'username'
password = 'security'
hostname = 'localhost'
service_name = 'Xal' or None
port = '1521'


# Tell cx_Oracle where to find the libraries. Optional.
cx_Oracle.init_oracle_client(lib_dir='oracle-instantclient/instantclient_21_1')


# Connection string
oracle_connection_string_fmt = (
    'oracle+cx_oracle://{username}:{password}@' +
    cx_Oracle.makedsn('{hostname}', '{port}', service_name='{service_name}')
)
url = oracle_connection_string_fmt.format(
    username=username, password=password, 
    hostname=hostname, port=port, 
    service_name=service_name,
)

# Create SqlAlchemy Engine
engine: sa.engine.Engine = sa.create_engine(url, echo=True)

Here's how to make your first query:

res: sa.engine.cursor.LegacyCursorResult = engine.execute("SELECT * FROM tablename")
for row in res:
    print(row)

notice that the query does not have ; at the end ;) Oracle does not like semicolons.

Answer 6

"... if you already have a connection string that works with cx_Oracle":

import pandas as pd
from sqlalchemy import create_engine
import cx_Oracle

conn_factory = lambda: conn_factory = lambda: cx_Oracle.connect(user=dbuser, password=dbpass, dsn=dsn)

engine = create_engine(
    "oracle://", 
    creator=conn_factory
)

data = pd.read_sql("SELECT 1 FROM DUAL", engine)

https://gist.github.com/DGrady/7fb5c2214f247dcff2cb5dd99e231483?permalink_comment_id=4035380#gistcomment-4035380