Arbitrary selects failing using cx_Oracle with instant client

Question

I am trying to select data from an ORACLE 12c database using cx_Oracle, but I am getting the exception: "cx_Oracle.OperationalError: ORA-03113: end-of-file on communication channel".

My query behaves fine using Pycharm (jdbc:oracle:thin driver). Using cx_Oracle in python 3.6, however, the query fails unless I reduce the number of IDs in the IN clause from 500 to about 250. The Cursor.fetchall() function is what throws the exception. I do not have privileged access to the database in order to check things like locks or load, but could these be the cause of the issue? According to our DBA, there is nothing wrong on the Oracle db server, and since the query works fine otherwise, I am inclined to believe it. I have messed with the client-side sqlnet.ora as well, which has allowed exceptions to eventually be thrown instead of hanging forever, but I still cannot fetch the data.

def select(self, query, *args):
    cur = self.dbh.cursor()
    cur.prepare(query)
    try:
        cur.execute(None, args)
        return cur.fetchall()
    # my attempt to handle the issue
    except (cx_Oracle.OperationalError, cx_Oracle.DatabaseError) as e:
        # cx_Oracle.OperationalError: ORA-03113: end-of-file on communication channel
        self.logger.error('Oracle Error: {}'.format(traceback.format_exc()))
        raise e

The code calls select like this. For brevity, I've omitted the full string IDs

ids = ['1', '2', '3', ...]
query = """\
select * from my_table where id in(:0,:1,:2,:3,:4, ...)
"""
self.select(query, *ids)

The query fails without the placeholders (with the IDs placed directly in the query) as well.

I expect to be able to run any select query using an IN clause with up to 1000 IDs without receiving the ORA-03113 Exception.

Edit: I installed oracle-instantclient18.5-basic-18.5.0.0.0-3.x86_64.rpm* on ubuntu 18.04.2, have cx_Oracle version 7.1.2, and I am connecting to Oracle 12.1.0.2.0.

The query is on the underlying tables of BMC Software's ARS. I will start working to try to replicate the problem with a local table structure, but it is a mess and will take some time. If I am able to create a local copy of the tables, I'm not sure I'd be able to replicate the issue, as identical queries with different IDs work fine. That makes it seem data driven, however, after I reduced the query to 250 IDs, I swapped the 250 from the first half to the second half, and got the same success result, so it doesn't seem to be just one bad row.

Is there more helpful logging I can enable on the client side to get more information?

Edit2: I should also add that the issue does not just occur with one query. I've seen the same issue with select queries to completely different tables.

Edit3: I just found out that by commenting out some of the columns that I'm selecting also can make the query work. columns like this:

to_char(to_date('1970-01-01','YYYY-MM-DD') + numtodsinterval(EventStart,'SECOND'),'YYYY-MM-DD HH24:MI:SS')

This may indicate that some kind of timeout is being reached which may or may not be configured in my sqlnet.ora:

DISABLE_OOB=on
SQLNET.RECV_TIMEOUT=60
SQLNET.SEND_TIMEOUT=60
TCP.CONNECT_TIMEOUT=300
SQLNET.OUTBOUND_CONNECT_TIMEOUT=300
ENABLE=BROKEN
TRACE_LEVEL_CLIENT=ADMIN
TRACE_FILE_CLIENT=sqlnet

Edit 4: I've tried some more things.

I installed the same version of instant client, except on a windows 7 machine, and ran the same query against the same db instance. The query succeeded.

I also narrowed down that for this particular query, it will accept 499 IDs, but fails with 500. it doesn't matter which ID I comment out from the query.

I also tried tricking the query into thinking there were fewer IDs by using a sub-select instead:

IN(
    select regexp_substr(:0,'[^,]+', 1, level) from dual connect by regexp_substr(:0, '[^,]+', 1, level) is not null
)

I got the error "cx_Oracle.DatabaseError: ORA-01460: unimplemented or unreasonable conversion requested", after which I realized made sense because Oracle will only allow a string to be up to 4000 bytes long.

Answer 1

I think I finally found a way to get everything functioning. I finally came across this link:

https://ardentperf.com/2010/09/08/mysterious-oracle-net-errors/

It turns out that this solved my problem. I am still having trouble getting cx_Oracle to honor a connection string of the same format as the tnsnames.ora file, but I changed my code to refer to the tnsnames.ora for now as follows:

connection_info = {
    'user': self.config.get(self.db, 'user'),
    'pass': self.config.get(self.db, 'password')
}
connection_string = '{user}/{pass}@TEST'\
    .format(**connection_info)
connection = cx_Oracle.connect(connection_string)

where my tnsnames.ora contains the following:

TEST =
 (DESCRIPTION =
   (ADDRESS_LIST =
     (ADDRESS = (PROTOCOL = TCP)(Host = myhost.com)(Port = 1521))
   )
   (SDU=1024)
 (CONNECT_DATA =
   (SID=mysid)
 )
)

The key here is the SDU=1024, which inexplicably fixes this issue.

https://docs.oracle.com/cd/B28359_01/network.111/b28317/sqlnet.htm#NETRF184

Documentation from the above link indicates that the default for SDU is 8192 bytes (8 KB) and my understanding is that there is supposed to be auto-negotiation of this value. This does not appear to be the case, and I don't know what past defaults have been.