SSIS incomplete data load from Oracle to SQL Server

Question

I have a SSIS DFT that decided to break that's responsible for moving data from Oracle 12c to SQL Server 2016 with no intermediary task - it just executes a fairly simple query using a date range against Oracle and inserts to a staging table on SQL Server with fast load. On days when records exceed ~2.5 million, the task will report success after writing a number of records perfectly divisible by the buffer size to SQL Server, despite verifying ~3 million records expected from Oracle.

The Oracle source is using the 32-bit Oracle OLE DB Provider and SQL Server is using the 32-bit MSOLEDBSQL provider. I am currently unable to test 64-bit drivers to see if it corrects the issue. The DFT is currently setup with AutoAdjustBufferSize to true with DefaultBufferMaxRows at 50k. The staging table is setup with all cols defined as VARCHAR , even though the source columns are not, so there are some implicit conversions happening. The package is executed via a scheduling server running TWS, which just points to a bat script containing parameter values that get passed to the DTEXEC util.

I've tried testing changes to AutoAdjustBufferSize , DefaultBufferMaxRows , Maximum Insert Commit Size, Rows Per Batch, and appending to the Oracle connection string UseSessionFormat=True (some similar issues had success doing this). Nothing so far has resolved the issue.

Has anyone else encountered this issue and found a resolution plus what the underlying problem was?

Answer 1

I managed to find a solution for the problem. The Oracle OLE DB provider has an optional connection string attribute called FetchSize . This value is 100 by default and can be modified by a reg key entry or by modifying the Oracle connection string. It affects how many rows are fetched and impacts round trips to the database. In my case, I modified the connection string with a XML config file and set the FetchSize value to 5,000. Increasing the FetchSize allowed the DFT to move all records.

More info on the Oracle OLE DB Provider connection string attributes can be found here.