I've got an ADO NET Source component which queries a table for all of it's 700,000 records.
This is connected to a SQL Server Destination component.
it takes 23 seconds (+ - 1 second) to process the first 100,000 records.
The next 100,000 records take an additional 50 seconds. And it only gets worse.
Below are the specifics and details
I've already tried the following three different commands on the destination database.
ALTER DATABASE AdamDB SET RECOVERY SIMPLE;
ALTER DATABASE AdamDB SET RECOVERY BULK_LOGGED ;
ALTER DATABASE AdamDB SET RECOVERY FULL ;
In all 3 cases, the time remained at 23 seconds for the first 100,000 records.
I've also tried putting in a Conditional Split component based on the ID value
Default
SKU % 4 < 1
SKU % 4 < 2
SKU % 4 < 3
Each output condition has a different SQL Server Destination, and I've even gone so far as to create a different Connection Manager for each destination.
None of this seems to have had any noticeable affect on performance.
Here is the Source Connection Manager
Here are my source object properties.
And here is the connection manager for the destination
and here are my destination object properties.
How can I speed this up?
edit
From Lamak's suggestion, I've tried using an OLE DB Destination instead of a SQL Server Destination. This seems to have the same speed result and slowdown over time.
For those who may run into the same situation... it was a simple fix, after some very guided & helpful comments!
I ended up using an OLE DB Source and OLE DB Destination object instead of ADO objects.
Usage on just the destination, while the source remains ADO still gives slow results.
Usage of OLE DB on both source and destination fixed the issue.
I am now processing 750,000 records in about 55 seconds total!
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.