简体   繁体   中英

SSIS DataFlow slows down considerably over time

I've got an ADO NET Source component which queries a table for all of it's 700,000 records.

This is connected to a SQL Server Destination component.

it takes 23 seconds (+ - 1 second) to process the first 100,000 records.

The next 100,000 records take an additional 50 seconds. And it only gets worse.

Below are the specifics and details


I've already tried the following three different commands on the destination database.

ALTER DATABASE AdamDB SET RECOVERY SIMPLE;

ALTER DATABASE AdamDB SET RECOVERY BULK_LOGGED ;

ALTER DATABASE AdamDB SET RECOVERY FULL ;

In all 3 cases, the time remained at 23 seconds for the first 100,000 records.


I've also tried putting in a Conditional Split component based on the ID value

Default
SKU % 4 < 1
SKU % 4 < 2
SKU % 4 < 3

Each output condition has a different SQL Server Destination, and I've even gone so far as to create a different Connection Manager for each destination.

有条件拆分


None of this seems to have had any noticeable affect on performance.

Here is the Source Connection Manager

源连接管理器

Here are my source object properties.

源对象属性

And here is the connection manager for the destination

目标连接管理器

and here are my destination object properties.

目标连接属性



How can I speed this up?

edit

From Lamak's suggestion, I've tried using an OLE DB Destination instead of a SQL Server Destination. This seems to have the same speed result and slowdown over time.

For those who may run into the same situation... it was a simple fix, after some very guided & helpful comments!

I ended up using an OLE DB Source and OLE DB Destination object instead of ADO objects.

Usage on just the destination, while the source remains ADO still gives slow results.

Usage of OLE DB on both source and destination fixed the issue.

I am now processing 750,000 records in about 55 seconds total!

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM