SQL Server目标与OLE DB目标

Question

I was using OLE Db destination for Bulk import of multiple Flat Files. After some tuning I ended up with SQL Server Destination to be 25 - 50 % faster.

Though I am confused about this destination as there are contradictory information on the web, some are against it, some are suggesting using it. I would like to know, are there any serious pitfalls before I deploy it to production? Thanks

Answer 1

In this answer, I will try to provide information from official documentation of SSIS and I will mention my personal experience with SQL Server destination.

1. SQL Server Destination

According to the official SQL Server Destination documentation :

The SQL Server destination connects to a local SQL Server database and bulk loads data into SQL Server tables and views. You cannot use the SQL Server destination in packages that access a SQL Server database on a remote server. Instead, the packages should use the OLE DB destination.

The SQL Server destination offers the same high-speed insertion of data into SQL Server that the Bulk Insert task provides; however, by using the SQL Server destination, a package can apply transformations to column data before the data is loaded into SQL Server.

For loading data into SQL Server, you should consider using the SQL Server destination instead of the OLE DB destination

2. OLEDB Destination

According to the official OLEDB Destination documentation :

OLEDB Destination - fast load option: Load data into a table or view in the OLE DB destination and use the fast load option, which are optimized for bulk inserts

3. OLEDB Destination vs SQL Server Destination

According to SQL Server Destination Vs OLE DB Destination - MSDN topic :

Donald Farmer, the former Group Program Manager for Integration Services said that you can get a 5 to 10% increase in performance using the SQL Server Destination .

In addition, refering to the following post of Matt Masson a data integration specialist at Microsoft where he answered the following question:

Should I use the SQL Server Destination?

The Answer was

No

...

My recommendation is that if you need every bit of performance (a 10% perf increase on a 10 hour load can be significant), try out the SQL Server Destination to see how it works for you. However – keep in mind the following limitations of the SQL Server Destination:

• You must have SSIS running on the same machine as the destination database
• You must run the package as an administrator
• It is very difficult to debug when things go wrong

Given these limitations, I recommend using the OLE DB Destination even if you are seeing a performance increase with the SQL Server Destination.

3.1. The Data Loading Performance Guide

(Update @ 2019-03-25)

While searching on SSIS best practices i found a very helpful Microsoft artcile that can be used as a reference:

• The Data Loading Performance Guide

In this article they made a comparison between all data loads methods including SQL Server destination and OLEDB destination, they mentioned that:

SQL Server Destination The SQL Server destination is the fastest way to bulk load data from an Integration Services data flow to SQL Server. This destination supports all the bulk load options of SQL Server – except ROWS_PER_BATCH.

Be aware that this destination requires shared memory connections to SQL Server. This means that it can only be used when Integration Services is running on the same physical computer as SQL Server.

OLE DB Destination: The OLE DB destination supports all of the bulk load options for SQL Server. However, to support ordered bulk load, some additional configuration is required. For more information, see “Sorted Input Data”. To use the bulk API, you have to configure this destination for “fast load”.

The OLE DB destination can use both TCP/IP and named pipes connections to SQL Server. This means that the OLE DB destination, unlike the SQL Server destination, can be run on a computer other than the bulk load target. Because Integration Services packages that use the OLE DB destination do not need to run on the SQL Server computer itself, you can scale out the ETL flow with workhorse servers.

3.2. Personal experience

(Update @ 2019-03-25)

Since this question is used as a reference by many, and after being more experienced in this domain, i added this section to mention my personal experience using SQL Server destination.

While official documentation mentioned that SQL Server destination will increase performance, i don't recommend at all using this components due to many reasons:

1. It requires that destination server and the ETL server are the same (works only with Local SQL server)
2. It always throw exception that don't have any meaning
3. After testing on huge volume of data the performance difference with OLEDB destination is negligible (tested on about 500 GB data loaded in chunks and the time difference is less than one minute)

You can also refer to the following post (from @billinkc) to get more information about this topic:

• Should SSIS packages and SQL database be on same server?

---

4. Conclusion

Based on Microsoft articles, you can say that SQL Server Destination increase the performance of inserting data (it uses BULK insert) , but it is designed for a specific case which is the Local SQL server. OLEDB Destination is more general and recommended in the other cases and by using the Fast Load data access mode (which uses also BULK insert) on the OLE DB destination it will increase the performance of data load.

On the other hand, based on my experience and from many articles written by SSIS experts, it is not recommended at all to use SQL Server Destination since it is not stable and it often throws exception and the performance can be considered as negligible.

---

Additional Information

Recently, I published a detailed article about this topic. You can check it at:

• SSIS OLE DB Destination Vs SQL Server Destination

Answer 2

To augment Hadi's fine answer, don't use the SQL Server Destination.

In my experience, the performance benefit does not outweigh the restriction that the package must be executed on the same machine as the destination database. It forces a processing architecture that may or may not be right for you today or a year from now. It's just too inflexible for my tastes.

The other, bigger reason I advocate avoiding the SQL Server Destination is the flat out bugginess I've experienced with it. Same flat file to an empty table- round 1, it aborts with a vague error message (can't recall specific) that something went wrong. Immediately restart the package and it works as expected. Maybe you, most humble reader, can accept that trade off in processing time for the reprocessing time but for me, it's not been worth it since probably 2008.