简体繁体 English

将具有不同架构的多个Excel文件加载到SQL Server

[英]Load multiple Excel files with different schema to SQL Server

原文 2017-11-30 08:39:16 4 1 sql/ sql-server/ excel/ ssis/ etl

I have around 300 hundred Excel files with sales data, but different schema (one has a column named "Product Name" , another has only "Product" ) but contains the same information about sales from different shops. 我大约有300百万个包含销售数据的Excel文件，但是模式不同（一个模式具有名为“ Product Name”的列，另一个模式仅具有“ Product” ），但是包含有关来自不同商店的销售的相同信息。 The files are generated manually by some people so typos are also possible. 文件是由某些人手动生成的，因此也可能出现拼写错误。 Is there any nice way to import this data or I have to create 300 ETL packages in SSIS? 有什么好的方法可以导入此数据，或者我必须在SSIS中创建300个ETL包？

1 个解决方案

You can do this in these steps 您可以按照以下步骤进行操作

Get all excel files 获取所有Excel文件
For each file 对于每个文件
Script task to get column names and sheet names 脚本任务获取列名和工作表名
Store column names and sheet names in ssis variables 将列名称和工作表名称存储在ssis变量中

EDIT: You can't easily do anything about the typos, the easiest thing I can do out of the box is to have a dictionary table ready to go of expected values and a fuzzy match transformation to check against the dictionary table 编辑：您不能轻易地对错别字做任何事情，开箱即用，我能做的最简单的事情就是准备一个字典表以准备期望值，并进行模糊匹配转换以对照字典表进行检查

For each sheet 每张纸
SQL task to create stage table if not exists 创建阶段表（如果不存在）的SQL任务
Script task to read from excel sheet and insert into table 从Excel工作表读取并插入表的脚本任务

And it is easy to find how to read the column names of an excel file dynamically with c#. 而且很容易找到如何使用c＃动态读取excel文件的列名的方法。 I've done something similar with vb but below is an example of how to do with c#. 我已经用vb做过类似的事情，但以下是如何使用c＃的示例。

Sheet names 工作表名称

Using Excel OleDb to get sheet names IN SHEET ORDER 使用Excel OleDb在SHEET ORDER中获取工作表名称

Column names 列名

Get column names from excel file of a specific sheet using c# with OleDbConnection 使用带有OleDbConnection的c＃从特定工作表的excel文件中获取列名

Also this guy is loading all the files in dynamically with out of the box ssis excel dataflow 另外，这个家伙还使用开箱即用的sis excel数据流来动态加载所有文件。

https://social.msdn.microsoft.com/Forums/sqlserver/en-US/53f08f44-e996-48ca-9f72-b7dfc5d9136b/import-different-excel-file-with-different-schema-to-different-table-in-sql-server?forum=sqlintegrationservices https://social.msdn.microsoft.com/Forums/sqlserver/zh-CN/53f08f44-e996-48ca-9f72-b7dfc5d9136b/import-different-excel-file-with-different-schema-to-different-table- in-sql-server？forum = sqlintegrationservices

To create the table you will need to create the SQL statement to create table and then the SQL statement to insert into the table you created. 要创建表，您将需要创建SQL语句来创建表，然后创建SQL语句以插入到您创建的表中。