使用 FME 或其他方法将多个 Excel 文件迁移到 SQL Server

Question

I have several Excel Files which I want to migrate to SQL Server 2017. I tried using SSIS 2017 (ForEachLoop) but I cannot get it to work (and I saw a previous answer referring me to an older version of SSIS). A work around for me is to convert the excel files to csv and import them on SSMS directly, one-by-one. I can also use FME to send "each" excel file as a table into SQL Server.

It will be very nice if I can somehow loop over the folder that has all the excel files (of different versions, 97-2003, 2013, etc.) and read them one-by-one into SQL Sever 2017, each as an individual table with its own name - just as if I import them one-by-one.

for example, abc.xlsx becomes dbo.abc, def.xlsx becomes dbo.def, ghi.xls becomes dbo.ghi, jkl.xls becomes dbo.jkl

Answer 1

Sorry. This is 8 month late.

This is not a solution to the stated problem, but I thought I share how I went around this task. Basically I used FME to import all excel files (by their specific names) into SQL Server. I did not write any code or loop through the files in the directory. I created my workflow by dragging all inputs on the canvas (GUI) and connected them all to their specific tables ( with their specific names ) on SQL Server.

I know this is not the efficient way of doing the task, but it worked!

Answer 2

This is a very interesting question! Well, this certainly sounds like a job for SSIS. I guess you tried that and it didn't work out, right. There are alternatives!

You can try this (I didn't try it).

https://www.mstsolutions.com/technical/importing-multiple-excel-files-into-sql-database/

Now, create a bunch of tables, dynamically, of course.

CREATE PROCEDURE sproc_BuildTable 
    @TableName NVARCHAR(128)
   ,@Column1Name NVARCHAR(32)
   ,@Column1DataType NVARCHAR(32)
   ,@Column1Nullable NVARCHAR(32)
   AS

   DECLARE @SQLString NVARCHAR(MAX)
   SET @SQLString = 'CREATE TABLE '+@TableName + '( '+ @Column1Name +' '+ @Column1DataType +' '+ @Column1Nullable +') ON PRIMARY '

   EXEC (@SQLString)
   GO

Then, you convert all those Excel files to CSV files, you can easily loop through each one and bulk insert all to separate tables in SQL Server.

DECLARE @intFlag varchar
SET @intFlag = 1
WHILE (@intFlag <=5)
BEGIN

PRINT @intFlag


declare @fullpath1 varchar(1000)
select @fullpath1 = '''C:\your_path_here\test\sample' + @intFlag + '.csv'''
print(@fullpath1)

declare @cmd1 nvarchar(1000)
select @cmd1 = 'bulk insert [dbo].[sample' + @intFlag + '] from' + @fullpath1 + ' with (FIELDTERMINATOR = '','', FIRSTROW = 2, ROWTERMINATOR=''\n'')'
print(@cmd1)

exec (@cmd1)


SET @intFlag = @intFlag + 1

END
GO

FYI, to convert all Excel files in a folder to CSV files, run the script below.

Sub WorkbooksSaveAsCsvToFolder()
'UpdatebyExtendoffice20181031
Dim xObjWB As Workbook
Dim xObjWS As Worksheet
Dim xStrEFPath As String
Dim xStrEFFile As String
Dim xObjFD As FileDialog
Dim xObjSFD As FileDialog
Dim xStrSPath As String
Dim xStrCSVFName As String

    Application.ScreenUpdating = False
    Application.EnableEvents = False
    Application.Calculation = xlCalculationManual
    On Error Resume Next
Set xObjFD = Application.FileDialog(msoFileDialogFolderPicker)
    xObjFD.AllowMultiSelect = False
    xObjFD.Title = "Kutools for Excel - Select a folder which contains Excel files"
    If xObjFD.Show <> -1 Then Exit Sub
    xStrEFPath = xObjFD.SelectedItems(1) & "\"

    Set xObjSFD = Application.FileDialog(msoFileDialogFolderPicker)

    xObjSFD.AllowMultiSelect = False
    xObjSFD.Title = "Kutools for Excel - Select a folder to locate CSV files"
    If xObjSFD.Show <> -1 Then Exit Sub
    xStrSPath = xObjSFD.SelectedItems(1) & "\"

    xStrEFFile = Dir(xStrEFPath & "*.xls*")

    Do While xStrEFFile <> ""
        Set xObjWB = Workbooks.Open(Filename:=xStrEFPath & xStrEFFile)
        xStrCSVFName = xStrSPath & Left(xStrEFFile, InStr(1, xStrEFFile, ".") - 1) & ".csv"
        xObjWB.SaveAs Filename:=xStrCSVFName, FileFormat:=xlCSV
        xObjWB.Close savechanges:=False
        xStrEFFile = Dir
  Loop
    Application.Calculation = xlCalculationAutomatic
    Application.EnableEvents = True
    Application.ScreenUpdating = True
End Sub

In the example above, I have 5 CSV files, so I am looping through the files in the folder 5x.

You could easily load all Excel files into separate tables in Access, and then export each Access table to a separate table in SQL Server.

Option1:
        Dim strPathFile As String, strFile As String, strPath As String
        Dim strTable As String
        Dim blnHasFieldNames As Boolean

        ' Change this next line to True if the first row in EXCEL worksheet
        ' has field names
        blnHasFieldNames = False

        ' Replace C:\Documents\ with the real path to the folder that
        ' contains the EXCEL files
        strPath = "C:\Documents\"

        ' Replace tablename with the real name of the table into which
        ' the data are to be imported
        strTable = "tablename"

        strFile = Dir(strPath & "*.xls")
        Do While Len(strFile) > 0
              strPathFile = strPath & strFile
              DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel9, _
                    strTable, strPathFile, blnHasFieldNames

        ' Uncomment out the next code step if you want to delete the
        ' EXCEL file after it's been imported
        '       Kill strPathFile

              strFile = Dir()
        Loop


Option2:
Dim blnHasFieldNames as Boolean
Dim strWorksheet As String, strTable As String
Dim strPath As String, strPathFile As String

' Change this next line to True if the first row in EXCEL worksheet
' has field names
blnHasFieldNames = False

' Replace C:\Documents\ with the real path to the folder that
' contains the EXCEL files
strPath = "C:\Documents\"

' Replace worksheetname with the real name of the worksheet that is to be
' imported from each file
strWorksheet = "worksheetname"

' Import the data from each workbook file in the folder
strFile = Dir(strPath & "*.xls")
Do While Len(strFile) > 0
      strPathFile = strPath & strFile
      strTable = "tbl_" & Left(strFile, InStrRev(strFile, ".xls") - 1)

      DoCmd.TransferSpreadsheet acImport, _
            acSpreadsheetTypeExcel9, strTable, strPathFile, _
            blnHasFieldNames, strWorksheet & "$"

      ' Uncomment out the next code step if you want to delete the
      ' EXCEL file after it's been imported
      ' Kill strPathFile

      strFile = Dir()
Loop


Option3:
        Dim blnHasFieldNames As Boolean, blnEXCEL As Boolean, blnReadOnly As Boolean
        Dim intWorkbookCounter As Integer
        Dim lngCount As Long
        Dim objExcel As Object, objWorkbook As Object
        Dim colWorksheets As Collection
        Dim strPath As String, strFile As String
        Dim strPassword As String

        ' Establish an EXCEL application object
        On Error Resume Next
        Set objExcel = GetObject(, "Excel.Application")
        If Err.Number <> 0 Then
              Set objExcel = CreateObject("Excel.Application")
              blnEXCEL = True
        End If
        Err.Clear
        On Error GoTo 0

        ' Change this next line to True if the first row in EXCEL worksheet
        ' has field names
        blnHasFieldNames = False

        ' Replace C:\MyFolder\ with the actual path to the folder that holds the EXCEL files
        strPath = "C:\MyFolder\"

        ' Replace passwordtext with the real password;
        ' if there is no password, replace it with vbNullString constant
        ' (e.g., strPassword = vbNullString)
        strPassword = "passwordtext"

        blnReadOnly = True ' open EXCEL file in read-only mode

        strFile = Dir(strPath & "*.xls")

        intWorkbookCounter = 0

        Do While strFile <> ""

              intWorkbookCounter = intWorkbookCounter + 1

              Set colWorksheets = New Collection

              Set objWorkbook = objExcel.Workbooks.Open(strPath & strFile, , _
                    blnReadOnly, , strPassword)

              For lngCount = 1 To objWorkbook.Worksheets.Count
                    colWorksheets.Add objWorkbook.Worksheets(lngCount).Name
              Next lngCount

              ' Close the EXCEL file without saving the file, and clean up the EXCEL objects
              objWorkbook.Close False
              Set objWorkbook = Nothing

              ' Import the data from each worksheet into a separate table
              For lngCount = colWorksheets.Count To 1 Step -1
                    DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel9, _
                          "tbl" & colWorksheets(lngCount) & intWorkbookCounter, _
                          strPath & strFile, blnHasFieldNames, _
                          colWorksheets(lngCount) & "$"
              Next lngCount

              ' Delete the collection
              Set colWorksheets = Nothing

              ' Uncomment out the next code step if you want to delete the
              ' EXCEL file after it's been imported
              ' Kill strPath & strFile

              strFile = Dir()

        Loop

        If blnEXCEL = True Then objExcel.Quit
        Set objExcel = Nothing

Just to round things off, please see the link below for more info.

https://www.red-gate.com/simple-talk/sql/t-sql-programming/questions-about-using-tsql-to-import-excel-data-you-were-too-shy-to-ask/