Import Big Data from a text file to Excel

Question

I have the next problem, I need to compare an excel file with 127,000 rows approximately with a text file with 1'500,000 lines approximately. Every line in the text file is divided by eight pipes, having 8 columns in excel. I need to do a VLookUp with the fifth column in the text line with certain column in my excel file.

What I've been doing is:

1. I take the first row in my excel file, I look for it in the text file and I import only that line, then I fit the line in their respective columns in excel and at the end I do the VLookUp.

---

Sub test()
Dim textline As String, rw As Long, lRow As Long, rw1 As Long
Dim sPath As String, text As Variant

rw1 = 1
sPath = "My_path"

With ActiveWorkbook.Sheets("Sheet1")
   For rw = 2 To .Cells(Rows.Count, 1).End(xlUp).Row
       text = range("A" & rw).Value2
       Open sPath For Input As #1
       Do Until EOF(1)
           Line Input #1, textline
             If InStr(textline, CStr(text)) Then
               Sheets(2).Cells(rw1, 1) = textline
               Sheets(2).range("A" & rw1).TextToColumns Destination:=range("A" & rw1 & ":H" & rw1), DataType:=xlDelimited, _
                   TextQualifier:=xlDoubleQuote, ConsecutiveDelimiter:=False, Tab:=False, _
                   Semicolon:=False, Comma:=False, Space:=False, Other:=True, OtherChar _
                   :="|", TrailingMinusNumbers:=True
               rw1 = rw1 + 1
               Exit Do
           End If
       Loop
       Close #1
   Next rw
End With
End Sub

2. I import all the text file to excel, then I fit the line in their respective columns and at the end I do the VLookUp.

---

Sub test()
Dim textline As String, rw As Long, lRow As Long, rw1 As Long
Dim sPath As String, text As Variant, sh As String, c As Integer

sh = "Sheet"
c = 1
rw = 1
rw1 = 1
sPath = "My_path"


With ActiveWorkbook.Sheets("Sheet1")
       Open sPath For Input As #1
       Do Until EOF(1)
           Line Input #1, textline
             Sheets("Sheet1").range("A" & rw) = textline
               Exit Do
             End If
             rw = rw + 1
       Loop
       Close #1
End With

With ActiveWorkbook.Sheets(1)
   For rw1 = 1 To .Cells(Rows.Count, 1).End(xlUp).Row
       Sheets("Sheet1").range("A" & rw1).TextToColumns Destination:=range("A" & rw1 & ":H" & rw1), DataType:=xlDelimited, _
           TextQualifier:=xlDoubleQuote, ConsecutiveDelimiter:=False, Tab:=False, _
           Semicolon:=False, Comma:=False, Space:=False, Other:=True, OtherChar _
           :="|", TrailingMinusNumbers:=True
   Next rw1
End With
End Sub
Now

The problem is that the two methods takes too long, Does anyone know a better solution that can help me with this?

Thank you in advice

Answer 1

由于您超出了正常的行数限制，如果您设置使用Excel，建议您使用Get＆Transform （对于Excel 2016）或Power Query （对于Excel 2010和2013）。