如何解析非常大的 Excel 文件（>6 GB）并在 VBScript 中解析？

Question

I have a very Large Excel file (>6 GB) and I need to Parse it in VBScript.

Function LineCount(sFName As String) As Long
Const Bloc = 32 * 1024& '-- 32K bloc
Dim n       As Long
Dim sText   As String
Dim LfCount As Long

Dim t As Single:  t = Timer '-- simple timing

Open sFName For Input As #1
If LOF(1) = 0 Then Close #1: Exit Function
n = LOF(1) Mod Bloc
If n = 0 Then n = Bloc
LineCount = 1
sText = Input(n, #1)
Do
    '-- short code: --------------------------------
    'LineCount = LineCount + UBound(Split(sText, vbLf))
    '-----------------------------------------------
    '-- longer code: ~10% faster -------------------
    n = -1
    Do
        n = InStrB(n + 2, sText, vbLf)
        If n Then LineCount = LineCount + 1 Else Exit Do
    Loop
    '-----------------------------------------------
    If EOF(1) Then Exit Do
    sText = Input(Bloc, #1)
Loop
Close #1
'-- subtract blank line at the bottom of the file
If Right(sText, 1) = vbLf Then LineCount = LineCount - 1

Debug.Print LineCount, Timer - t
End Function

 

I need to be able to read very large Excel file and create 1,000,000 line excel files off of it. Any suggestions for fast way to read without running into runtime errors of no memory left?

Answer 1

You'd open the large file for input, reading a line at a time, then open a succession of other files for output, writing 1M lines to each one before opening the next, and so on.

Scaled-down version:

Sub SplitTextFile()
    
    Dim fso As Object, t, n As Long, ln, t2 As Object, outNum As Long
    Set fso = CreateObject("scripting.filesystemobject")
    
    'create a dummy text file for testing
    Set t = fso.createtextfile("C:\Temp\dummy.txt")
    For n = 1 To 1000
        t.writeline "This is line " & n
    Next n
    t.Close

    Set t = fso.opentextfile("C:\Temp\dummy.txt")
    
    n = 0
    outNum = 0
    Do While Not t.atendofstream
        
        If n Mod 100 = 0 Then
            If Not t2 Is Nothing Then t2.Close
            outNum = outNum + 1
            Debug.Print "Writing file # " & outNum
            Set t2 = fso.createtextfile("C:\Temp\dummy_" & outNum & ".txt", 2)
        End If
        
        t2.writeline t.readline
        n = n + 1
    Loop
    
    t.Close
    t2.Close

End Sub