Why my Html to Excel conversion slows down using VBA over time when looping through files?

Question

I need to convert html files to excel from time to time. There are approximately 9000 html files that has tables on it.

I find it useful to convert them using excel 2007 vba, and made a macro to do the job, I have taken into account a bug of excel that halts the macro on Workbooks.Open function when SHIFT key is pressed, aside from that I disabled alerts, events and screen updating and make invisible the application as I don't want to interrupt me while I do something else.

'Declare API
Declare Function GetKeyState Lib "User32" _
(ByVal vKey As Integer) As Integer
Const SHIFT_KEY = 16

Function ShiftPressed() As Boolean
'Returns True if shift key is pressed
    ShiftPressed = GetKeyState(SHIFT_KEY) < 0
End Function

Sub ConvertHtmlToExcel()
    Dim wb As Workbook
    Dim strFile As String
    Dim strPath As String

    With Application
        .EnableEvents = False
        .DisplayAlerts = False
        .ScreenUpdating = False
        .Visible = False
    End With

    strPath = "c:\FolderToConvert\"
    strFile = Dir(strPath & "*.html")

    Do While strFile <> ""
        Do While ShiftPressed()
            DoEvents
        Loop
        Set wb = Workbooks.Open(strPath & strFile)
        strFile = Mid(strFile, 1, Len(strFile) - 5) & ".xls"
        wb.SaveAs strPath & strFile, XlFileFormat.xlWorkbookNormal
        wb.Close
        Set wb = Nothing
        strFile = Dir
    Loop

    With Application
        .EnableEvents = True
        .DisplayAlerts = True
        .ScreenUpdating = True
        .Visible = True
    End With
End Sub

The macro seems to be fine but when running I have the following conversion rate per minute:

1. 40
2. 31
3. 25
4. 21
5. 19
6. 18

And it keeps decreasing right now after 500 file converted its current rate is 8 per minute.

After 2359 files the rate decreased to 2 per minute, when testing I had visible = true and saw that it took more time to OPEN the workbook.

So the problem seem to be on Workbooks.Open that at the beginning of the loop works as fast as it can but in further loops it starts to slow down.

Has anyone stumbled on this? Are there any workaround? Is my code missing something? And sometimes the macro still halts the execution I assume that the shift key wasn't catched by that function.

Answer 1

There is no standard way to override the "shift bypass" function within Excel (in contrast to the "AllowByPassKey" property in Access). If you have problems with the code you applied (I haven't tried it), have you considered creating an add in?

http://msdn.microsoft.com/nl-nl/library/aa671739(v=vs.71).aspx On the Microsoft site is explicitly mentioned:

Several characteristics distinguish an Excel add-in from a typical workbook file:

• Users cannot use the SHIFT key to bypass events that are built into the add-in. This feature makes sure any event procedures you have written in the add-in will run at the proper time.

Regarding the speed: you seem to correctly allocate (.open) and release (set to nothing) memory for the "workbook" object in the loop so the memory it takes in the RAM should not increase or slow down during the processes.

This interesting thread could provide an explanation:

http://www.add-ins.com/support/out-of-memory-or-not-enough-resource-problem-with-microsoft-excel.htm

"The conclusion we reached from the above testing is that VBA add-ins do not use a significant amount of memory. Workbooks are the major user of memory. And Excel does not release all memory when workbooks are closed. Which supports one approach of closing and restarting Excel after doing a lot of work. You should close and re open Excel at least once every two hours."

Perhaps it is worth the try to each time set a new application and using application.quit, which clears the entire Excel application used from memory.

 Dim appExcel As Excel.Application

 Set appExcel = CreateObject("Excel.Application")
 appExcel.Workbooks.Add
 'Do smtg
 appExcel.Quit

Hope this works out for you! Interesting question!