Excel VBA speed up Vlookup with 2 conditions over a large range

Question

I have a sample of some VBA that lookup a concatenation of 2 columns. This looksup a database feed, with a row count from between 35k and 250k.

Doing vlookups is way too slow with times from 60 to 500 seconds. What would be the most efficient way to get the the same result.

Sequence

• Turns of screen updating
• Turns off all calculations
• Disables database
• Clears clipboard cache
• Refreshes db data
• Sets the lookups
• Turns on calculations
• Turns off calculations
• Copy and pastes values of the vlookups.
• Enables database
• Turns everything back on

S

 Sub startcom()
  Dim ii As Long, lastrow As Long
  Dim StartTime As Double
  Dim SecondsElapsed As Double

' starts timer
     StartTime = Timer

 'freeze screens, clears cache and stops cals
    stopall

'Set error traps and start and end times
     On Error GoTo errortrap:


Set sht1 = wsRag
Set sht2 = wsComdata

sht2.Select
    reflist

'Find the last row (in column A) with data. and set start row for data copy
   lastrow = sht1.Range("A:A").Find("*", SearchDirection:=xlPrevious).Row
ii = 9

'disables db connection
  wsConfig.Cells(7, 2) = 0

sht1.Select

 Range("AM" & ii & ":AM" & lastrow).Formula = "=IF(VLOOKUP(CONCATENATE(A"& ii &",B" & ii &"),Comment_data!A:F,4,0)="""","""",VLOOKUP(CONCATENATE(A" & ii   & ",B" & ii & "),Comment_data!A:F,4,0))" ' Get comments
    calcon
    calcoff
    Range("AM" & ii & ":AM" & lastrow & "").Select
        Selection.Copy
            Selection.PasteSpecial Paste:=xlPasteValues, Operation:=xlNone, SkipBlanks _
            :=False, Transpose:=False

'enable DB connection
    wsConfig.Cells(7, 2) = 1
'Determine how many seconds code took to run
  SecondsElapsed = Round(Timer - StartTime, 2)

'get lenghth of runtime
Debug.Print "Ran successfully in " & SecondsElapsed & " seconds",    vbInformation
startall
  Exit Sub
   errortrap:
     errormess
 Debug.Print "Location: Comments start"
End Sub

Answer 1

The issue

As I understand your issue lies with the VLOOKUP operations , this bit here (spread over a couple of rows to make it more readable):

Range("AM" & ii & ":AM" & lastrow).Formula =
    "=IF( 
        VLOOKUP(CONCATENATE(A"& ii &",B" & ii &"),Comment_data!A:F,4,0)="""",
        """",
        VLOOKUP(CONCATENATE(A" & ii   & ",B" & ii & "),Comment_data!A:F,4,0)
    )" ' Get comments

Solution 1

2 solutions were already suggested in the comments:

1. Binary VLOOKUP - see here
2. Reducing one of your VLOOKUPs

These will definitely optimize your formulas but if you want your query to run in a couple of seconds max use MS Query...

Solution 2 (the fastest - couple sec)

Use this SQL in an MS Query:

SELECT com.F FROM [CurrentSheet$] as curr 
LEFT JOIN [Comment_data$] as com 
ON (curr.A + curr.B) = com.A

This is how it works. Below I created two example tables.

Worksheet name: CurrentSheet

Worksheet name: Comment_data

The F column in CurrentSheet is the MS Query (appended to the original table). All you need to do is refresh the Query using VBA or right-click and hit refresh.

How to create an MS Query in Excel?

Two ways:

1. Go to Data -> From Other Sources -> From Microsoft Query
2. Download my SQL AddIn (free and open sources) here and just input the output range of the query (F1) and input the SQL and hit Ok