how to copy data based on a specific cell value from several workbooks into a master workbook

Question

I've have a problem where I need to copy data from 2 workbooks into a master one based on a specific set of values (several names) in a column 3. I'm new to VBA, and probably I can't precisely ask the question to find an answer, apologies for this. Would you please help me, i need to pull rows of data from each workbook only if a column 3 contains a name I'm looking for. I have the below code to pull the data from every workbook in a specific folder, however it grabs absolutely everything.

Sub copyDataFromManyFiles()

With Application
    .DisplayAlerts = False
    .EnableEvents = False
    .ScreenUpdating = False
End With

Dim FolderPath As String, FilePath As String, FileName As String

FolderPath = "C:\Users\Jasiek\Desktop\Yuuuge MacroTest\"
FilePath = FolderPath & "*ennik*.xl*"
FileName = Dir(FilePath)

Dim lastrow As Long, lastcolumn As Long

Do While FileName <> ""
Workbooks.Open (FolderPath & FileName)

lastrow = ActiveSheet.Cells(Rows.Count, 1).End(xlUp).Row
lastcolumn = ActiveSheet.Cells(1, Columns.Count).End(xlToLeft).Column

Range(Cells(2, 1), Cells(lastrow, lastcolumn)).Copy
Application.DisplayAlerts = False
ActiveWorkbook.Close

erow = Sheet1.Cells(Rows.Count, 1).End(xlUp).Offset(1, 0).Row
ActiveSheet.Paste Destination:=Worksheets(1).Range(Cells(erow, 1), Cells(erow, 30))

FileName = Dir

Loop

With Application
    .DisplayAlerts = True
    .EnableEvents = True
    .ScreenUpdating = True
End With

'Call removeDuplicates

End Sub

How should I modify the code to filter the data before rows are copied? I do care about performance as there are 100k+ records. I would really appreciate help. Thanks.

Answer 1

Presuming that you're dealing with a fixed set of names (ie: 2-3) the easiest way to proceed would probably to use the instr() method to determine whether the cell value has a name that you're looking for.

If you're dealing with a larger list of names you could also just iterate through a collection (or array) of names for each row check. The instr() method below returns the integer position of the searched string from the string you're examining. If it finds something it will return some value greater than 0 for that position (ie: position 1 being the smallest value where it finds a match). The vbTextCompare option just ensures that it is case insensitive as it does the comparison (ie: upper or lower case mismatch).

I like to refer to worksheets and workbooks explicitly with my calls to ensure things are clean and specific, but you may be able to use ActiveWorkbook or ActiveWorksheet if you have a strong preference for it.

In terms of efficiency if you store each cell's value in a temporary string variable and just run the comparison against the tempStr string you will save retrieval time compared to referring to the cell value each time (which takes a much longer time to complete execution for x comparison checks for each row for x names).

Option Compare Text

Option Explicit

Public Sub start()
    Dim foundBool As Boolean
    Dim oxl As Excel.Application
    Dim wb1 As Excel.Workbook

    Dim tempName1 As String
    Dim tempName2 As String

    Dim tempStr1 As String
    Dim tempStr2 As String
    Dim tempInt1 As Integer
    Dim tempInt2 As Integer

    Set oxl = New Excel.Application

    Set wb1 = oxl.Workbooks().Open("[path here]\Book1.xlsm")

    wb1.Activate

    'ActiveWorkbook.Worksheets ("") can also possibly be used
    'if needed in lieu of a oxl and wb1 declaration, or perhaps
    'even ActiveSheet.

    'cells().value provides the apparent value of the cell taking formatting into account. Applies to things like dates
    'or currencies where the underlying value might be different than what's shown.
    tempStr1 = CStr(wb1.Worksheets("Sheet1").Cells(1, 1).Value)
    tempName1 = "John"

    'cells().value2 provides the underlying actual value of the cell (i.e.: without any formatting or interperetation).
    'if you deal a lot with dates/currencies and don't want excel formatting to get between you and the source
    'data use this one. Used here only to show the two different options
    tempStr2 = CStr(wb1.Worksheets("Sheet1").Cells(2, 1).Value2) '<---next row's value
    tempName2 = "Peter"


    tempInt1 = InStr(1, tempStr1, tempName1, vbTextCompare)
    tempInt2 = InStr(1, tempStr1, tempName2, vbTextCompare)

    If tempInt1 > 0 Or _
        tempInt2 > 0 Then

        foundBool = True

    End If

    wb1.Close
    oxl.Quit
    Set wb1 = Nothing
    Set oxl = Nothing

End Sub