PowerShell - Filter filenames using pipeline where-object with array of strings

Question

Windows 10 PS 5.1

I am trying to confirm if a filename contains one or more strings. They are not exact matches, so should be able to handle a wildcard, though, usage of wildcards is not necessary if the comparison operator doesn't accept them. In my scenario, I am iterating through many filenames in a pipeline and filtering filenames using where-object. I've only been able to filter using a single string whereas I haven't been able to filter using an array of strings. I've uploaded my code to github.

https://github.com/ChrisK847/WordSearcher

Here are the key lines of code

#Line 29
$FileNameLike = "2016-09", "2016-10" #, "2016-08", "2016-10"

#Line 70
if($FileNameLike -eq ""){$FileNameLike = "*"}elseif($FileNameLike -ne ""){$FileNameLike = $FileNameLike | %{ $_ -replace $_,"*$_*"}} #{$FileNameLike = "*$FileNameLike*"}

#Line 103
Where-Object FullName -like $FileNameLike |

The most important line where I'm trying to make the changes is line 103.

I've tried using -in, -match (without using *), -contains, and -like. Neither of them worked on arrays. The Microsoft documentation "about comparison operators" does not contain the word array. I've tried flipping the objects, like "Where-object {$FileNameLike -like $_.FullName} using each of the comparison operators, but that didn't work. The Microsoft Document "About Where-Object" contains one example, but it doesn't work in my situation.

#From About Where-Object
Get-Process | where -Property ProcessName -in -Value "Svchost", "TaskHost", "WsmProvHost"

In the About Where-Object example, they don't provide an example with wildcards. I do not know the full name of the file before hand, so a wildcard or an operator that performs similarly to an operator with a wildcard is necessary. I only want to list my keywords on line 29.

The directory that I am searching in has 56,000 files, so I need to use the pipeline, or it ends up maxing out my PC's memory.

So that you don't have to create a directory with 56,000 files with various names and create new directories, and so that testing is faster, I have been testing with sample code that should replicate what I'm trying to do. I still haven't had any luck with the simplest of examples

cls
$Matches = $null
$keyWords = "3", "r"
$fileNames = "file1","file2","file3"

#$null -ne ($keyWords | ? { $fileNames -match $_ }) #THIS WORKS, BUT DOES NOT CONFORM WITH THE ORDER IN MY SCRIPT.
ForEach($fileName in $fileNames){
    $fileName
    $fileName | ?{$keyWords -contains $_}
    $fileName | ?{$keyWords -like $_}
    $fileName | ?{$keyWords -in $_}
    $fileName | ?{$keyWords -match $_}
    $fileName | ?{$_ -contains $keyWords}
    $fileName | ?{$_ -like $keyWords}
    $fileName | ?{$_ -in $keyWords}
    $fileName | ?{$_ -match $keyWords}
}

In the sample code, notice this line #$null -ne ($keyWords |? { $fileNames -match $_ })

That works outside of my pipeline where-object Fullname -like $FileNameLike and is the only working example, however, how do I put that inside my where-object line 103?

Answer 1

You cannot specify multiple patterns with -like operator. To specify multiple patterns with like you would have to specify multiple expressions separated by -and or -or

Where-Object {$_.Name -like "*2016-09*" -or $_.Name -like "*2016-10*"}

You can build the expression and then use it as your Where-Object filter.

(Updated to use scriptblock and not use invoke-expression based on comments - Thank you zelt42)

$FileNameLike = "2016-09", "2016-10" #, "2016-08", "2016-10"

$filter = if (!$FileNameLike) {
    [scriptblock]::Create('$_.Name -like "*"')
}
else {
    $output = foreach ($filter in $FileNameLike) {
        "`$_.Name -like '*$filter*'"
    }
    [scriptblock]::Create($output -join ' -or ')
}

#Line 103
Where-Object -FilterScript $filter |

Alternatively, the -match operator utilizes regular expression so you could do something like this instead

Where-Object {$_.Name -match "2016-09|2016-10"} 

Build your regex pattern and use that in your Where-Object filter block

$FileNameLike = "2016-09", "2016-10" #, "2016-08", "2016-10"

$regexPattern = if (!$FileNameLike) {
    ".*"
} else {
    ($FileNameLike | % {[regex]::Escape($_)}) -join '|'
}

Where-Object {$_.name -match $regexPattern }

Answer 2

I am trying to confirm if a filename contains one or more strings.

Multiple search strings represent multiple queries. I suggest writing them in test cases.

$location = "C:\lab"

Get-ChildItem -Path $location -Recurse -File | 
  Where-Object {
    switch -Wildcard ($_.Name)
    {
      '*2016-08*' {$true; Break}
      '*2016-09*' {$true; Break}
     #'*2016-10*' {$true; Break}
     #'*2016-11*' {$true; Break}
      default     {$false}
    }
  } | 
  Select-Object -Property Name 


PS > .\Stack Overflow Demo.ps1

Name
----
report 2016-09-01.txt

---

I only want to list my keywords on line 29.

It is possible to generate code from a variable, but this obscures your intent and is nonstandard.

If you accept user input, this makes your script vulnerable to code injection .

$location = "C:\lab"

$FileNameLike = "*2016-08*", "*2016-09*" , "*2016-10*", "*2016-11*"

$beginning = @"
Get-ChildItem -Path `$location -Recurse -File | 
  Where-Object {
    switch -Wildcard (`$_.Name)
    {

"@

$middle = [System.Text.StringBuilder]::new()

foreach ($myString in $FileNameLike)
{  [void]$middle.AppendLine( "      '$myString' {`$true; Break}" )  }

$end = @"
      default     {`$false}
    }
  } | 
  Select-Object -Property Name 
"@

$myScript = $beginning + $middle.ToString() + $end 

Invoke-Expression -Command $myScript




PS > .\dynamic code generation 02.ps1

Name
----
report 2016-09-01.txt
report 2016-10-01.txt

Explanation

$location = "C:\lab"

Save the path to the folder being scanned.

---

Get-ChildItem -Path $location -Recurse -File | 

List the contents of $location . Scan subfolders with -Recurse . Limit output to -File s. Begin a pipeline | .

---

  Where-Object {
  } | 

Where-Object returns all objects for which the script block statement is true.

Continue the pipeline |

---

    switch -Wildcard ($_.Name)
    {
      '*2016-08*' {$true; Break}
      '*2016-09*' {$true; Break}
     #'*2016-10*' {$true; Break}
     #'*2016-11*' {$true; Break}
      default     {$false}
    }

Use a switch to handle multiple If statements .

-Wildcard indicates that the condition is a wildcard string. The comparison is case-insensitive.

Optionally, you might use the -CaseSensitive switch.

$_ contains the current object in the pipeline. You can use this variable in commands that perform an action on every object or on selected objects in a pipeline.

$_.Name is the file name of the current file being processed.

'*2016-08*' {$true; Break}

Each case consists of a wildcard string and an action . Define the string including wildcards .

Remember that Where-Object returns all objects for which the script block statement is true. So, when one of our strings matches the file name, use the action to return $true and then Break . The Break keyword stops processing and exits the Switch statement.

If none of the strings match, output $false .

---

Select-Object -Property Name 

Output the Name s of the files that matched.

---

PS > .\Stack Overflow Demo.ps1

Name
----
report 2016-09-01.txt

Run the script and show the result.

Dynamic Script Explanation

$FileNameLike = "*2016-08*", "*2016-09*" , "*2016-10*", "*2016-11*"

Store the search strings in an array including their wildcards.

---

$beginning = @"
"@

$middle = [System.Text.StringBuilder]::new()

$end = @"
"@

$myScript = $beginning + $middle.ToString() + $end 

Invoke-Expression -Command $myScript

If we are going to generate dynamic code, we will need a string and Invoke-Expression .

We can dynamically generate test cases based on the value of $FileNameLike and store the result in $middle .

$beginning and $end are static strings.

---

$beginning = @"
Get-ChildItem -Path `$location -Recurse -File | 
  Where-Object {
    switch -Wildcard (`$_.Name)
    {

"@

Use a double-quoted here-string to define $beginning

Because we're using double quotes, escape any dollar signs with `

Note that embedded variables like $location will work as intended because expressions are evaluated and run in the current scope .

Leave a trailing newline to prepare for the $middle .

---

$middle = [System.Text.StringBuilder]::new()

foreach ($myString in $FileNameLike)
{  [void]$middle.AppendLine( "      '$myString' {`$true; Break}" )  }

Use StringBuilder as the container for your dynamically generated code. StringBuilder saves us the cost of copying strings in memory.

Use AppendLine() to add the string and a newline character to the string we are building.

Cast the method call to [void] to discard output from the method.

Use foreach to iterate over each value of $FileNameLike

Note that $myString is not escaped. This is what allows the values in $FileNameLike to be introduced to the code dynamically.

---

$myScript = $beginning + $middle.ToString() + $end 

To output a string from a StringBuilder object, use the ToString() method.

---

Invoke-Expression -Command $myScript

Once the command has been dynamically generated, use Invoke-Expression to execute the contents of the string.

Using Invoke-Expression to execute a string represents a security risk . Any user input that appears inside the string can do or delete anything the underlying user can.

---

You mentioned that there are 56,000 files to process. If you are able to use PowerShell 7 , you might be able to make use of ForEach-Object -Parallel to increase the processing speed. The examples discuss some of the coding practices you will need to use -Parallel.

If you want a fast and reliable way to search files, you might like X1 Search .

If you choose to use regex instead, note that [regex]::Escape()

Escapes a minimal set of characters (, *, +, ?, |, {, [, (,), ^, $, ., #, and white space) by replacing them with their escape codes.

For example, bracket ] and brace } are not escaped. This might not affect this use case. Please understand the limitations of [regex]::Escape() and be prepared to test cases.

---

Here is a demonstration using -FilterScript to hijack the shell.

$location = "C:\lab" 

$FileNameLike = "'; Start-Process -FilePath powershell.exe -ArgumentList `"-Command ```{```"Hello```"```}`" -NoNewWindow; '", "2016-10" #, "2016-08", "2016-10"

$filter = if (!$FileNameLike) {
    [scriptblock]::Create('$_.Name -like "*"')
}
else {
    $output = foreach ($filter in $FileNameLike) {
        "`$_.Name -like '*$filter*'"
    }
    [scriptblock]::Create($output -join ' -or ')
}

#Line 103
Get-ChildItem -Path $location -Recurse -File | Where-Object -FilterScript $filter




PS > .\proof of concept.ps1

    Directory: C:\lab

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---            3/5/2021  7:45 AM              0 report 2016-09-01.txt
-a---            3/5/2021  7:45 AM              0 report 2016-10-01.txt
-a---            3/5/2021  7:45 AM              0 report 2017-09-01.txt

PS > Hello
HelloHello