Powershell filter for only unique entries in array from pipeline within custom module

Question

This is a tangent/continuation from my other question where I need to sanitize the input to one of my custom modules. Here's the code:

function Get-MyTest {

    param(
        [string[]]$MyInput
    )


    [array]$Output = @()
    $i=0

    foreach($Item in $MyInput){

        $i++

        $Output += [pscustomobject]@{
            Name = $Item
            Id = $i
        }

    }

    return $Output

}

function Get-IncorrectResults {

    param(
        [parameter(ValueFromPipeline)][array]$MyIncorrectInput
    )

    process {
    
        foreach ($Item in $MyIncorrectInput) {
        
            Write-Host "Current object: $Item `n "

        }

    }

}

Problem
Adding begin{} , process{} , and end{} allows me to individually process each entry in the pipeline, but I can no longer filter the input. A user could end up with several identical objects in the $MyIncorrectInput , for example by doing:

[array]$TestVariable = @()
$TestVariable += Get-MyTest "Test1","Test2"
$TestVariable += Get-MyTest "Test3","Test2"

This would end up with two identical entries for Test2. If I then run either of the following:

Get-IncorrectResults -MyIncorrectInput $TestVariable
$TestVariable | Get-IncorrectResults

Then the Test2 object will be processed twice. Since the goal here is to use this in GET/POST/PUT/DELETE requests for a RESTAPI then I can't have the code process the same entry (or ID) twice for DELETE since the resource has already been deleted. I therefore need to filter out to only process unique values.

Attempted solution
I tried to change the foreach loop to use $item in $UniqueInput instead, and add the following code:

$UniqueInput = $MyIncorrectInput | Get-Unique -AsString

If I add it in the process{} block it doesn't do anything, it still processes every entry including duplicates. If I add it in the begin{} block well then the value doesn't seem to carry over to the process{} block and the Write-Host section is empty because $UniqueInput seems to be empty.

Even changing to the following has no effect:

$UniqueInput = $MyIncorrectInput | Select-Object * -Unique

What is the correct way to filter an array input for only unique values in a custom module that needs to take the array from the pipeline? I don't need to have the full PSCustomObject be unique, it would be enough to have the .Id unique if that makes any difference.

Answer 1

You can do this with a HashSet<T> , this class is designed to contain only unique elements:

function Get-IncorrectResults {
    param(
        [parameter(Mandatory, ValueFromPipeline)]
        [array] $MyIncorrectInput,

        [parameter(Mandatory)]
        [string] $On,

        [parameter()]
        [switch] $CaseSensitive
    )

    begin {
        if($CaseSensitive.IsPresent) {
            return $hash = [Collections.Generic.HashSet[string]]::new()
        }
        $hash = [Collections.Generic.HashSet[string]]::new([StringComparer]::InvariantCultureIgnoreCase)
    }
    process {
        foreach ($Item in $MyIncorrectInput) {
            if($hash.Add($Item.$On)) {
                $Item
            }
        }
    }
}

$TestVariable = @(
    Get-MyTest "Test1", "Test2"
    Get-MyTest "Test3", "Test2"
)

Get-IncorrectResults $TestVariable -On Id
$TestVariable | Get-IncorrectResults -On Id