Powershell GCI，达到深度速度

Question

I need a way to speed up the following code for use on a VERY large server. All I need is the list of the fullname param. I would like to use something like the following: http://newsqlblog.com/2012/05/22/concurrency-in-powershell-multi-threading-with-runspaces/

I was reading about it and it seems pretty simple, however I cannot wrap my head around allowing threads to create new Threads in the existing pool... and how to pass that pool along.

Any help would be great, because the speed of this is just terrible and I have SOOOO many more resources then I am utalizing.

Function Get-ChildItemToDepth {   
    Param(     
        [String]$Path = $PWD,    
        [int]$ToDepth = 255,     
        [Byte]$CurrentDepth = 0
    ) 
    if ($ToDepth -lt 0) {
        return get-item $path
    }
    $CurrentDepth++
    Get-ChildItem $Path | Where-Object { $_.PSIsContainer } | %{ $_ 
        If ($CurrentDepth -le $ToDepth) {
            Get-ChildItemToDepth -Path $_.FullName `
            -ToDepth $ToDepth -CurrentDepth $CurrentDepth
        } 
    }
}   

NOTE: I am restricted to v2.0

Summary: I basically need the path to all folders up to $depth into an array as fast as digitally possible.

MISSERABLE FAILURE ATTEMPT:

$func = `
{
    Param(     
        [String]$Path = $PWD,    
        [int]$ToDepth = 255,     
        [Byte]$CurrentDepth = 0,
        $p = $pool
    ) 
    if ($ToDepth -lt 0) {
        return $path
    }
    $CurrentDepth++
    $folders = Get-ChildItem $Path | Where-Object { $_.PSIsContainer } | select -expand FullName
    If ($CurrentDepth -le $ToDepth) {
        foreach ($path in $folders) {
            $pipeline  = [System.Management.Automation.PowerShell]::create()  
            $pipeline.RunspacePool = $pool 
            $pipeline.AddScript($func).AddArgument($path).AddArgument($ToDepth).AddArgument($CurrentDepth).AddArgument($pool)
            $AsyncHandle = $pipeline.BeginInvoke() 
            $folders += $pipeline.EndInvoke($AsyncHandle)  
            $pipeline.Dispose()  
        }
    }
    return $folders
}    

$path = "\\server\users-folder\"
$toDepth = 3

$pool = [RunspaceFactory]::CreateRunspacePool(1, 4) 
$pool.ApartmentState = "STA" 
$pool.Open() 
$pipeline  = [System.Management.Automation.PowerShell]::create()  
$pipeline.RunspacePool = $pool  
$pipeline.AddScript($func).AddArgument($path).AddArgument($toDepth).AddArgument($CurrentDepth).AddArgument($pool)
$AsyncHandle = $pipeline.BeginInvoke() 
$RESULTS = $pipeline.EndInvoke($AsyncHandle)  
$pipeline.Dispose()  
$pool.Close()

Answer 1

try to invoke this in a workflow - a powerful native powershell multithreading solution available out of the box with PowerShell 3.0

Workflow Invoke-Function
{
Function Get-ChildItemToDepth {   
    Param(     
        [String]$Path = $PWD,    
        [int]$ToDepth = 255,     
        [Byte]$CurrentDepth = 0
    ) 
    if ($ToDepth -lt 0) {
        return $path
    }
    $CurrentDepth++
    Get-ChildItem $Path | Where-Object { $_.PSIsContainer } | %{ $_ 
        If ($CurrentDepth -le $ToDepth) {
            Get-ChildItemToDepth -Path $_.FullName `
            -ToDepth $ToDepth -CurrentDepth $CurrentDepth
        } 
    }
}  
Get-ChildItemToDepth 
}

This is a pretty messy implementation, I just wrapped your function and a call around a workflow so you could copy-paste it into a session for ease of testing. Please let me know how that works for you. If you want to see the difference in speed I would pipe the results of the function to Measure

Get-ChildItemToDepth | Measure
Invoke-Function | Measure

Usage (after you pasted the workflow into a session)

Invoke-Function

Answer 2

Go to http://psasync.codeplex.com and download the psasync module (documentation here http://newsqlblog.com/2012/12/17/psasync-module-multithreaded-powershell/ ). Then ...

Import-Module psasync

$AsyncPipelines = @()

# Allow 4 concurrent processes ... adjust up / down as hardware allows
$pool = Get-RunspacePool 4

# You can play with this to get better performance by going deeper into the directory structure to populate more dirs in the array
# gross generalization -> more dirs + more runspaces in $pool = faster performance

$rootDirs = Get-ChildItem c:\ | Where-Object{$_.PSIsContainer}

# Create a script block to run code per directory stored in $rootDirs
$ScriptBlock =  { ` 
    Param(     
        [String]$Path = $path
    ) 
    Get-ChildItem $path -recurse | Where-Object{$_.PSIsContainer}

}   

# Load up processes on the runspace pool
foreach($dir in $rootDirs) {
    $AsyncPipelines += Invoke-Async -RunspacePool $pool -ScriptBlock $ScriptBlock -Parameters $dir.FullName
}

# "Listen" to the pipelines and grab the results
$dirs = Receive-AsyncResults -Pipelines $AsyncPipelines

# PROFIT
$dirs | Foreach-Object{$_.FullName}

And thanks for reading newsqlblog.com ;)