Recursively downloading files from FTP server

Question

Hi All i am new to power shell scripting, i am trying to download all files in root directory and files in sub folder based on file name from FTP server. As i am looking to download files which got matched on files names in like, as my path will always point to root directory and root directory is having text files and sub folders, the sub folders contains text files.

I found an example to get text files to download to local from ftp server, but the example i found is only downloading text files from root directory it is not scanning sub folders and not getting files in sub folders.

Here is the example i tried

#    $url = "ftp://XXX.com/" 
    $user = 'UserName' 
    $pass = 'Password'
    $folder = 'FTP_Folder'
    $target = "C:\Folder\Folder1\"

    #SET CREDENTIALS
    $credentials = new-object System.Net.NetworkCredential($user, $pass)

    function Get-FtpDir ($url,$credentials) {
        $request = [Net.WebRequest]::Create($url)
        $request.Method = [System.Net.WebRequestMethods+FTP]::ListDirectory
        if ($credentials) { $request.Credentials = $credentials }
        $response = $request.GetResponse()
        $reader = New-Object IO.StreamReader $response.GetResponseStream() 
        $reader.ReadToEnd()
        $reader.Close()
        $response.Close()
    }

    #SET FOLDER PATH
    $folderPath= $ftp + "/" + $folder + "/"

    $Allfiles=Get-FTPDir -url $folderPath -credentials $credentials
    $files = ($Allfiles -split "`r`n")

    $files 

    $webclient = New-Object System.Net.WebClient 
    $webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass) 
    $counter = 0
    foreach ($file in ($files | where {$_ -like "*.txt"})){
        $source=$folderPath + $file  
        $destination = $target + $file 
        $webclient.DownloadFile($source, $target+$file)

        #PRINT FILE NAME AND COUNTER
        $counter++
        $counter
        $source
    }

I tried a module PSFTP module functions like Get-ftpchilditem function with recursive, but it not worked for me.

Please help me in this to get files from root folder and sub folders of FTP server to local system.

Answer 1

I have done it in the past using WinSCP and SFTP. The example is below. This uses SFTP SCP which is basically over SSH. I am sure you can change the WInSCP protocol to ftp and try different combination. Hope this helps you. Change the FileMasks to only include .txt and this will go through sub folders. Also for this to work you will require WinSCPnet.dll and WinSCP.exe which you can easily download from WinSCP site. Download the ".Net Assembly/COM Library" and keep the script and the required files in the same folder and update the path in the script. Good Luck.

# Load WinSCP .NET assembly
Add-Type -Path "C:\Users\administrator.AD\Documents\WinSCP    Script\WinSCPnet.dll"


$hostname = "ftp.xyz.net"
$user = "me"
$pass = "mypassword"
$sfolder = "/home/me"
$dfolder = "C:\tmp"
$sshHostKeyFingerprint = "ssh-rsa 2048 eb:2c:f9:ab:19:2e:0c:22:02:d4:8f:64:44:75:ec:04"
$mask = "*.jpg;*.jpeg;*.png;*.tiff;*.txt"
####################### END USER CONFIGURABLE SETTINGS #########################

# Main script
# Setup session options

$sessionOptions = New-Object WinSCP.SessionOptions -Property @{

Protocol = [WinSCP.Protocol]::Scp

HostName = $hostname

UserName = $user

Password = $pass

SshHostKeyFingerprint = $sshHostKeyFingerprint

}


$session = New-Object WinSCP.Session

$transferOptions = New-Object WinSCP.TransferOptions

$transferOptions.FileMask = "*.jpg; *.jpeg; *.png; *.tiff;"

try

{
# Will continuously report progress of synchronization

$session.add_FileTransferred( { FileTransferred($_) } )

# Connect

$session.Open($sessionOptions)

# Synchronize files

$synchronizationResult = $session.SynchronizeDirectories(

[WinSCP.SynchronizationMode]::Local,$dfolder, $sfolder, $False, $False, 1, $transferOptions)

# Throw on any error

$synchronizationResult.Check()

}

finally

{

# Disconnect, clean up

$session.Dispose()

}