I Have erratic file transfers with Powershell, FileSytemWatcher and WinSCP

Question

I have a problem with my PowerShell script putting files to an ESB. Often it works, but not always, especially with large files things go wrong. Does anyone see errors in my script or a way to tackle/resolve my problem?

Note:

• Resume support is not an option because I have no change rights.
• With "WinSCP GUI" I manage to successfully upload large files. (resume support off)
• The directory which contains the files for upload, is monitored for file creation, then the files are copied to a separate folder for upload. When upload is successful, the file is deleted(from the second folder).
• I'm a beginner at PowerShell, so there's probably the reason for error(s). English is also not my native language, so excuse me for any mistakes.

The script is started in TaskScheduler:

    Powershell.exe -NoExit -File "script.ps1"

Here's the script:

    $folder = 'Original_Folder'
    $filter = '*.xml'                             
    $destination = 'CopyTo_Folder'
    $fsw = New-Object IO.FileSystemWatcher $folder, $filter -Property @{
    IncludeSubdirectories = $false              
    NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'
    }
    $onCreated = Register-ObjectEvent $fsw Created -SourceIdentifier FileCreated -Action {
    $path = $Event.SourceEventArgs.FullPath
    $name = $Event.SourceEventArgs.Name
    $changeType = $Event.SourceEventArgs.ChangeType
    $timeStamp = $Event.TimeGenerated
    Write-Host "$timeStamp The file '$name' was $changeType and copied for SFTP to $destination"
    "$timeStamp The file '$name' was $changeType and copied for SFTP to $destination" | Set-Content 'Powershell_Script.log'
    Copy-Item $path -Destination $destination -Force -Verbose

    # Load WinSCP .NET assembly
    Add-Type -Path "C:\Program Files (x86)\WinSCP_5_11_3\WinSCPnet.dll"

    # Set up session options
    $sessionOptions = New-Object WinSCP.SessionOptions -Property @{
    Protocol = [WinSCP.Protocol]::Sftp
    HostName = "ftp-server"
    UserName = "User account"
    Password = "User password"
    SshHostKeyFingerprint = "ssh-rsa bla bla"
    }

    $sessionOptions.AddRawSettings("FSProtocol", "2")

    $session = New-Object WinSCP.Session
    $Session.SessionLogPath="WinSCP.log"
    try
    {
        # Connect
        $session.Open($sessionOptions)

        # Upload files

            # Set up transfer options
            $transferOptions = New-Object WinSCP.TransferOptions -Property @{
            ResumeSupport = New-Object WinSCP.TransferResumeSupport -Property @{ State = [WinSCP.TransferResumeSupportState]::Off }
            }

            # Upload with transferoptions and delete original when succesfull    
            $transferResult=$session.PutFiles("CopyTo_Folder\*", "/Upload_Folder on ftp-server/", $True, $transferOptions)

            # Throw on any error
            $transferResult.Check()

            # Print results
            foreach ($transfer in $transferResult.Transfers)
            {
                Write-Host "Upload of $($transfer.FileName) succeeded"
                "$timeStamp The file $($transfer.FileName) was successfully uploaded" | Add-Content 'Powershell_Script.log'
            }

    }
    finally
    {
        $session.Dispose()
    }
    }

Answer 1

I seem to have tackled the problem.

When getting the copy action in between pauses, it works fine

So:

    Start-Sleep -s 30
    Copy-Item $path -Destination $destination -Force -Verbose
    Start-Sleep -s 30

FileSystemWatcher acts inmediatly, as should be expected? So, the file is copied while it isn't totally created. After the copy command the upload will immediately start so I put a pause there too so the copy action can finish before upload starts.

It would be better to monitor these actions too and let the script resume after the actions are completed. Although I don't know if that is possible for the file creation with FSW...