Problems getting desired output from Command line process

Question

I am trying to add Windows natives notifications for the lftp linux process. To do that I'm trying to launch the command from c# Console App to catch the status of lftp. The problem is that I don't have the same output if I launch it from c# that if I run it from bash.

I have an helper class to do my Command Line calls :

public static class ShellHelper
{
    public static void Bash(this string cmd)
    {
        var escapedArgs = cmd.Replace("\"", "\\\"");

        var process = new Process()
        {
            StartInfo = new ProcessStartInfo
            {
                FileName = "C:/cygwin64/bin/sh.exe",
                Arguments = $"-c \"{escapedArgs}\"",
                RedirectStandardError = true,
                RedirectStandardOutput = true,
                UseShellExecute = false,
                CreateNoWindow = true,
                StandardOutputEncoding = Encoding.UTF8,
                WindowStyle = ProcessWindowStyle.Hidden,
            }
        };

        process.OutputDataReceived += new DataReceivedEventHandler(ReadOutput);
        process.ErrorDataReceived += new DataReceivedEventHandler(ErrorOutput);

        process.Start();
        process.BeginOutputReadLine();
        process.BeginErrorReadLine();
        process.WaitForExit();
    }

    private static void ErrorOutput(object sender, DataReceivedEventArgs e)
    {
        if (e.Data != null)
        {
            Console.WriteLine("Error: " + e.Data);
        }
    }

    private static void ReadOutput(object sender, DataReceivedEventArgs e)
    {
        if (e.Data != null)
        {
            Console.WriteLine(e.Data);
        }
    }
}

And my call is :

ShellHelper.Bash("unbuffer lftp -p 22 -u login,password sftp://XXXXXXXXX -e \"set mirror:use-pget-n 20; mirror --parallel=3 -c -P5 /home29/qualinost/files/tests ./tests; quit; \"");

This is the expected output that I get when I launch the command from bash

cd `/home29/qualinost/files/tests' [Connecting...]
cd `/home29/qualinost/files/tests' [Connected]      
`Kaamelott - S01E17 - Le Signe.mkv', got 0 of 48941955 (0%)
    ............................................................................................................................................................................................................................................
[A`Kaamelott - S01E17 - Le Signe.mkv', got 32768 of 48941955 (0%)
................................................................................    ................................................................................    .............................................................................
[A`Kaamelott - S01E17 - Le Signe.mkv', got 131072 of 48941955 (0%)  
................................................................................    ................................................................................    .............................................................................
[A`Kaamelott - S01E17 - Le Signe.mkv', got 458752 of 48941955 (0%) 235.7K/s
oo..............................................................................    ................................................................................    .............................................................................
[A`Kaamelott - S01E17 - Le Signe.mkv', got 720896 of 48941955 (1%) 265.0K/s   

And this is what I get when I launch the same command from c# :

cd `/home29/qualinost/files/tests' [Connecting...]
cd `/home29/qualinost/files/tests' [Connected]
`...1E17 - Le Signe.mkv', got 0 of 48941955 (0%)
`...1E17 - Le Signe.mkv', got 131072 of 48941955(0%)..........................
`...1E17 - Le Signe.mkv', got 917504 of 48941955 (1%) 598.5K/s.................
`...1E17 - Le Signe.mkv', got 1671168 of 48941955 (3%) 730.4K/s  ..............
`...1E17 - Le Signe.mkv', got 1900544 of 48941955 (3%) 734.7K/s   .............
`...1E17 - Le Signe.mkv', got 2588672 of 48941955 (5%) 650.1K/s   .............
`...1E17 - Le Signe.mkv', got 3473408 of 48941955 (7%) 727.3K/s   .............
`...1E17 - Le Signe.mkv', got 4096000 of 48941955 (8%) 718.8K/s   .............
`...1E17 - Le Signe.mkv', got 4816896 of 48941955 (9%) 747.9K/s   .............
`...1E17 - Le Signe.mkv', got 5668864 of 48941955 (11%) 954.0K/s  .............

As you can see the name of the file is truncate. My guess is it's maybe a buffer size problem or an encoding format problem with some specials chars that c# doesn't like.

I also tried to directly redirect the output in a file from the command like so:

ShellHelper.Bash("unbuffer lftp -p 22 -u login,password sftp://XXXXXX -e \"set mirror:use-pget-n 20; mirror --parallel=3 -c -P5 /home29/qualinost/files/tests ./tests; quit; \" > output.log");

But it's the same, works like a charm from bash but missing parts when I launch from c#..

Any ideas on what's happening here?

EDIT : Someone posted a comment (and deleted it??) and pointed out that the maximum number of characters that "c#" give me by output line is 80. He asked me to resize the Console with

Console.SetWindowSize(400, 800);

at the begining of my program but it doesn't changed anything. But the 80 chars max is probably an interresting remark!

EDIT 2: I cleaned up the output of lftp with a sed command to remove the progress bar and only get the name of the file and it's progression. But still the output is truncate when I launch my script from C#. I did a test with an extremly short file name (yo.mkv) and everything is here! So I think it's definitly a buffer size problem of StandardOutput. Is there a way to define a bigger size of this buffer?

Answer 1

I finally found a workaround on this broken MS async output redirection. I used this code https://stackoverflow.com/a/13860551/1502444 to be able to read each chars of the output by using StandardOutput.Read() method in a different thread and changing the Read buffer size. Would like to thanks the author of this solution and raterus to pointed out the async read problem.