Cannot get correct output from program redirected via pipe C++ (Windows)

Question

I would like to achieve a result, so that the output of the program goes splitted to the deque structure.

To describe to problem: I am dealong with with the redirecting the output of the program created with CreateProcess. I need to read the output of program and process it line by line. The program itself provides the output in the convenient form, but the output which I receive via unnamed pipe is delivered slower (with smaller frequency) they appear in the portions and in such a way that the last line is somewhere cut in half. The next portion of the stream from the pipe will match and finish the line, but it will make some problems with the construction of the program.

What be a source of this discrepancy betweenn the cout from the program and the output redirected throught the pipe?

EDIT: According to the proposition by user4581301 I've tried to use the stringstream and getline, but it seems that the lines are still cut in half even though the direct cout from the program without redirection to pipe does not have this problem. This leads to the issue, that the lines are splitted in the different elements of the queue (please look at the code below).

Sample from the console

The method ReadFromPipe is run in the loop.

void ProcessManagement::ReadFromPipe(void)

// Read output from the child process's pipe for STDOUT

{
    DWORD dwRead, dwWritten;
    char buffer[BUFSIZE];
    BOOL bSuccess = FALSE;
    std::deque<std::deque<std::string>> elems;

while (ReadFile(g_hChildStd_OUT_Rd, buffer, sizeof(buffer)-1, &dwRead, NULL) != FALSE)
{
    /* add terminating zero */
    buffer[dwRead] = '\0';
    std::stringstream streamBuffer;
    streamBuffer << buffer;
    BufferToQueue(streamBuffer, ' ', elems);

    // Print deque
    for (std::deque <std::string> a : elems)
    {
        for (std::string b : a)
            std::cout << b;
        std::cout << std::endl;
    }

    }
}

And the method BufferToQueue.

void ProcessManagement::BufferToQueue(std::stringstream &streamBuffer, char delim, std::deque<std::deque<std::string>> &elems) {
    std::string line;
    std::string word;
    std::deque<string> row;

 // Splitting the stringstream into queue of queue (does not work properly)
    while (std::getline(streamBuffer, line))
    {
        std::istringstream iss(line);
        while (std::getline(iss, word, delim))
            row.push_back(word);
    elems.push_back(row);
    row.clear();
    }
}

Answer 1

Expanding on @Captain Obvlious's comment regarding flush:

The problem you are facing is because the WriteToPipe function does not flush at end of line. You can fix this in the reader by ensuring that you append to the previous string if the the previous ReadFromPipe call did not have a newline as the last character.

Modified functions:

bool ProcessManagement::BufferToQueue(std::stringstream &streamBuffer, char     delim, std::deque<std::deque<std::string>> &elems)
{
    std::string line;
    std::string word;
    std::deque<string> row;

    bool is_unflushed_line = streamBuffer.str().back() != '\n';

    // Splitting the stringstream into queue of queue (does not work properly)
    while (std::getline(streamBuffer, line))
    {
        std::istringstream iss(line);
        while (std::getline(iss, word, delim)) {
            row.push_back(word);
        }
        elems.push_back(row);
        row.clear();
    }
    return is_unflushed_line;
}

void ProcessManagement::ReadFromPipe(void)
// Read output from the child process's pipe for STDOUT
{
    DWORD dwRead, dwWritten;
    char buffer[BUFSIZE];
    BOOL bSuccess = FALSE;
    std::deque<std::deque<std::string>> elems;

    while (ReadFile(g_hChildStd_OUT_Rd, buffer, sizeof(buffer)-1, &dwRead, NULL) != FALSE)
    {
        /* add terminating zero */
        buffer[dwRead] = '\0';
        std::stringstream streamBuffer;
        streamBuffer << buffer;
        bool is_unflushed_line = BufferToQueue(streamBuffer, ' ', elems);

        for(auto idx = 0; idx != elems.size(); ++idx)
        {
            for (std::string const& b : elems[idx])
                std::cout << b;
            if(idx == elems.size() - 1 && is_unflushed_line)
                break;// don't print a newline if input did not end with a newline
            std::cout << std::endl;
        }
    }
}

Answer 2

The answer from @indeterminately sequenced was correct, thank you for your help. The problem was, that the buffer to which the pipe was copied was too small and it was deviding it to the seperate ones.

The complete solution based on the help of @indeterminately sequenced to push the output to queue structure, maybe it will help someone. The only problem is that the opened program is never closed, the TerminateProcess function has to be used somewhere.

void ProcessManagement::CreateChildProcess()
// Create a child process that uses the previously created pipes for STDIN and STDOUT.
{
    SECURITY_ATTRIBUTES saAttr = { sizeof(SECURITY_ATTRIBUTES) };

    saAttr.bInheritHandle = TRUE;   //Pipe handles are inherited by child process.
    saAttr.lpSecurityDescriptor = NULL;

    // Create a pipe to get results from child's stdout.
    if (!CreatePipe(&hPipeRead, &hPipeWrite, &saAttr, 0))
        return;

    si.dwFlags = STARTF_USESHOWWINDOW | STARTF_USESTDHANDLES;
    si.hStdOutput = hPipeWrite;
    si.hStdError = hPipeWrite;
    si.wShowWindow = SW_HIDE;       // Prevents cmd window from flashing. Requires STARTF_USESHOWWINDOW in dwFlags.

    std::string command_temp = (" -i \"LAN 3\"");
    LPSTR st = const_cast<char *>(command_temp.c_str());

    BOOL fSuccess = CreateProcessA(program_path.c_str(), st, NULL, NULL, TRUE, CREATE_NEW_CONSOLE, NULL, NULL, &si, &pi);
    if (!fSuccess)
    {
        CloseHandle(hPipeWrite);
        CloseHandle(hPipeRead);
        return ;
    }

    /* Needs to be used somewhere
    TerminateProcess(pi.hProcess,exitCode);
    CloseHandle(hPipeWrite);
    CloseHandle(hPipeRead);
    CloseHandle(pi.hProcess);
    CloseHandle(pi.hThread);*/
}

void ProcessManagement::ReadFromPipe(void)
// Read output from the child process's pipe for STDOUT
{
    std::deque<std::deque<std::string>> elems;

    // Give some timeslice (50ms), so we won't waste 100% cpu.
    bProcessEnded = WaitForSingleObject(pi.hProcess, 50) == WAIT_OBJECT_0;

    // Even if process exited - we continue reading, if there is some data available over pipe.
    for (;;)
    {
        char buf[8192];
        DWORD dwRead = 0;
        DWORD dwAvail = 0;

        if (!::PeekNamedPipe(hPipeRead, NULL, 0, NULL, &dwAvail, NULL))
            break;

        if (!dwAvail) // no data available, return
            break;
        if (!::ReadFile(hPipeRead, buf, min(sizeof(buf)-1, dwAvail), &dwRead, NULL) || !dwRead)
            // error, the child process might ended
            break;
        buf[dwRead] = '\0';
        std::stringstream streamBuffer;
        streamBuffer << unflushed_line << buf; // add to buffer also last unflashed line
        unflushed_line = BufferToQueue(streamBuffer, ' ', elems);

        for (auto idx = 0; idx != elems.size(); ++idx)
        {
            for (std::string const& b : elems[idx])
                std::cout << b;

            std::cout << std::endl;
        }

    }
}

std::string ProcessManagement::BufferToQueue(std::stringstream &streamBuffer, char delim, std::deque<std::deque<std::string>> &elems) {
    std::string line;
    std::string word;
    std::deque<string> row;

    bool is_unflushed_line = streamBuffer.str().back() != '\n';

    // Splitting the stringstream into queue of queue (does not work properly)
    while (std::getline(streamBuffer, line, '\n'))
    {
        std::istringstream iss(line);
        while (std::getline(iss, word, delim)) {
            row.push_back(word);
        }
        elems.push_back(row);
        row.clear();
    }

    if (is_unflushed_line)
    {
        elems.pop_back(); // pop not fully flushed line
    }
    else line.clear(); // if the line was fully flushed return empty string

    return line; // to add to buffer for next push to queue if the last was not flushed at the end

}