Why does libc++ getline block when reading from pipe, but libstdc++ getline does not?

Question

TL;DR

A program that uses the libc++ version of the getline function will block when it reads input from a pipe until the pipe's buffer is full.

The same is NOT true for the libstdc++ version of the getline function: Here the function immediately reads and returns a line of input as soon as it becomes available.

Should I expect this difference in behaviour between libstdc++ and libc++ ? [ EDIT: I am not fishing for an opinion here, I simply don't know enough about pipes nor the difficulties of implementing a C++ standard library. To me this difference in behaviour certainly was surprising, but maybe someone knows better and can assure me that such differences are to be expected, that maybe this is simply an implementation detail?]

More importantly, what can I do to make libc++ behave like libstdc++ does? That is to say, the getline function should not wait until the pipe buffer is full, it should immediately return a line of input as soon as it is available.

See below for a code example that shows how I read from and write to pipes.

Environment

• macOS 10.13.1 (High Sierra)
• Xcode 9.1
• Apple LLVM version 9.0.0 (clang-900.0.38)

I suspect that the problem is not limited to macOS, but I haven't got a Linux dev system with clang that I can use to test.

Test preparation

Open three shells, let's call them A, B and C.

In shell A: Create a new file pipe-test.cpp and add the source code from below. Compile the source code once with libstdc++ and once with libc++ :

g++ -stdlib=libstdc++ -o pipe-test-libstdc++ pipe-test.cpp
g++ -stdlib=libc++ -o pipe-test-libc++ pipe-test.cpp

In shell A: Create two pipes:

mkfifo input-pipe output-pipe

Test 1, using the libstdc++ version of the program

• In shell A, run this command: pipe-test-libstdc++ input-pipe output-pipe
• In shell B, run this command: cat output-pipe
• In shell C, run this command: cat >input-pipe
• In shell C, type the line "foo" and press ENTER
• Switch to shell B: You will see the string "foo".
• What happened? The program that is running in shell A has read the line "foo" from the input pipe using the getline function and has immediately printed the line to the output pipe.
• In shell C, type CTRL+D. This concludes the test, all shells should now be back on the command line.

Test 2, using the libc++ version of the program

• In shell A, run this command: pipe-test-libc++ input-pipe output-pipe
• In shell B, run this command: cat output-pipe
• In shell C, run this command: cat >input-pipe
• In shell C, type the line "foo" and press ENTER
• Switch to shell B: You will NOT see the string "foo".
• What happened? The program that is running in shell A is still blocking, the getline function has not yet received the line "foo" from the input pipe because that pipe's buffer is not yet full.
• In shell C, type CTRL+D.
• Switch to shell B: NOW you will see the string "foo".
• Note that instead of typing CTRL+D you can also paste a lot of text into shell C. Once the pipe's buffer becomes full the getline function will start to read the input line-by-line until it has emptied the pipe's buffer.

The source code

#include <string>
#include <iostream>
#include <fstream>


int main(int argc, char** argv)
{
  if (argc != 3)
  {
    std::cout
      << "Usage: pipe-test /path/to/input-pipe /path/to/output-pipe"
      << std::endl;
    return 1;
  }

  std::string pathToInputPipe = argv[1];
  std::string pathToOutputPipe = argv[2];

  std::cout
    << "Input pipe = " << pathToInputPipe << std::endl
    << "Output pipe = " << pathToOutputPipe << std::endl;

  std::ifstream inputPipeStream;
  inputPipeStream.open(pathToInputPipe.c_str());
  if (! inputPipeStream)
  {
    std::cout
      << "Failed to open input pipe " << pathToInputPipe
      << std::endl;
    return 1;
  }
  else
  {
    std::cout
      << "Input pipe: result of eof() = " << inputPipeStream.eof() << std::endl
      << "Input pipe: result of bad() = " << inputPipeStream.bad() << std::endl
      << "Input pipe: result of good() = " << inputPipeStream.good() << std::endl
      << "Input pipe: result of fail() = " << inputPipeStream.fail() << std::endl;
  }

  std::ofstream outputPipeStream;
  outputPipeStream.open(pathToOutputPipe.c_str());
  if (! outputPipeStream)
  {
    std::cout
      << "Failed to open output pipe " << pathToOutputPipe
      << std::endl;
    return 1;
  }
  else
  {
    std::cout
      << "Output pipe: result of eof() = " << outputPipeStream.eof() << std::endl
      << "Output pipe: result of bad() = " << outputPipeStream.bad() << std::endl
      << "Output pipe: result of good() = " << outputPipeStream.good() << std::endl
      << "Output pipe: result of fail() = " << outputPipeStream.fail() << std::endl;
  }

  int lineNumber = 0;
  while (true)
  {
    lineNumber++;
    std::string line;
    bool getlineFailResult = getline(inputPipeStream, line).fail();
    if (getlineFailResult)
    {
      std::cout << "Failed to read stream, stopping" << std::endl;
      break;
    }
    else
    {
      std::cout << "Received line " << lineNumber << ": " << line << std::endl;
      outputPipeStream << line << std::endl;
      outputPipeStream.flush();
    }
  }

  return 0;
}

Answer 1

As you are probably aware, the libc++ is a non-GPL (MIT) version of the standard library. libstdc++ is the GPL version. libc++ is part of LLVM. libstdc++ is a GPL library which usually ships with GCC.

They are two entirely different implementations. Which one is better, or which one you should use is an opinion. There are a number of threads on this, for instance: Should I use libc++ or libstdc++?

I could not readily find a spec on getline() which would specify the behavior you are concerned about.

Here is the source for getline in the libc++ version https://github.com/llvm-mirror/libcxx/blob/018a3d51a47f7275c59e802709104498b729522b/include/istream#L1037

The fact that they behave differently does not suprise me. If you need a specific implementation / behavior of getline() I would either write your own, as it's not a complex operation, and use standard POSIX system calls to do it which are spec'd out, and should be the same - regardless of OS or compiler.

Also (opinion here), my advice is to avoid using the standard C++ libraries, especially iostream, implementation aside, unless they are already in a large code base and you must use them. They are full of errors, have differing behavior, and are generally not fast.