How can I inject a newline when a std::ostream object is std::flush'ed?

Question

How can I minimally wrap a std::ofstream so that any call to sync ( std::flush ) gets turned into a call to std::endl .

(everything below is an answer to the question "why would you do that?" and is not relevant to the above question)

---

I have an application which contains a custom std::ostream implementation (which wraps around zmq), let's call it zmq_stream .

This zmq_stream also internally implements a custom streambuffer , and works very nicely.

One of the motivations of this design is that in my unit tests, I can simply swap out the zmq_stream in my unit tests with a regular old std::ostringstream and test the results, completely hiding out the.network layer.

One of the features of the application itself is to redirect application output to a file or stdout via a command line flag.

Once again, using a basic switch at startup, I can pass either zmq_stream , std::ofstream or plain old std::cout to my main loop.

This is accomplished something like this:

std::ostream* out;
switch(cond)
{
  case 1:
    out = &std::cout;
    break;
  case 2:
    out = new ofstream("file");
    break;
  case 3:
    out = new zmq_stream(method, endpoints);
    break;
}

main_loop(out);

The usage of zmq_stream is as follows:

zmq_stream out(method, endpoints);

out << "do something " << 42 << " or other";
out << "and another" << std::flush; // <- full constructed buffer is sent over the network

Note: it is by design that I use std::flush and not std::endl when flushing to the.network. I do not wish to append a newline to all of my.network transmissions. As such, all.network outputs by the application use std::flush and not std::endl and changing this is not the correct answer.

Question: while for testing purposes, everything works as expected, for the std::cout and std::ofstream options, I'd like to be able to inject a newline when std::flush is called, so that my.network transmissions can be separated into newlines on stdout . This would allow me to pipe them onto the standard posix suite of tooling...

Without this injection, there is no way to determine (aside from timing) where the.network boundaries are.

I'm looking for the most minimal possible override here so that I don't have to write out a bunch of new classes. Ideally, overriding the sync() virtual method so that it calls overflow('\n'); and then calls base sync() .

However, this is proving to be much more challenging than I expected, so I'm wondering if I'm doing it wrong.

Answer 1

how do I wrap the file stream.

You can write a << overload that intercepts std::flush and adds the \n . I am only outlining the idea, no production ready code.

#include <iostream>

struct foo {
    std::ostream& out = std::cout;

    template <class T> 
    foo& operator<<(const T& t) {
        out << t;
        return *this;
    }

    foo& operator<<(std::ostream& (*f)(std::ostream&)) {
        if (f == &std::flush<std::ostream::char_type,std::ostream::traits_type>) { out << "\n"; }
        out << f;
        return *this;
    }
};


int main() {
    foo f;
    f << "a"  << std::endl << "b" << std::flush << "c";
}

In the output you can see that now both std::endl and std::flush add a \n . std::endl because it did that anyhow, and std::flush because the wrapper adds it. Live Demo :

a
b
c

Usually taking the reference of std function should be done with care. It is only allowed when they are explicitly addressable functions. Actually I didnt find that mentioned, but passing them as function pointers is how the io-manipulators are supposed to be used.

Answer 2

The initial solution I wrote doesn't work because it is a compile time solution and I can't switch on it without re-implementing inheritance (so that the main_loop function gets a base class as a parameter).

Instead, here's the answer to original problem:

#include <iostream>
#include <fstream>
#include <iomanip>

typedef std::ostringstream zmq_stream;

template <class T>
class newline_injector_streambuf: public std::basic_streambuf<T> {
public:
    using int_type = typename std::basic_streambuf<T>::int_type;
    newline_injector_streambuf(std::basic_streambuf<T>& dest) : sink(dest) {}

protected:

    virtual int_type sync() override { overflow('\n'); return sink.pubsync(); }
    virtual int_type overflow(int_type c) override { return sink.sputc(c); }

    std::basic_streambuf<T>& sink;
};

template <class T>
struct newline_injector_stream : public std::basic_ostream<typename T::char_type> {
    newline_injector_streambuf<typename T::char_type> buf;
    newline_injector_stream(T* file) : buf(*file->rdbuf())
    {
        std::basic_ostream<typename T::char_type>::rdbuf(&buf);
    }
};

void test(std::ostream& out)
{
    out << std::setfill('x') << std::setw(10) << "" << std::flush;
    out << "Hello, world!" << std::flush << "asdf" << std::endl;
}

int main() {

    newline_injector_stream out1(&std::cout);
    newline_injector_stream out2(new std::ofstream("test.output", std::ios::out | std::ios::ate));

    test(std::cout);
    test(out1);
    test(out2);

    return 0;
}

---

This is the solution that was devised thanks to @463035818-is-not-a-number's initial answer:

    struct zmq_stream;
    template<typename T>
    struct output_device {
        T& output;

        output_device(T& backing_stream) : output(backing_stream){}

        template <typename U>
        output_device& operator<<(const U& u)
        {
            output << u;
            return *this;
        }

        output_device& operator<<(std::ostream& (*f)(std::ostream&)) {

          if constexpr (!std::is_same<T, zmq_stream>::value)
            if (f == &std::flush<std::ostream::char_type,std::ostream::traits_type>)
                output << "\n";
            output << f;
            return *this;
        }
    };

// Usage:

int main()
{
   /* prelude code */
   auto out1 = output_device(std::cout);
   auto out2 = output_device(std::ofstream("filename"));
   auto out3 = output_device(zmq_stream(method, endpoints));

   out1 << "foo" << std::flush << "bar"; // results in foo\nbar
   out2 << "foo" << std::flush << "bar"; // results in foo\nbar
   out3 << "foo" << std::flush << "bar"; // results in foobar

}

template <class T>
class separator: public std::basic_streambuf<T> {
public:
    using int_type = typename std::basic_streambuf<T>::int_type;
    using char_type = typename std::basic_streambuf<T>::char_type;
    using traits_type = typename std::basic_streambuf<T>::traits_type;

    separator(std::basic_streambuf<T> *dest) : sink(dest) {}
protected:

    virtual int_type sync() override {
        if (nullptr == sink)
            return 1;
        //std::basic_streambuf<T>::overflow('\n');
        overflow('\n');
        return sink->pubsync();
    }

    virtual int_type overflow(int_type c) override {
        if (sink == nullptr)
            return 1;
        return sink->sputc(c);
    }

    std::basic_streambuf<T> *sink = nullptr;
};

template <class T> struct newline_injector_stream : public std::basic_ostream<T> {
    separator<T> buf;
public:
    newline_injector_stream(std::basic_ostream<T> &file) : buf(file.rdbuf())
    {
        std::basic_ostream<T>::rdbuf(&buf);
    }
};

The main_loop function signature now goes from taking a std::ostream* type to a output_device& type.