Asynchronous thread-safe logging in C++

Question

I'm looking for a way to do asynchronous and thread-safe logging in my C++ project, if possible to one file. I'm currently using cerr and clog for the task, but since they are synchronous, execution shortly pauses every time something is logged. It's a relatively graphics-heavy app, so this kind of thing is quite annoying.

The new logger should use asynchronous I/O to get rid of these pauses. Thread-safety would also be desirable as I intend to add some basic multithreading soon.

I considered a one-file-per-thread approach, but that seemed like it would make managing the logs a nightmare. Any suggestions?

Answer 1

I noticed this 1 year+ old thread. Maybe the asynchronous logger I wrote could be of interest.

http://www.codeproject.com/KB/library/g2log.aspx

G2log uses a protected message queue to forward log entries to a background worker that the slow disk accesses.

I have tried it with a lock-free queue which increased the average time for a LOG call but decreased the worst case time, however I am using the protected queue now as it is cross-platform. It's tested on Windows/Visual Studio 2010 and Ubuntu 11.10/gcc4.6.

It's released as public domain so you can do with it what you want with no strings attached.

Answer 2

This is VERY possible and practical. How do I know? I wrote exactly that at my last job. Unfortunately (for us), they now own the code. :-) Sadly, they don't even use it.

I intend on writing an open source version in the near future. Meanwhile, I can give you some hints.

1. I/O manipulators are really just function names. You can implement them for your own logging class so that your logger is cout/cin compatible.
2. Your manipulator functions can tokenize the operations and store them into a queue.
3. A thread can be blocked on that queue waiting for chunks of log to come flying through. It then processes the string operations and generates the actual log.

This is intrinsically thread compatible since you are using a queue. However, you still would want to put some mutex-like protection around writing to the queue so that a given log << "stuff" << "more stuff"; type operation remains line-atomic.

Have fun!

Answer 3

I think the proper approach is not one-file-per-thread, but one-thread-per-file. If any one file (or resource in general) in your system is only ever accessed by one thread, thread-safe programming becomes so much easier.

So why not make Logger a dedicated thread (or several threads, one per file, if you're logging different things in different files), and in all other threads, writing to log would place the message on the input queue in the appropriate Logger thread, which would get to it after it's done writing the previous message. All it takes is a mutex to protect the queue from adding an event while Logger is reading an event, and a condvar for Logger to wait on when its queue is empty.

Answer 4

Have you considered using a log library.

There are several available, I discovered Pantheios recently and it really seems to be quite incredible.

It's more a front-end logger, you can customize which system is used. It can interact with ACE or log4cxx for example and it seems really easy to use and configure. The main advantage is that it use typesafe operators, which is always great.

If you just want a barebone logging library:

• ACE
• log4c*
• Boost.Log

Pick any :)

I should note that it's possible to implement lock-free queues in C++ and that they are great for logging.

Answer 5

I had the same issue and I believe I have found the perfect solution. I present to you, a single-header library called loguru: https://github.com/emilk/loguru

It's simple to use, portable, configurable, macro-based and by default doesn't #include anything (for that sweet, sweet compilation times).