Can we create separate log files for each thread in boost library

Question

I want to know that can we create separate log files for each thread with executing some function or by using add_file_log function.

The following program creates number of log files based on number of threads created. But we are executing same set of codes for same number of times. It may result in slow down of application for larger number of threads.

#include <iostream>
#include <boost/move/utility.hpp>
#include <boost/log/sources/logger.hpp>
#include <boost/log/sources/record_ostream.hpp>
#include <boost/log/sources/global_logger_storage.hpp>
#include <boost/log/utility/setup/file.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>
#include <cstring>
#include <stdlib.h>
#include <boost/thread.hpp>
#include <boost/date_time.hpp>
namespace logging = boost::log;
namespace src = boost::log::sources;
namespace keywords = boost::log::keywords;
int count = 0;
class class_logging {
 public:
  void logging_function() {
    char filename[15] = "sample";
    char extension[5] = ".log";
    int c = count++;
    char num[10];
    std::sprintf(num, "%d", c);
    std::strcat(filename, num);
    std::strcat(filename, extension);
    logging::add_file_log(filename);
    logging::add_common_attributes();
    src::logger lg;
    logging::record rec = lg.open_record();
    if (rec) {
      logging::record_ostream strm(rec);
      strm << "Count Value :" << c;
      strm.flush();
      lg.push_record(boost::move(rec));
    }
  }
};
int main(int, char* []) {
  class_logging object[100];
  int total;
  std::cout << "\nEnter no. of threads to produce : ";
  std::cin >> total;
  for (int i = 0; i < total; i++) {
    boost::thread thread1(&class_logging::logging_function, &object[i]);
    std::cout << "\nThread " << i
              << " is created whose id is : " << thread1.get_id();
    thread1.join();
    std::cout << "\nThread " << i << " is done...";
    thread1.detach();
  }
  return 0;
}

Q-1) Are there any way to create log files dynamically?

Answer 1

In essence there is nothing wrong with writing logs to separate files given the individual (here threads) tasks being performed. If the volume is too much, reducing the logging would help more than just reducing the log file count.

Another strategy is to dedicate a thread to logging. All logging messages are then sent to that thread/task for processing. It can be implemented using the producer/consumer pattern.

Adding a timestamp to the log records is also a good idea. It will help with any sequencing issues or logs you may need to deal with.

---

It looks like you have a race condition on the count , in this line of code: int c = count++; The increment is not atomic.

I would either pass a "count" into the thread on its creation, or modify the count variable to be atomic (using one of the std::atomic types ).

std::atomic<int> count = 0;

Answer 2

I want to know that can we create separate log files for each thread with executing some function or by using add_file_log function.

I'm not familiar to this boost lib. if you want a variable local to a thread you can use thread_local see this page

The following program creates number of log files based on number of threads created. But we are executing same set of codes for same number of times. It may result in slow down of application for larger number of threads

True and False: threads are not slow down by executing the same code. Threads are slowing down each other by sharing resources. The main resources coming up to my mind are cpu cores and memory.

1. Memory: Indeed threads can be slow down when they use (once read/one write or both write) the same variable in memory (even without lock/mutex). you can read this " eliminate false sharing "
2. Cpu: Threads are sharing the cpu, only one thread at a time can run on one core. So if there are mores threads actively running, than cores on your computer, they are slowing down each other. OS has to stop execution of one to let next one running for a moment, etc... see scheduling , see context switch

Q-1) Are there any way to create log files dynamically?

In your code you're sharing int count you should use:

std::atomic<int> count = 0;
...
int c = count.fetch_add(1); // this is thread safe
int c = ++count; // or this

You don't have to call thread1.detach(); at then end of you code. after thread1.join(); the thread1 is stopped already.

Answer 3

Yes, it is possible to create separate log file for each by giving separate name likethis,

void ownlogger::start_logger(LPSTR filename, LPSTR file_location)
{
    logging::add_file_log
    (
        keywords::file_name = filename,         /*< file name pattern >*/
        keywords::time_based_rotation = sinks::file::rotation_at_time_interval(boost::posix_time::seconds(6)),
        keywords::format = 
        (
            expr::stream
            //<< std::hex   //To print the LineID in Hexadecimal format
            << std::setw(8) << std::setfill('0') 
            << expr::attr< unsigned int >("LineID")
            << "\t"
            << expr::format_date_time<boost::posix_time::ptime>("TimeStamp","%H:%M:%S.%f")
            << "\t: <" << logging::trivial::severity
            << "> \t" << expr::smessage
        )
    )->locked_backend()->set_file_collector
        ( 
            sinks::file::make_collector
            ( 
            keywords::target = file_location, 
            keywords::max_size = 5 * 1024 * 1024 // just for test limit to 5M
            )
        );

    logging::add_common_attributes();
    WRITE_TO_LOG << "Logger Starts";    
}

And here filename (ieargument) may be given different for each time.