What is the correct way to write data to a file using worker thread in Qt?

Question

I am developing an application in Qt, which is UI intensive. It is UI intensive because, my application displays logs,which come at a very high speed and UI has to reflect that.

Now after the number of logs exceed a certain amount, My previous logs will start to get deleted, because my UI window has a limit(100000 logs, to keep app fast).

So in order to save the old logs , I want to write the old logs to a file, before they get deleted.

Problem

If I write the file in main thread,my UI hangs(becomes very slow). So I decided to write file in a worker thread. This is what I did this:

I made my own class WorkerThread that inherits class QThread and inside that class run() method, I write the data to a file.

The data that I want to write is stored in threads member variables itself:

So my code is:

Some other class function . . . .

WorkerThread *workerThread = new WorkerThread();
connect(workerThread, SIGNAL(resultReady()), workerThread, SLOT(quit()));
workerThread->attribute1 = dataToWrite1;
workerThread->attribute2 = dataToWrite2;
workerThread->start();

WorkerThread class

class WorkerThread : public QThread
{
Q_OBJECT

public:

  QString attribute1;
  QString attribute2;  

protected:
    void run() {    
            // DELIMITER IS ..:  //                 

            QFile myFile("C:/shiftedlines/myFile.txt");
            if(myFile.open(QIODevice::WriteOnly|QIODevice::Append))
            {    
                QTextStream stream(&myFile);
                stream<< attribute1<<"..:";
                stream<< attribute2<<"\n";   
            }
        emit resultReady();
    }
signals:
    void resultReady();
};

But after writing about 500 lines, my application crashes . How do I go about solving this problem?

Answer 1

This does not address the issue you are having but since you asked this in the comments, this is how I implemented such tasks in the past.

worker class

class worker : public QObject
{
    Q_OBJECT

    public:
        worker();

    public slots:
        void start();

    signals:
        void finished();
};

main

worker* cur_worker = new worker();

// move to its own thread
QThread* worker_thread = new QThread(this);
cur_worker->moveToThread(worker_thread);
QObject::connect(worker_thread, SIGNAL(started()), cur_worker, SLOT(start()));
QObject::connect(cur_worker, SIGNAL(finished()), worker_thread, SLOT(quit()));
QObject::connect(worker_thread, SIGNAL(finished()), worker_thread, SLOT(deleteLater()));

// start
worker_thread->start();

This example does not handle the destruction of the worker object, which in your case you probably want to do immediately.

Answer 2

Writing to a file should be quite fast as long as it's done correctly.

What is slow is not writing to the file, but opening it.

Make sure you don't open / close the file each time you receive a log, but open the file in write only and append mode, and keep it open while your application is running and writing log.

Usually you can write in recent SSD drive at a speed around 700MB/s of data, so I doubt that you would have any noticeable impact on the UI.

---

If you really want to go for thread (which IMO is probably overkill) don't forget that to be thread safe, you have to communicate with your thread using signals / slots.

From your code, it seems you start a new thread for each log write, which each tries to open the file. Your file can be written only by one thread at a time and this will be blocked by the system, so above 1 thread for writing into a file, it is useless to have more.

Probably your threads hangs into trying to open the file which is already accessed by some other, and end up with hundreds of hanging threads which ends up in a memory overflow if you have them in the stack.

The roule of Open your file once and write as long as you have something to write also applies in the thread.