How to output program logs to SD card without causing damage

Question

I am working on an application that runs on a small Linux computer with an SD card for storage. The application runs automatically on startup and we want to be able to easily check the logs that it produces. Normally I would just write to a file, since that also seems to be what most normal software would do. But I am hesitant about doing this because I think continuously writing logs is a bad idea because of the SD card for storage.

The problem is that sometimes when we want to check what is happening on the system, say for debugging purposes, we have stop the application via SSH and then start it again so that we can see the output messages.

So my question is: is there a way to say write logs to some kind of circular list that can then be viewed when connecting to the system over SSH? The application is written in C and C++ if that matters.

Answer 1

Is your application on a Raspberry Pi?

The Linux Operating system, and all other technology, is probably writing so much to the SD card, that your 500 KB/ hour would be next to nothing in comparison.

I would personally just have the program log to the file.

If you really do not want this, you have a few other options:

• Have the application send the logs via the internet to some service, which you can then monitor
• have your application store the logs in a buffer in-memory, and then write to file when you reach some threshold. Expose an endpoint on localhost which listens for a message, and when received, writes the in-memory contents to the file. This allows you to see log-files for current in-memory logs without having to wait.

Answer 2

First thing, I think that the SD driver care about writing and about I/O operation scheduling them in the better way for the safety of the SD card itself (using a virtual filesystem). Maybe you can work on your log level to be sure to write the necessary information and nothing more.

Answer 3

Based on shared inputs SD card wear-out might not happen easily, however, there are many ways to handle this scenario based on the hardware and software architecture of your system :

1. Check if you can write the logs to some other storage device within your system (Depends on your architecture).

2. If external communication peripherals are available in your system, check if logging can be done by redirecting the logs to remotes servers or other devices.

3. Perform selective logging based on some log-level as per your architecture / framework. Also, you can do only critical logging in SD card and the other logs can be re-directed based on your architecture. This can reduce the number of writes.

4. Based on your need/architecture, check if the data can be compressed and logged. This can reduce the number of times, the logs are written to SD card.

To continue working and simultaneously view the logs :

1. Based on your architecture/need, check if you can write to a file periodically or based on threshold so that you can view the file irrespective of operation.

2. Send selective logs to external server / device