How to write a file atomically and asynchronously in Linux using C

Question

Question

How can I update a file atomically without requiring my program to wait for slow physical media (such as with fsync )?

My hope is that the OS could "buffer" the typical fsync and rename operations in RAM, and then write them to disk in the proper order whenever it is convenient.

---

Background

I am developing software that runs in a custom embedded Linux environment with an ext4 filesystem. The program makes periodic updates to a file on disk. I need to maintain the integrity of this file without sacrificing application performance.

From what I have read , the accepted practice for safely updating a file is as follows:

1. create a new temp file
2. write data to the temp file
3. fsync() the temp file
4. rename the temp file to the appropriate name
5. fsync() the containing directory

This process makes sense to me, but in my particular application, I would like to avoid a blocking call to fsync() . I don't care when the data is written to disk, so long as the file is always in a valid state. If the file is out-of-date, that is OK.

---

What I've Learned So Far

It seems that there is already quite a bit of discussion around ext4 and the proper use of fsync . If I understand correctly, I might be able to forgo the use of fsync if auto_da_alloc is enabled for my filesystem ( link ), but I'm not convinced that is the best solution.

Answer 1

I've found that Linux file systems are remarkably resilient and reliable. I'd be suspicious discussions of ext4 issues that are mostly from 2009. Linux info goes stale.

That said, if the requirement is to guarantee that you use the latest correct version of the file, add a step to check that the file is correct. One way is to use Linux tools to generate a hash of the file. Maybe there's a faster approach that works with your specific data, like looking for a specific file length.

With that in place, your algorithm could be:

1. Rename the current file to indicate that it is an old copy.
2. Write the new file
3. Calculate the hash of the new file

Now when you boot, or whenever you use the file, you can check the hash and if that fails, revert to the latest old copy.

The advantage to this plan is that the hash will protect you from any source of corruption to the file, not just ext4 file system issues.