When is the file loaded into memory - for fread, fopen and fwrite calls?

Question

When I do a fopen and then a fread, when is the file actually/partially loaded in the memory during fopen or fread?

Or is it partially loaded at fopen based on size of file and then fully loaded at time of fread?

Similarly what happens internally at the OS level when fwrite is called? Is the file loaded into memory at that time, or a page swap happens retriving just that part of file in memory?

What happens at the OS level at each of these calls with respect to file loading in memory?

Answer 1

• fopen() only creates a handle to the file.
• fread() actually reads the file into a memory buffer (OS-level buffering may occur transparently to the client.)
• fwrite() writes data into the file, though its committing to the storage may get delayed (eg with journalled filesystem.)

Answer 2

Typically, the file is not loaded into memory upon opening it. Instead, parts are loaded in for each read; due to all kinds of buffering, greater chunks may be loaded then you ask for in each fread .

When you fwrite some data, it is eventually copied into the kernel which will then write it to disk (or wherever) after buffering. In general, no part of a file needs to be loaded in order to write.

Answer 3

Generally it depends on the file system and OS. in windows there is a caching mechanism which deals with a file in 256KB chunks and loads each chunk upon read request falling in that chunk. A call to fopen should not cause reading the file content from media. And fread will cause partial read (or complete read for small files) from the media. Partial read usually is equal to cache line size in cache manager (256KB).

fwrite also may/may not cause a actual write to the media. It usually causes client data to be transferred to the cached file area in RAM, but there is no guaranty that data actually is written to media. in Windows, cache manager decides when to flush a cached area of a file to media. If you want to make sure all dirty data is flushed to media after fwrite , you need to call fflush afterwards.

Answer 4

While this is OS-dependent, in modern operating system all disk activity is transparenly cached, so that when you open a file in reality it is mapped to a portion of the virtual memory space.

This mean that no disk activity occur before the actual reading/writing.

This is true even if you open the file without memory-mapping (eg: fopen), while if you open it with memory-mapping (eg: mmap) you just lose the "transparency".