Are open/read/write buffered or not?

Question

Only open/read/write functions are used in my program, but oprofile shows me that:

20537     2.9883  tyn_indexer              tyn_indexer              nodes_term32_flush
11966     1.7411  vmlinux                  vmlinux                  jbd2_journal_commit_transaction
11733     1.7072  vmlinux                  vmlinux                  __strnlen_user
10741     1.5629  vmlinux                  vmlinux                  nobh_truncate_page
9728      1.4155  vmlinux                  vmlinux                  generic_file_buffered_write
9443      1.3740  vmlinux                  vmlinux                  mpage_da_map_and_submit
9023      1.3129  vmlinux                  vmlinux                  do_get_write_access
7283      1.0597  vmlinux                  vmlinux                  invalidate_interrupt31
5894      0.8576  vmlinux                  vmlinux                  write_cache_pages_da
5332      0.7758  vmlinux                  vmlinux                  journal_submit_commit_record
5316      0.7735  vmlinux                  vmlinux                  hugetlbfs_symlink

How generic_file_buffered_write comes out, and are open/read/write functions buffered?

Answer 1

It depends on what you mean by buffering. When C programmers say these functions are unbuffered, what it means is that there is no buffer local to the application process that could prevent other processes from seeing data in a consistent fashion. The implementation of these functions (in the kernel) is free to do as much or as little buffering as it likes as long as it's done in a way that's transparent to application processes (ie does not interfere with whether or when they see data on the file).

On the other hand, stdio is (usually) buffered, meaning that if you use stdio (the FILE * functions from stdio.h ) to access a file, data may be read from the underlying file descriptor and buffered in your process's memory space where other processes cannot see it before you actually intend to read and use it , or data you have written might remain buffered in your process's memory space where other processes cannot see it long after you wrote it (unless you call fflush ).

Answer 2

The read and write system calls have quite arbitrary semantics, which depend entirely on the device that you are interacting with.

The read function can do diverse things. For instance, if you're reading from a TTY driver in cooked mode ("canonical input processing"), and ask for 1 byte, read will not return that one byte until a complete line is actually available. But even in a raw mode, of course there are still buffers. Data is received in an interrupt-driven fashion from the keyboard or serial port even while your process is not calling read , and sits in a buffer.

The read function can discard data. If you're reading sectors from a traditional Unix raw tape device, or datagrams from a datagram socket, and the buffer you're supplying is too small to hold an entire sector or packet, the excess data is truncated.

write to a file can return long before the data actually goes out to disk.