写入后从磁盘读取而不是缓存

Question

Here's the task I'm trying to perform on a linux host, with a C program:

Write random data to the disk, call fysnc() to flush data to disk, then read back what was written from the disk to ensure the disk controller wrote the data correctly. The problem I am running into is that reads appear to be answered by server-side caching rather than from the device itself. Here's what I've already tried:

 1. O_DIRECT (a gigantic pain in the butt, abandoned)
 2. posix_fadvise(fd,0,0,POSIX_FADV_DONTNEED)
 3. posix_fadvise(fd,0,0,POSIX_FADV_NOREUSE)
 4. O_SYNC
 5. O_ASYNC

In every case, iostat shows 0 rrqm/s and thousands of write requests. I could be a woefully uninformed linux user, but it is my belief that if no IOs are shown in rrqm/s then reads are being answered by the OS cache instead of the device itself.

"Why not use iozone or iometer, or any of the billions of other tools that already stress disks?" Well, to be honest, if HP-UX's HAZARD worked on anything except HP-UX, I would, but nothing else comes close to what hazard can do, so I'm making my own.

Answer 1

You need to do the equivalent of the following shell commands:

sync                               # Instruct all data to get flushed to disk
echo 3 > /proc/sys/vm/drop_caches  # Instruct VM system to clear caches

and then try reading the file again.

One way to do it from C would be something approximating:

sync();
int fd = open("/proc/sys/vm/drop_caches", O_WRONLY|O_TRUNC)
write(fd, "3\n");
close(fd);

Answer 2

You should not go thru the file system to test a disk. You should read and write the raw partitions (eg /dev/sdc5 )

On most current Linux systems and hardware, disks have a SMART interface. You should use it, see smartmontools and study its source code. (I guess that there are some ioctl(2) related to that.)