当文件变大时，小文件的读写速度是否变慢？

Question

I am on Ubuntu 12.04 using ext4. I wrote a python program that does small size (mostly 512 byte) read and write with somewhat random access pattern. I found that as the file gets larger and larger. It takes more and more time to do the same number of I/Os. The relationship is linear. In other words, I get O(n ² ) where n is the cumulative number of I/Os.

I wonder if there is an inherent reason why small I/O being slower as file size increases.

One more observation: When I mounted a ramdisk and did my File I/O to the ramdisk I do NOT observe this performance degradation.

Answer 1

取决于您执行IO的方式，可能是您在保存之前试图过多地调用内存。

Answer 2

When you read 512 bytes out of 1024 bytes large file, the whole file is in cache. As the file size grows, smaller portion of file is in the cache, and data is read from the disk more and more frequently. Ie you get more cache misses as the file grows. Maybe this is what you are experiencing.