为什么我的程序在增加哈希表的大小时变慢

Question

I am using murmur hash to store 150,000 words in a hashtable I am using linear probing to resolve collisions in my program. I thought that if the size of my hashtable is large, then there will be a large number of free spaces, and I won't have to probe for a long time. But something strange happens. I got the fastest running time when the size of the hashtable was 250,000. After that the running time increases. Why does this happen?

Answer 1

While Robert covers the general issue (Locality) the issue is probably Spatial Locality .

When you have a smaller hash table, it fits into cache. When you have a very large hash table, each lookup runs a high risk of page fault. Should you page fault, then your operating system needs to pause the execution until the memory management unit can copy blocks from slower access memory to the caches that are closer to the CPU.

In extreme cases, the slower access memory might even be an on-disk resource provided by the operating system.

Answer 2

"Hash tables in general exhibit poor locality of reference—that is, the data to be accessed is distributed seemingly at random in memory. Because hash tables cause access patterns that jump around, this can trigger microprocessor cache misses that cause long delays. Compact data structures such as arrays searched with linear search may be faster, if the table is relatively small and keys are compact. The optimal performance point varies from system to system." - https://en.wikipedia.org/wiki/Hash_table