为什么LMDB数据库占用的数据超过实际数据？

Question

I put around 11K key&values in LMDB database . LMDB database file size become 21Mb.

For the same data the leveldb is taking 8Mb only (with snappy compression).

LMDB env info ,

VERSION=3
format=bytevalue
type=btree
mapsize=1073741824
maxreaders=126
db_pagesize=4096

TO check why LMDB file size is more ,I iterated through all keys & values inside the database. The total size of all key & value is 10Mb.

But the actual size of the file is 21Mb. Remaining file size of 11Mb (21Mb - 10Mb) used for what purpose???!!.

If i compress data before put operation ,only 2Mb got reduced

Why LMDB database file size is more than actual data size?

Any way to shrink it ?

Answer 1

The database is bigger than the original file because lmdb requires to do some bookeeping to keep the data sorted. Also, there is an overhead because even if your record (key + value) is say 1kb lmdb allocates a fixed size of space to store those. I don't know the actual value. But this overhead is always expected.

Compression doesn't work well on small records.

lmdb doesn't support prefix or block compression. Your best bet is to use a key-value store that does, like wiredtiger .