what is the most effective way to persist HashMap?

Question

I have a Hash Map (many-to-one relationship between texts and boolean values):

name         flag
---------------------
"abc"        TRUE
"cde 12"     TRUE
"foo"        FALSE
"some text"  TRUE
etc...

I need to persist this structure in Java application. This structure will be extended, but never changed. It means that I will add new records to it but never delete nor change existing ones. What is extremely important is the speed of search (I provide a name and it returns the flag ). It may be really big in size (millions of records). There are a number of options I consider: 1) relational database with one table and one index, 2) flat file(s), 3) pure JVM database.. What would you suggest?

Answer 1

I'd recommend JDBM3 library which provides a disk backed HashMap and TreeMap implementations. Its fast, scalable and easy to use. Apache 2 license.

From the site:

JDBM has outstanding performance; it can insert a million records per second and read them even faster

Answer 2

Given then number of records I would go with a relational db keyed on the name.

But what should it mean if a name is not found?

If not found is equivalent to one of your boolean values (say TRUE) then you have a whitelist (or blacklist depending on context) in which case I would be inclined to drop the flag column from the database and cache the names in a hash set.

If not found is a separate value then if you have sufficient available memory you might try caching the whole table in a hash map.

Answer 3

How many millions? We had a case with an array of 5mil records and we had 'em all in memory sucked up to get fast results using binary search it was the geoloc data with longitutes, latitiudes data and the DB search was taking quite awhile if you had many addresses to lookup. If you have lots of memory use in-memory array if not - use a small DB either sqlite or mysql. sqlite can handle large amounts of data too and doesn't require an extra server if there not many threads clients which will update your structure. about sqlite's limits - Can SQLite handle 90 million records?