Storing large, static data set

Question

I am making a game which has map data stored as tiles. These tiles will not be changed more than once a release, as the data that generates them is versioned, so they are as static as the server code.

Since they are static, I thought about using a SQLite database in ram memory to store them. However, with the number of tiles in the game (could get a map up to 10,000x10,000), it could easily end up taking gigabytes of ram to store if it was kept in memory (estimated at 15gb with a 10,000x10,000 map).

I also considered creating a sqlite db and just versioning that along with the rest of the code. But I am worried about concurrent reads.

However, I feel like using a postgres database is overkill for this purpose, since the data won't change.

What would be the accepted way of dealing with a data issue like this?

Answer 1

It sounds like you may want to store it in something like S3 (basically, use files, but offloaded to a service so your app can scale horizontally).

You can manage S3 directly or via an addon like https://elements.heroku.com/addons/filepicker

Answer 2

我最终使用 Redis 作为这个问题的通用解决方案，特别是 Heroku 中的 Redis Cloud 应用程序，因为它允许在免费层使用。