Which database engine is suitable for the 10^10 records?

Question

I am planning to use mysql to store my datasets. I have about 10^8 (hundred million) records: ID(int) , x(float) , y(float) , z(float) , property(float) .

Which database engine is suited for this kind of data-sets InnoDB or MyISAM? Or maybe ndb (I have no idea on scalability or performance)?

I am planning to query the static dataset with following questions:

Select getRectagularRegion or getPointsInSphere ;

Answer 1

I am assuming you are trying to store points in 3d space and then find all points within a region.

How the underlining database codes with a lot of records is a lot less important to you then having a very good 3d spatial indexing system build in to the database. Without the spatial indexing you can't do the queries with wish.

You should also consider writing your own data storage as a simple 3rd quod tree may give you good indexing depending on how your points are clustered – but using a "off the self" database would be less work for you.

So I think you need to investigate support for spatial indexing in databases, rather than ask about support for lots of rows. Storing lots of rows is a given for most databases these days…

Answer 2

Your table seems to be pretty simple and you won't need transactions and foreign keys. So I guess MyISAM would be better suited than InnoDB. But I guess MEMORY might be your fastest choice.