sharding a database with hierarchical data structure

Question

I am using MySQL to store my parent-child relation. The data is in one table.

I do not have any problem quering; however, the table has grown in size dramatically. I would like to shard or use some techniques to improve the performance of my queries (and joins). How would I do it?

Answer 1

There is not enough information in this post to give you a good answer. The first thing we would need to know is what the table structure is. Do you have non-clustered indexes defined or is it simply one clustered index.

Your application that uses this database, what does it do with the data?

In addition, are you searching for say a name field, or a specific date? If so these could be candidates for an index type.

The number one thing to look after though is indexes .

Answer 2

It is better to use professional sharding middleware, for example: Apache ShardingSphere

It can use simple way to shard databases and tables by SQL route and rewrite without join cross databases. It can use complex way to handle join cross databases with federation query powered by Apache Calcite.

Answer 3

I think you will need more advanced help if your product is growing that large. dbShards is a good place to start. They will help you design a sharding strategy for your data and can easily shard any tables that need it. If your one table is getting too large, sharding is not a bad choice. I would check out dbShards. Might work well for this situation.