Clustering google map markers

Question

I would like to know what is the best way to cluster map markers ? We are currently using google maps in our application, where every entity defines a point on a map and have a grid based clustering algorithm (64px X 64px grid) for clustering the entity points. This worked fine for around 80k entities. We have datasets varying from 5000 to 1.5 million entities. The problem with 1.5 million dataset is that with so many markers, almost every grid on the map is a clustered marker, which not only results in a bad UI (a clustered marker at every grid), but also increases the load time for the map.

1) Client Side - I have tried client side clustering ( Map Clusterer ). Which didn't work, as the json object itself with 1.5 million entities slowed the system, also Map Clusterer worked only upto 50k markers.

2) Server Side - The problem with such huge datasets is that i cannot even query all the entities, so i was thinking of having a first level of grid based clustering in SQL and then have some density based clustering after getting the first pass clustered result from SQL with every cluster having a clusterCount and avg lat/long. Or if there is a better idea for such huge datasets, (I read about MAFIA, CLIQUE clustering http://www.siam.org/proceedings/datamining/2001/dm01_07NageshH.pdf ).

Also, at some point, i would really like to cache this data (although that should be asked in a separate question), but if someone could also give some caching technique keeping in mind the clusters, view port and zoom level, that'll be really helpful.

Answer 1

You can try my kmeans server-side solution:

https://github.com/biodiv/anycluster

(it is in python/geodjango/postgis)

If you have a truckload of cash, you can take a look at:

http://www.maptimize.com/

Answer 2

Have a look into GeoHash: http://en.wikipedia.org/wiki/Geohash

Could also use Solr for indexing spatial data: Search API21 and Apache Solr Search Integration22