用图表处理大量数据

Question

I have a crawler which crawl only one website. But a website can have thousand pages. Each page is a vertex, I use JGraphT for that.

But If I crawl a huge website, I get a StackOverflowError after a moment.

How can I do to handle a lot of data in this case ? I have to use a database ? There is a library which handle this specific kind of data (vertex, edge,...) in a database ?

Thanks

Answer 1

您可能想要使用Neo4j ，它是一个高性能的 NOSQL 图形数据库。

Answer 2

you can take a look of following may its helps:

1. bigdata®
2. TITAN