[英]Java OutOfMemoryError with ArrayList<List<Integer>>
I want to create a very large graph (with ~10 million edges) in Java. 我想用Java创建一个非常大的图形(有大约1000万条边)。 I plan to
List<List<Integer>>
to describe the edges, with the inside List<Integer>
describing the two vertices of each edge (and the vertices are Integer type). 我计划
List<List<Integer>>
来描述边缘,内部List<Integer>
描述每条边的两个顶点(并且顶点是整数类型)。
The following code throws the OutOfMemoryError
after about 1 million edges are added to the graph. 在将约100万条边添加到图形后,以下代码抛出
OutOfMemoryError
。 (I simplified how the edge is generated for the sake of discussion.) (为了讨论起见,我简化了边缘的生成方式。)
public static void main(String[] args) {
List<List<Integer>> graph = new ArrayList<List<Integer>>();
for (int i = 0; i < 10000000; i++) {
List<Integer> edge = new ArrayList<Integer>();
// the real edges are more complicated (than from vertex i to vertex i+1)
// this is simplified for the sake of the discussion here
edge.add(i);
edge.add(i+1);
graph.add(edge);
}
}
I have searched for OutOfMemoryError
, and I have increased the initial heap size to 2G for Eclipse: -Xms2g -Xmx4g -Xss2m
(which get passed to JVM). 我搜索了
OutOfMemoryError
,并且我已经将Eclipse的初始堆大小增加到2G: -Xms2g -Xmx4g -Xss2m
(它被传递给JVM)。 But that did not solve the problem. 但这并没有解决问题。
Then I thought maybe I should garbage collect the List<Integer> edge
variable, by calling System.gc()
, in case its memory does not get cleared. 然后我想也许我应该通过调用
System.gc()
来垃圾收集List<Integer> edge
变量,以防它的内存没有被清除。 That did not work either. 那也行不通。
I was thinking maybe the problem is with the List<List<Integer>>
data structure. 我在想可能问题在于
List<List<Integer>>
数据结构。 I tried List<int[]>
, which lasted a bit longer: more edges are added before OutOfMemoryError
happens. 我尝试了
List<int[]>
,这持续了一段时间:在OutOfMemoryError
发生之前添加了更多边。 I do not have a better idea right now. 我现在没有更好的主意。
I have searched around for similar problems, but have not find much help. 我一直在寻找类似的问题,但没有找到太多帮助。 I wonder if anyone has experience with this kind of situation.
我想知道是否有人有这种情况的经验。
To let your program use more memory from Eclipse: 让程序使用Eclipse中的更多内存:
Go to Run -> Run Configurations. 转到“运行” - >“运行配置”。 You will see this window
你会看到这个窗口
Click on Arguments 单击Arguments
Enter your arguments to the VM 输入您的VM参数
Since you are using a lot of RAM besides setting the max heap parameter, make sure you use 64-bit Java. 由于除了设置max heap参数之外还要使用大量RAM,因此请确保使用64位Java。 The 32-bit is limited to 2 Gigs or something like that.
32位仅限于2 Gigs或类似的东西。
Also, for large graphs you should consider using a database. 此外,对于大型图表,您应该考虑使用数据库。
And last but not least, maybe you can rethink your algorithm, sometimes you just don't need all the nodes and edges. 最后但并非最不重要的是,也许您可以重新考虑您的算法,有时您不需要所有的节点和边缘。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.