简体   繁体   English

如何在Java中管理堆空间

[英]How to manage Heap Space in Java

I'm trying to recursively traverse through my Drive to search some files. 我试图递归遍历我的驱动器来搜索一些文件。 The code is working fine when there are limited folders/files but when I target my search to C drive where in I have lots of files it throws Out of Heap memory. 当有限的文件夹/文件时,代码工作正常,但是当我将搜索目标指向C驱动器时,我有很多文件,它会抛出堆内存。

Exception in thread "Thread-4" java.lang.OutOfMemoryError: Java heap space 线程“Thread-4”中的异常java.lang.OutOfMemoryError:Java堆空间

  1. Please suggest me some good memory management tricks especially when we do recursive calls. 请给我一些好的内存管理技巧,特别是当我们进行递归调用时。
  2. Or give me better approach to traverse through directories without recursion. 或者给我更好的方法遍历目录而不递归。

and I don't want to increase the maximum allowable heap space as it is just like postponing the issue for time being. 而且我不想增加最大允许堆空间,因为它就像暂时推迟问题一样。

Code: 码:

void iterateDirectory(String somedir) {


        File dir = new File(somedir);
        File[] files = dir.listFiles();
        if (files != null) {
            for (int id = 0; id < files.length; id++) {
                if (files[id].isDirectory() == false) 
                {
                   fsTree.add(files[id].toString()); // taking list of files
                } 
                else 
                {
                    iterateFilesInDirectory(files[id].getAbsolutePath());
                }
            }
        }
    }

The culprit, as I see it is this line: 罪魁祸首,正如我所看到的那样:

fsTree.add(files[id].toString()); // taking list of files

It appears that you add every single file to a global data structure ( fsTree ), and then search there. 您似乎将每个文件添加到全局数据结构( fsTree ),然后在那里搜索。

My bet is: 我的赌注是:

A. It won't go away if you 'convert' your recursive function into an iterative one. 答:如果你将你的递归函数转换为迭代函数,它就不会消失。

B. It will go away if, instead of appending to a global data structure and searching in the end, you do the search/matching locally, and only globally cache the matching hits: B.如果不是追加到全局数据结构并最终搜索,而是在本地进行搜索/匹配,而只是全局缓存匹配的匹配,它将会消失:

void iterateDirectory (String somedir, String search_term) {

    File dir = new File(somedir);
    File[] files = dir.listFiles();
    if (files != null) {
        for (int id = 0; id < files.length; id++) {
            if (files[id].isDirectory() == false) 
            {
               if (/* files[id].isDirectory() MATCHES search_term */)
                 // add to list of matching files:
                 matching_hits.add(files[id].toString());
            } 
            else 
            {
                iterateFilesInDirectory(files[id].getAbsolutePath());
            }
        }
    }
}

There are two possibilities: 有两种可能性:

  1. There is infinite recursion in your code (for example, because you're not processing . and/or .. correctly). 代码中存在无限递归(例如,因为您没有正确处理.和/或.. )。 If that's the case, you have to fix the code. 如果是这种情况,您必须修复代码。
  2. Your code genuinely requires more heap space than what's available. 您的代码确实需要比可用的更多的堆空间。 You have two options: 您有两种选择:
    • reduce you process's memory requirements (a memory profiler could help you understand what's using all that heap space); 减少进程的内存需求(内存分析器可以帮助您了解使用所有堆空间的内容);
    • increase the heap size by specifying the -Xmx JVM option. 通过指定-Xmx JVM选项来增加堆大小。

There are limitations as to what recursion can do, namely with respect to stack/heap usage. 递归可以做什么,即堆栈/堆使用方面存在限制。 Remember that whatever you can do recursively, you can do iteratively; 请记住,无论你以递归方式做什么,你都可以迭代地做; rewrite your code to use an iterative solution instead. 重写代码以使用迭代解决方案。

Alternate solution: There is an interface in java.nio that can be used to recursively walk down a filesystem's structure. 替代解决方案: java.nio中有一个接口,可用于递归地向下走一个文件系统的结构。 Take a look at this trail , and SimpleFileVisitor . 看一下这条路径SimpleFileVisitor

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM