如何在使用遞歸逐步執行大型目錄結構時管理Java內存

Question

我有一個遞歸方法，遍歷包含數千個音樂文件的大型目錄。 每次擴展符合條件時，它都會將音樂文件添加到observableList <>。 在遞歸方法執行之前，該列表被掛接到另一個線程中的TableView <>，以便用戶可以實時查看正在添加到TableView <>的文件。

問題是我對如何在java中管理內存知之甚少，並認為我可能會妨礙垃圾收集。 在大約3,000首歌曲之后，遞歸方法會占用近6 GB的內存，然后開始忽略它應該能夠讀取的文件。 此外，在“完成”逐步通過目錄結構之后，ram不會減少，（即，遞歸方法的堆棧沒有被破壞，我認為所引用的所有對象仍然在堆內存中）。

它更進一步..我將播放列表導出到XML文件並關閉程序。 當我重新啟動它時，內存是完全合理的，所以我知道它不是包含文件的大型列表，它必須與遞歸方法有關。

這是位於音樂處理程序中的recusive方法：

 /**
 * method used to seek all mp3 files in a specified directory and save them
 * to an ObservableArrayList
 * 
 * @param existingSongs
 * @param directory
 * @return
 * @throws FileNotFoundException
 * @throws UnsupportedEncodingException
 */
protected ObservableList<FileBean> digSongs(ObservableList<FileBean> existingSongs,
        File directory) throws FileNotFoundException,
        UnsupportedEncodingException {
    /*
     * Each directory is broken into a list and passed back into the digSongs().
     */
    if (directory.isDirectory() && directory.canRead()) {

        File[] files = directory.listFiles();
        for (int i = 0; i < files.length; i++) {
            digSongs(existingSongs, files[i]);
        }

        /*
         * if a file is not a directory, then is it checked to see if it's
         * an mp3 file
         */
    } else if (directory.getAbsolutePath().endsWith(".mp3") 
            || directory.getAbsolutePath().endsWith(".m4a")
            ) {
        FileBean songBean = new FileBean(directory).getSerializableJavaBean();

        existingSongs.add(songBean);

        songBean.getPlayer().setOnReady(new OnMediaReadyEvent(songBean));
        songBean.getPlayer().setOnError(new OnMediaPlayerStalled(existingSongs, songBean));

        /*
         * if it's not a directory or mp3 file, then do nothing
         */
    } else {

        return existingSongs;

    }

    return existingSongs;
}

如果可能的話，這是用於讀取thr ID標簽的MediaPlayer的監聽器，它也位於音樂處理程序中

/**
 * This class will populate the FileBean metaData after the MediaPlayer's
 * status has been changed to READY. Uses the FileBean's setter methods so
 * that they will be picked up by the XMLEncoder. This allows the use of the
 * Media's ID3v2 tag reading abilities. If tags are not read due to
 * incompatibility, they are not changed.
 * 
 * This step is computationally expensive but should not need to be done
 * very often and it saves a ton of memory during normal use. Setting the 
 * Media and MediaPlayer objects to null make this run much faster and uses
 * less memory
 * 
 * @author Karottop
 *
 */
protected class OnMediaReadyEvent implements Runnable {
    private FileBean fileBean;

    public OnMediaReadyEvent(FileBean fileBean) {
        this.fileBean = fileBean;
    }

    @Override
    public void run() {
        String songName = null;
        String album = null;
        String artist = null;
        double duration = 0.0;
        try{
            // Retrieve track song title
            songName = (String) fileBean.getMedia().getMetadata()
                    .get("title");

            // Retrieve Album title
            album = (String) fileBean.getMedia().getMetadata()
                    .get("album");

            // Retrieve Artist title
            artist = (String) fileBean.getMedia().getMetadata()
                    .get("artist");

            // Retrieve Track duration
            duration = fileBean.getMedia().getDuration().toMinutes();
        }catch(NullPointerException e){
            System.out.println(e.getMessage());
        }
        // Set track song title

        if (songName != null)
            fileBean.setSongName(songName);

        // Set Album title

        if (album != null)
            fileBean.setAlbum(album);

        // Retrieve and set Artist title

        if (artist != null)
            fileBean.setArtist(artist);

        // Set Track duration
        fileBean.setDuration(Double.parseDouble(
                XMLMediaPlayerHelper.convertDecimalMinutesToTimeMinutes(duration)));

        fileBean.setMedia(null);
        fileBean.setPlayer(null);

    }

}

這是我在控制器中為FXML調用方法的地方：

    public class LoadAllMusicFiles implements Runnable{

    private TableView<FileBean> tableView;

    public LoadAllMusicFiles(TableView<FileBean> tableView) {
        this.tableView = tableView;
    }   

    @Override
    public void run() {
        try {
            musicHandler.loadAllPlaylists();
            tableView.setItems(musicHandler.getMainPlaylist().getSongsInPlaylist());
            playlistTable.setItems(musicHandler.getPlaylists());

        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (NoPlaylistsFoundException e) {
            String title = "Mine for mp3s";
            String header = "No playlists were found.\n"
                    + "These are your mp3 mining options...";
            String content = "Do you want to import a single mp3\n"
                    + "or a folder containing many mp3s?\n\n"
                    + "**Note For large volumes of songs this may take a while.\n"
                    + "Grab some coffee or something..**";
            findNewSongs(title, header, content);
            // need to handle file not found exception in new thread
            tableView.setItems(musicHandler.getMainPlaylist().getSongsInPlaylist());
            playlistTable.setItems(musicHandler.getPlaylists());
            Platform.runLater(new SelectIndexOnTable(playlistTable, 0));
            tableView.getSelectionModel().selectFirst();

        }

    }

}

/**
 * The method will display an Alert box prompting the user to locate a 
 * song or directory that contains mp3s
 * 
 * The parameters passed is the text the user will see in the Alert box.
 * The Alert box will come with 3 new buttons: 1)Single mp3, 2)Folder of mp3s
 * and 3)Cancel. If the user selects the first button they will be
 * presented with a FileChooser display to select a song. If they press
 * the second button, the user will be prompted with a DirectoryChooser
 * display. The third button displays nothing and closes the Alert box.
 * 
 * The following outlines where each parameter will be displayed in the
 * Alert box
 * 
 * title: very top of the box in the same latitude as the close button.
 * header: inside the Alert box at the top.
 * content: in the middle of the box. This is the best place to explain
 * the button options to the user.
 * @param title
 * @param header
 * @param content
 */
private void findNewSongs(String title, String header, String content){
    Alert importType = new Alert(AlertType.CONFIRMATION);
    importType.setTitle(title);
    importType.setHeaderText(header);
    importType.setContentText(content);

    ButtonType singleMp3 = new ButtonType("Single mp3");
    ButtonType folderOfmp3s = new ButtonType("Folder Of mp3s");
    ButtonType cancel = new ButtonType("Cancel", ButtonData.CANCEL_CLOSE);
    importType.getButtonTypes().setAll(singleMp3, folderOfmp3s, cancel);

    Optional<ButtonType> result = importType.showAndWait();
    if(result.get() == singleMp3){
        FileChooser fileChooser = new FileChooser();
        fileChooser.setTitle("Location of mp3s");
        ArrayList<String> extensions = new ArrayList<>();
        extensions.add("*.mp3");
        fileChooser.getExtensionFilters().add(
                new ExtensionFilter("Audio Files", getSupportedFileTypes()));

        File selectedFile = fileChooser.showOpenDialog(playBackButton.getScene().getWindow());

        if(selectedFile == null){
            return;
        }
        Thread findSongs = new Thread(new DigSongs(selectedFile.getAbsolutePath()));
        findSongs.start();

    }else if(result.get() == folderOfmp3s){
        DirectoryChooser fileChooser = new DirectoryChooser();
        fileChooser.setTitle("Location to mine for mp3s");

        File selectedFile = fileChooser.showDialog(playBackButton.getScene().getWindow());

        if(selectedFile == null){
            return;
        }
        Thread findSongs = new Thread(new DigSongs(selectedFile.getAbsolutePath()));
        findSongs.start();

    }else{
        return;
    }
}

public class DigSongs implements Runnable{
    String path;

    public DigSongs(String path) {
        this.path = path;
    }
    @Override
    public void run() {
        Platform.runLater(new UpdateLabel(digLabel, "loading..."));
        try {
            musicHandler.findNewSongs(path);

        } catch (FileNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (UnsupportedEncodingException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        ObservableList<FileBean> songArray = musicHandler.getMainPlaylist().getSongsInPlaylist();
        Platform.runLater(new UpdateLabel(digLabel, "complete: " + songArray.size()));
    }

}

此方法位於音樂處理程序中，基本上只調用遞歸方法digSongs（ObservableList，File）：

/**
 * This method will search for songs in a new directory and add them to the song list
 * in the main playlist
 * @param newDirectory
 * @return
 * @throws FileNotFoundException
 * @throws UnsupportedEncodingException
 */
public PlaylistBean findNewSongs(String newDirectory) 
        throws FileNotFoundException, UnsupportedEncodingException{
    PlaylistBean main = getMainPlaylist();
    File file = new File(newDirectory);

    // add new songs to existing main playlist
    digSongs(main.getSongsInPlaylist(), file);

    return main;
}

大家好，我知道這是很多代碼和東西要讀。 我似乎無法在谷歌上找到我需要的答案。 我懷疑問題與傳遞給TableView <>的引用有關但我老實說不知道。 我希望有人可以花點時間看看。 如果有人需要，我會發布更多代碼

編輯：FileBean類

package fun.personalUse.dataModel;

import java.io.File;
import java.io.FileNotFoundException;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.Comparator;
import javafx.beans.property.SimpleStringProperty;
import javafx.scene.media.Media;
import javafx.scene.media.MediaPlayer;

/**
 * Data model for use with a media player. This object is intended to store
 * song data for 1 song
 * @author Karottop
 *
 */
public class FileBean implements Comparator<FileBean>, Comparable<FileBean>{
private File file;
private SimpleStringProperty location;
private SimpleStringProperty songName;
private SimpleStringProperty  album;
private SimpleStringProperty  artist;
private SimpleStringProperty  url;
private Media media;
private MediaPlayer player;
private SimpleStringProperty  duration;

/**
 * inserts default or null values for every field. This constructor
 * should be used when making a serializable FileBean. setters should
 * be used to initialize the object
 */
public FileBean(){
    media = null;
    file = null;
    location = new SimpleStringProperty();
    songName = new SimpleStringProperty();
    album = new SimpleStringProperty();
    artist = new SimpleStringProperty();
    url = new SimpleStringProperty();

    /**
     *  must initialize with a number because this field will be called
     *  before the MediaPlayer's status has changed which would cause a 
     *  null pointer exception to be thrown if not initialized
     */
    duration = new SimpleStringProperty("0.0");
}

/**
 * Initializes the file bean using a file
 * @param file
 * @throws FileNotFoundException
 * @throws UnsupportedEncodingException
 */
public FileBean(File file) throws FileNotFoundException, UnsupportedEncodingException{
    location = new SimpleStringProperty();
    songName = new SimpleStringProperty();
    album = new SimpleStringProperty();
    artist = new SimpleStringProperty();
    url = new SimpleStringProperty();

    /**
     *  must initialize with a number because this field will be called
     *  before the MediaPlayer's status has changed which would cause a 
     *  null pointer exception to be thrown if not initialized
     */
    duration = new SimpleStringProperty("0.0");
    this.file = file;
    location.set(file.getAbsolutePath().replace("\\", "/"));

    /*
     * encode all special characters.
     * URLEncoder puts a '+' where a ' ' is so change all '+' to encoded space '%20'.
     */
    url.set(URLEncoder.encode(location.get(), "UTF-8").replace("+", "%20"));

    /*
     * Could not easily figure out how to set an action event for when the Media
     * object is done loading. Using the MediaPlayer status change event instead.
     * Looking for a better option
     */
    media = new Media("file:///" + url.get());
    this.player = new MediaPlayer(media);
    setDefaultSongNameAndArtist();
}

public FileBean(String absolutePath) throws FileNotFoundException, UnsupportedEncodingException{
    this(new File(absolutePath));
}

/**
 * This method uses the parent directory strucutre to guesstimate
 * what the song name, artist and album name is. a '?' is appended at the
 * end of each item to indicate this is a guessed value
 * 
 * media file that do not adhere to the following directory structure 
 * will not be named correctly:
 * 
 * pathToMedia/Artist/Album/song
 */
private void setDefaultSongNameAndArtist(){
    String[] songLocation = getLocation().split("/");
    String[] songFragment = songLocation[songLocation.length - 1].split("[.]");
    setSongName(songFragment[0]);

    setAlbum(songLocation[songLocation.length - 2] + "?");
    setArtist(songLocation[songLocation.length - 3] + "?");

}



/**
 * @return the player
 */
public MediaPlayer getPlayer() {
    return player;
}

/**
 * @param player the player to set
 */
public void setPlayer(MediaPlayer player) {
    this.player = player;
}

/**
 * @return the duration
 */
public double getDuration() {
    return Double.parseDouble(duration.get());
}



/**
 * @param duration the duration to set
 */
public void setDuration(double duration) {
    this.duration.set(String.format("%.2f", duration));
}



/**
 * @return the album
 */
public String getAlbum() {
    return album.get();
}



/**
 * @param album the album to set
 */
public void setAlbum(String album) {
    this.album.set(album);
}



/**
 * @return the artist
 */
public String getArtist() {
    return artist.get();
}



/**
 * @param artist the artist to set
 */
public void setArtist(String artist) {
    this.artist.set(artist);
}



/**
 * @return the media
 */
public Media getMedia() {
    return media;
}



/**
 * @param media the media to set
 */
public void setMedia(Media media) {
    this.media = media;
}



/**
 * @return the url
 */
public String getUrl() {
    return url.get();
}


/**
 * @param url the url to set
 */
public void setUrl(String url) {
    this.url.set(url);
}


/**
 * @return the file
 */
public File getFile() {
    return file;
}

/**
 * @param file the file to set
 */
public void setFile(File file) {
    this.file = file;
}

/**
 * @return the location
 */
public String getLocation() {
    return location.get();
}

/**
 * @param location the location to set
 */
public void setLocation(String location) {
    this.location.set(location);
}

/**
 * @return the name
 */
public String getSongName() {
    return songName.get();
}

/**
 * @param name the name to set
 */
public void setSongName(String name) {
    this.songName.set(name);
}

/**
 * returns the songName property
 * @return
 */
public SimpleStringProperty songNameProperty(){
    return songName;
}

/**
 * returns the artist property
 * @return
 */
public SimpleStringProperty artistProperty(){
    return artist;
}

/**
 * returns the album property
 * @return
 */
public SimpleStringProperty albumProperty(){
    return album;
}

/**
 * returns the duration property
 * @return
 */
public SimpleStringProperty durationProperty(){
    return duration;
}

/**
 * Creates a serializable copy of this object
 * by using it's setters. The purpose of this
 * method is so that the FileBean objects can
 * be exported to an XML
 * @return
 */
public FileBean getSerializableJavaBean(){
    FileBean temp = new FileBean();
    temp.setAlbum(this.getAlbum());
    temp.setArtist(this.getArtist());
    temp.setDuration(this.getDuration());
    temp.setFile(this.getFile());
    temp.setLocation(this.getLocation());
    temp.setMedia(this.getMedia());
    temp.setPlayer(player);
    temp.setSongName(this.getSongName());
    temp.setUrl(this.getUrl());

    return temp;
}

/**
 * Method used to return a fully populated FileBean after decoded from XML.
 * @return
 */
public FileBean getFullFileBean(){

    try {
        return new FileBean(new File(getLocation()));
    } catch (FileNotFoundException | UnsupportedEncodingException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        FileBean temp = new FileBean();
        temp.setLocation("error");
        return temp;
    }
}

/**
 * Returns are string in the following format:
 * 
 * [song name], [artist name], [album name]
 */
@Override
public String toString(){
    return String.format("%s, %s, %s", getSongName(), getArtist(), getAlbum());
}

/**
 * uses FileBean.toSting().compareTo(this.toString())   to determine if the two
 * beans are equal
 */
@Override
public boolean equals(Object fileBean){
    FileBean newBean = (FileBean)fileBean;
    return newBean.toString().compareTo(this.toString()) == 0;
}


/**
 * Uses the String.compare() to order FileBeans based on their absolute path
 */
@Override
public int compareTo(FileBean bean) {
    if(this.getLocation().compareTo(bean.getLocation()) > 0){
        return 1;
    }else if(this.getLocation().compareTo(bean.getLocation()) < 0){
        return -1;
    } else{
        return 0;
    }
}

/**
 * uses the compareTo method to compare two files beans.
 * 
 * This method uses the String.compare() to order FileBeans
 * based on their absolute path
 */
@Override
public int compare(FileBean bean1, FileBean bean2) {
    // TODO Auto-generated method stub
    return bean1.compareTo(bean2);
}


}

Answer 1

使用File.listFiles()幾乎總是一個壞主意，因為它急切地分配一個可能非常耗費內存的文件數組。

因此遞歸的digSongs方法可能會產生顯着的峰值內存使用（甚至導致OutOfMemoryError ）。

看看Files.walkFileTree(...) 。 它是一種用於目錄遍歷的高效內存高效解決方案。