I have an application that needs to download files from sftp. I'm currently using apache commons-vfs2
I have a scheduler that runs every 1 minute. 1. Get the list of files that's on remote (open connection, get the list, then close connection) 2. Download the files from step 1 (open connection, download each files, then close connection)
How can I keep the connections to minimum? Is there a way to limit how many connections I have with commons-vfs2?
Here is my code
private List<FileObject> getRemoteFilesList() throws FileSystemException {
FileObject[] remoteFiles;
try {
manager.init();
final @Cleanup FileObject remoteDirectoryObject = manager.resolveFile(uri, fileSystemOptions);
remoteFiles = remoteDirectoryObject.getChildren();
} finally {
manager.freeUnusedResources();
manager.close();
}
return Arrays.stream(remoteFiles)
.collect(Collectors.toList());
}
private List<File> downloadRemoteFiles(final List<FileObject> remoteFiles) {
if(remoteFiles.isEmpty()) {
return Collections.emptyList();
}
final List<File> myCollection = new ArrayList<>();
try {
manager.init();
for (final FileObject myfile : remoteFiles) {
final File localFile = downloadFile(myfile);
myCollection.add(localFile);
myfile.delete();
}
} catch (final IOException exception) {
log.warn("Unable to download because ", exception);
} finally {
manager.freeUnusedResources();
manager.close();
}
return myCollection;
}
The apache commons wiki for VFS ( https://wiki.apache.org/commons/VfsFaq ) says to use the following when closing an SFTP connection in certain circumstances:
((DefaultFileSystemManager) fsManager).close();
This forces the close
method on the DefaultFileSystemManager
to be called, rather than the close
method on the FileSystemManager
class.
It may not be your issue, but it's something that may be related.
You can try this alternative method to clean up any temporary files and close all providers.
FileObject src = null;
/**
* Release system resources, close connection to the filesystem.
*/
public void release() {
FileSystem fs = null;
this.src.close(); // Seems to still work even if this line is omitted
fs = this.src.getFileSystem(); // This works even after the src is closed.
this.fsManager.closeFileSystem(fs);
}
More details can be found on: https://cwiki.apache.org/confluence/display/COMMONS/SimpleSftpFileDownload
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.