[英]Camel keeps SFTP Connections open leading to memory leak
I have been analyzing a memory leak in one of our applications and while analyzing a heapdump I found that >98% of the memory is occupied by Camels SharedProducerServicePool
. 我一直在分析我们的一个应用程序中的内存泄漏,在分析堆转储时,我发现> 98%的内存被Camels
SharedProducerServicePool
占用。 Essentially it keeps a lot of RemoteFileProducer
in its pool (in this particular sample about 41k). 本质上,它在其池中保留了很多
RemoteFileProducer
(在此特定示例中约为41k)。 All of them have an SftpEndpoint
. 它们都有一个
SftpEndpoint
。
The following is the Camel route that eventually uploads a file via SFTP: 以下是骆驼路线,该路线最终通过SFTP上传文件:
from("activemq:queue:transform")
.routeId("TransformJson2Avro")
.process(new FileListCountProcessor())
.split(body()).streaming()
.setHeader("CURRENT_FILE", simple("${body}"))
.log("File to process: ${header.CURRENT_FILE}; Flags: "
+ Json2AvroTransformationFlags.FILE_AVRO_BACKUP + "=${header." + Json2AvroTransformationFlags.FILE_AVRO_BACKUP + "}, "
+ Json2AvroTransformationFlags.FILE_AVRO_SFTP + "=${header." + Json2AvroTransformationFlags.FILE_AVRO_SFTP + "}, "
+ Json2AvroTransformationFlags.FILE_JSON_BACKUP + "=${header." + Json2AvroTransformationFlags.FILE_JSON_BACKUP + "}, "
+ Json2AvroTransformationFlags.FILE_JSON_DELETE + "=${header." + Json2AvroTransformationFlags.FILE_JSON_DELETE + "}, "
+ "EXPORT_METHOD=${header.EXPORT_METHOD}")
.convertBodyTo(File.class)
.process(new Json2AvroProcessor(filenamePattern, tempFilePath))
.choice()
.when(header(Json2AvroTransformationFlags.FILE_AVRO_BACKUP).isEqualTo("true"))
.to("file:///?fileName=${header.ARCHIVE_FOLDER}/${property.FILENAME}")
.log("Transformed file: '${property.FILENAME}' (archived to '${header.ARCHIVE_FOLDER}')")
.end()
.choice()
.when(header(Json2AvroTransformationFlags.FILE_AVRO_SFTP).isEqualTo("true"))
.log("Upload to SFTP '" + getSFTPExportStringForLogging() + "'")
.recipientList(simple("sftp://${header.SFTP_USER}@${header.SFTP_HOST}:${header.SFTP_PORT}/${header.SFTP_DIR}?password=${header.SFTP_PASSWORD}&fileName=${property.FILENAME}&disconnect=true")).end()
.end()
.process(new JsonFileOperationProcessor(backupPath))
.aggregate(constant(true), new DisabledAggregationStrategy())
.completionSize(simple("${property.fileListCount}"))
.log("Number of files: ${property.fileListCount}")
.to("activemq:queue:transformCompleted")
.id("insertIntoMessageQueue");
What could be the reason that the SFTP connections don't get closed or the Producers get created but not stopped again? SFTP连接没有关闭或创建生产者但又没有停止的原因可能是什么? I can't find a hint in the official documentation.
我在官方文档中找不到提示。
Any help is greatly appreaciated :) 任何帮助都感激不尽:)
Note: The Caml version used here is Camel 2.17.0 注意:此处使用的Caml版本为Camel 2.17.0
In your receipientList its better to set the dynamic file name as a header, eg Exchange.FILE_NAME
as the key, to avoid creating too many unique endpoints, instead you will reuse the same endpoint for the same hosts. 在您的receipientList中,最好将动态文件名设置为标头,例如将
Exchange.FILE_NAME
为键,以避免创建过多的唯一端点,而是将相同的端点重用于相同的主机。
And if you dont want any endpoint pooling you can configure the cacheSize
option on the recipient list to a lower value, or turn it off. 而且,如果您不希望任何端点池,则可以将收件人列表上的
cacheSize
选项配置为较低的值,或将其关闭。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.