How to download a file in chunks using camel?

Question

I want to download a very large file using camel, but I don't want to hold the entire file in memory and THEN save it to file. I want to stream the file in and save or write to a file in chunks.

Is this possible with Camel, and if so, how do I do this?

Note: Is it possible that the endpoint I am downloading the file does not support streaming/chunking? If yes, how can I verify this?

Answer 1

Camel's HTTP component uses Netty to make the request. Netty reads the entire response into memory, so there is no way to do what you are asking for.

You would need to implement your own endpoint for Camel that utilizes another HTTP library which has support for HTTP response streaming.

More documentation is available here :

https://cwiki.apache.org/confluence/display/CAMEL/Netty4+HTTP

Answer 2

You can 3 option to download the file ie using:

ftp://[username@]hostname[:port]/directoryname[?options] sftp://[username@]hostname[:port]/directoryname[?options] ftps://[username@]hostname[:port]/directoryname[?options]

There is a option of streamDownload in it.

For more check out http://camel.apache.org/ftp.html