How to simulate an ftp transfer of a big file with curl?

Question

I want to check the FTP connection with FTP server. To do that I want to make a transfer of big file (like 10 G) to the FTP server (upload) inorder to check my FTP connection.

But to do that I have to generate a big file of 10 G. and this is not good especially if my device does not have enough memory/disk. So I want to simulate a transfer of big file (of 10 G for example) without generating a file of 10G.

Is it possible to do it with curl command?

Answer 1

'truncate' (the command line tool) is your friend. It can create huge files that are "sparse" , meaning that the long sequence of zeroes is just a hole that won't actually occupy space on the drive.

Therefore, you can create a 10G file (containing all zeroes) in a much smaller file system and hard drive like this (works on Linux):

$ truncate --size=10G bigfile

Answer 2

With FTP, assuming that the client and server have small RT delay, sending 10G will take the same amount of sending 100 files of 100M, over the same open connection. Hopefully your device can have that much space allocated.

The key is to avoid re-openning the connection. Do something like the following to keep the connection open.

X=ftp://server/path/to/100M
X100=$(for i in {1..100} ; do echo " " $X ; done )
curl $X100