我们如何使用Python在网络上传输大量数据？

Question

I am working on a task to back up (copy) about 100 Gb of data (including a thousand files and sub folders in a directory) to another server. Normally, for the smaller scale, I can use scp or rsync instead. However, as the other server is not on the same LAN network, it could easily take hours, even days, to complete the task. I can't just leave my computer there with the terminal running. I don't think that's the best choice, and again, I have another good reason to use Python :)

Is there any library, or best practice for me to start with? As, it's just for in-house project, we don't need any fancy features, just some fundamental things such as logging, error tolerance, etc.

Answer 1

I think your best bet is to use scp or rsync from within screen. That way you can detach the screen session and logout and the transfer will keep going.

man screen