copy parts of a file from multiple sources over LAN using python

Question

Python seems to have functions for copying files (eg shutil.copy) and functions for copying directories.This also works with network paths.

Is there a way to copy only part of the file from multiple sources and merge them afterwards

Like a download manager downloads parts of a single file from multiple sources increasing the overall download speed.

I want to achive the same over lan.

I have a file on more than two machines on my network.

How could i copy parts of file to a single destination from multiple sources ?

Can it be done with standard shutil libraries?

Answer 1

• Stat the file to find the size
• Divvy up the the start:end points that each reader will handle
• Open your write file in binary mode
• Open your readers in binary mode
• Handle the merging/collating of data when writing it out

Answer 2

I suggest you use a free open source peer-to-peer library.

Peer-to-peer file transfers are effective for the problem you are describing: to copy some huge files ~1gb from one machine on a network to 40 -50 machines

Eg http://code.google.com/p/python-libtorrent/

Or use a client implementation.