How to copy bytes from a specific offset in a file to another file at a specific offset?

Question

Would like the most efficient way of copying, say, 1MB from an offset in file 1 to the same or different offset in file 2. Is it possible to have multiple threads doing the reads and the writes at the same time safely?

To clarify, I of course want to handle file->file as stated. However I also want to have multiple threads reading from a network IO bound location (internet, etc..), and then reassemble those back into a single file locally. If it makes more sense that the write operation be single-threaded that is fine too.

Answer 1

You almost certainly don't want multiple threads doing this - you're going to be IO-bound, and you don't want the overhead of the disk seeking all over the place.

I'd just do it in one thread:

• Open file 1 for reading
• Seek
• Open file 2 for writing
• Seek
• Repeatedly copy a buffer at a time (eg 32K) from one stream to the other.

You may find that some of the FileOptions (eg SequentialScan for the reader) could make a difference.

EDIT: As noted in comments, it may be worth using two threads - one to read and one to write, particularly if you're using two separate drives. However, it's also possible that with the operating system automatically doing prefetching etc, that wouldn't be helpful. It would certainly complicate the code.

Do you have a target time for this operation? How fast does a simple implementation take compared with that target time? I definitely wouldn't venture into multiple threads or async operations until you've established how long the simple approach takes.