Python - What is the fastest way i can copy files from a source folder to multiple destinations

Question

source = /tmp/src contains a,b,c,d files destinations = '/one' , '/two'

so i want to copy files a,b,c,d to both destinations '/one' and 'two'

something like

source = '/tmp/src'
destinations = []

def copy_files_multiple_dest(source,destinations)

right ? now, how would i loop through all the destinations

Answer 1

how about something like:

import os
import shutil


source = '/tmp/src/'
destinations = []

def copy_files_multiple_dest(source,destinations):
  sfiles = os.listdir(source) # list of all files in source
  for f in sfiles:
    for dest in destinations:
      shutil.copy(os.path.join(source,f), dest)

i'm not sure it's the but it should do the job. 但应该可以完成工作。

Answer 2

Reading the source files only once makes sense here:

def xcopy_to_multiple_destinations(srcDir, destinations):
    for filename in os.listdir(srcDir):
        with open(os.path.join(srcDir, filename), "rb") as srcFile:
            for destDir in destinations:
                with open(os.path.join(destDir, filename), "wb") as destFile:
                    # ...copy bytes from srcFile to destFile...

If you want to copy recursively, use os.walk (see other question: Python recursive folder read ). You can adapt the solution accordingly.

Note that "fastest" is a broad term. Hard linking should be faster, for example ;) Or using copy-on-write with the appropriate file system.

Answer 3

os package is the usual way to got but have a look at this new project https://github.com/amoffat/pbs . you could then just do:

import pbs
destinations =['/one', '/two']
for destination in destinations:
   pbs.copy("-R", '/tmp/src', destination)

maybe not the fastest but certainly wins the beauty contest

Answer 4

Since you say the files won't diverge, you can just hardlink them. You don't need python specifically, here's how I'd do it in bash.

dests=(a b c d)
for dest in "${dests[@]}"; do
  cp -rl /source/root "$dest"
done

If it has to be python, look at os.link and other functions in that module.