Move all files from one directory to another with limited disk space

Question

I have a mounted directory contained several large files. I would like to move those files onto a local directory. However, the local machine has very limited disk space and I've run into an issue where moving these files has failed due to disk space and were subsequently lost. I'm looking for a pythonic way to:

1. Attempt to move all files from the source directory to the destination
2. If we run out of space, move them all back and raise an error (or just return false) without changing ownership or permissions

I do not want to move the directory itself, only its contents. It's okay to overwrite existing files. I see plenty of "How do I move files" questions, but no "what happens if we run out of space" questions.

Local machine is running Centos 7, remote machine is running Solaris 10.

Answer 1

I think the most pythonic way would be to move chunk by chunk, checking for errors between. I'm going to assume that the file is too large to entirely store in memory at once, so something like this would probably be best:

chunk_size = 2048
successful = True

with f_in = open("original.file","rb"):
  chunk = f_in.read(chunk_size)#Only reads a small chunk of the old file at a time
  while chunk:
    try
      f_out = open("new_file","ab+")
      f_out.write(chunk) #Writes the small chunk to the end of the new file and 
      f_out.close()      #then closes it so as not to run out of memory
      chunk = f_in.read(chunk_size)
    except OSError as e:
      chunk = False
      successful = False
if successful:
  os.remove("original.file")

I'm fairly certain this would work for you, as you would end early in the event of a NOSPC error, meaning you ran out of disc space. The original file would only be deleted if you completed the write (I assume that was your intent, since you wanted to move it, not copy it).

There's an easier solution with renaming the file using os.rename("old/path/file.txt","new/path/file.txt"), but I'm not 100% sure that would work with a mounted disc to local disc. But it's probably worth trying.