Python - download all folders, subfolders, and files with the python ftplib module

Question

I have been working all day trying to figure out how to use the python ftplib module to download folders, subfolders, and files from an ftp server but I could only come up with this.

from ftplib import FTP
import sys, ftplib

sys.tracebacklimit = 0 # Does not display traceback errors
sys.stderr = "/dev/null" # Does not display Attribute errors

Host = "ftp.debian.org"
Port = 21
Username = ""
Password = ""

def MainClass():
    global ftp
    global con
    Host
    Port
    ftp = FTP()
    con = ftp.connect(Host, Port) # Connects to the host with the specified port

def grabfile():
    source = "/debian/"
    filename = "README.html"
    ftp.cwd(source)
    localfile = open(filename, 'wb')
    ftp.retrbinary('RETR ' + filename, localfile.write)
    ftp.quit()
    localfile.close()

try:
    MainClass()
except Exception:
    print "Not Connected"
    print "Check the address", Host + ":" + str(Port)
else:
    print "Connected"

if ftplib.error_perm and not Username == "" and Password == "":
    print "Please check your credentials\n", Username, "\n", Password

credentials = ftp.login(Username, Password)
grabfile()

This python script will download a README.html file from ftp.debian.org but, I would like to be able to download whole folders with files and subfolders in them and I cannot seem to figure that out. I have searched around for different python scripts using this module but I cannot seem to find any that do what I want.

Any suggestions or help would be greatly appreciated.

Note: I would still like to use python for this job but it could be a different module such as ftputil or any other one out there.

Thanks in advance, Alex

Answer 1

The short solution: You could possibly just run: "wget -r ftp://username:password@ftp.debian.org/debian/ *" to get all the files under the debian directory. Then you can process the files in python.

The long solution: You can go over every directory listing by using ftplib, getting a directory listing parsing it and then getting every file and recursing into directories. If you search the web you'd find previous posts on stackoverlow which solve this issue