Setting up proxy with ftp with Python

Question

We are trying to develop a Python module that will hit an FTP server and download files to my local machine. When we try to run the FTP portion of the module it is timing out.

We have a proxy server (let's call it "officeproxy.com:8080") to handle this and when using an FTP client like FileZilla or Windows Explorer to access FTP sites we are successful.

Let's call the ftp site "ftp.cal.com". User name is "papa". Password is "tango123".

So far we have:

Proxy = officeproxy.com:8080

FTP = ftp.cal.com

User = papa

PW = tango123 

The above are not real entities so if you want to swap them out for real ones, be my guest.

I need a module to first load the proxy service then run the FTP portion.

I am running Python 2.7.

I have searched around and have this code so far. The OP said it is just a short module to test connection to FTP and read one file.

(Note: I have intentionally put # in a lot of places to show when I don't know to fill in, or other reasons):

import urllib2
# I have filled in the proxy info
proxy_host = 'officeproxy.com:8080'

# I don't think this needs any modification, right?  
proxy_handler = urllib2.ProxyHandler({'ftp': proxy_host})

# ditto here 
proxy_auth_handler = urllib2.ProxyBasicAuthHandler() 

# now here is where I am unsure what to put; 
# also, I really need FTP user and FTP password, and NOT Proxy... 
# so what do I need to change here?
proxy_auth_handler.add_password(None, proxy_host, proxy_user, proxy_passwd) 
opener_thru_proxy = urllib2.build_opener(proxy_handler, proxy_auth_handler)

# I filled in this part
conn = opener_thru_proxy.open('ftp://ftp.cal.com/hello.txt') 

# I don't believe I need to change this, right?
print conn.read()

Answer 1

Note, I found some similar questions on SO, but as the OP requested a detailed, beginner-level answer, I decided to post this.

To clarify, here is my understanding of what you want to accomplish:

• You have an FTP server behind a proxy.
• The FTP server requires user/password credentials, but the proxy itself does not.

One option is to use the ftplib package. One of the great advantages of a popular language such as Python is the wide selection of packages the that provide good implementations of specific functionality. In this case, an ftp client:

import ftplib  # 1

ftp = ftplib.FTP("officeproxy.com:8080")  # 2
ftp.set_debuglevel(1)  # 3
ftp.login("papa", "tango123")  # 4

filename = "hello.txt"
f = open(filename, 'wb')  # 5 
ftp.retrbinary("RETR " + filename , f.write)  # 6

f.close()   # cleanup file handle
ftp.quit()  # cleanup ftp client 

Here is what the code is doing:

1. Import the ftplib package.
2. Connect to the FTP server through the proxy. The way I have written it assume that the proxy is setup to forward the FTP protocol on the port specified (8080 in your example). If this is not the case, this will not work.
3. Enable debug logging. From the ftplib documentation :

FTP.set_debuglevel(level) : 0 , produces no debugging output... 1 produces a moderate amount of debugging output... 2 or higher produces the maximum amount of debugging output

4. Login to the FTP server using user="papa", passwd="tango123". Again, this is from the ftplib documentation for login()
5. Open a local file for writing the contents of file you are about to retrieve from the FTP server. The first argument is the filename for the file you want to save to (it can be anything, I decided to use the same filename as the filename on the FTP server). The second argument is the mode you want to open the file as, which is a string of flags: w = 'write mode', b = 'binary mode', because we are going to retrieve the file as binary and write it as binary.
6. I recommend you read the ftplib documentation for retrbinary . This is a bit tricky to understand at first. The first argument, "RETR" + filename is the command to send to the FTP server, in this case RETR hello.txt . The second argument, f.write is the callback function you want the ftp.retrbinary function to call whenever it has a block of data from the server. In its implementation, whenever retrbinary has data, it will call f.write(data) where data is the latest block of data it has. To understand more how callbacks work in Python, see Introduction to Asynchronous APIs in Python

I hope this works for you, and I hope the answer is detailed enough that you understand how it works. If it does not work, please post the stack trace output for the error you get.

Answer 2

The question is a little bit out-dated, but I didn't find any recent solution myself so I figured I would provide the solution I found. In my case the office proxy requires credentials so those are included in the solution. I would presume that you can just remove the proxy_user and proxy_passwd if your proxy does not require credentials. The code I got working in Python 3.6 is:

from ftplib import FTP

ftp = FTP("officeproxy.com")
ftp.set_debuglevel(1)
ftp.login(user='ftp_user@ftp_host proxy_user', passwd='ftp_passwd', acct='proxy_passwd')

# Do whatever you need on the FTP server here

ftp.quit()

I hope this helps.

Answer 3

Adding this as it's one of the top results and struggled to find a solution.

If your proxy is a HTTP proxy and you need to authenticate it then this is how:

import socks
import socket

socks.set_default_proxy(socks.HTTP, 
        proxy_host, 
        proxy_port, 
        username=proxy_username, 
        password=proxy_password
)
socket.socket = socks.socksocket
ftp = FTP(ftp_host)
ftp.set_debuglevel(1) 
ftp.login(
        user=ftp_user,
        passwd=ftp_password
)

Otherwise for a FTP Proxy Martin's answer works. You can see the behaviour in FileZilla for FTP proxies too and that'll help to code for your requirements

Hope this helps someone!