Outbound FTP request from google appengine using python

Question

I need to make an outbound ftp request to retrieve a number of small files. There are 6 files each less than 10K and I only need to retrieve them once every couple of hours.

When I try to do this with urllib2.urlopen(" ftp://xxx.xxx.xxx ") I get an exception AttributeError: 'module' object has no attribute 'FTP_PORT'.

I have read through the documentation and see you are only allowed to make http and https requests from the appengine, unfortunately my application needs to consume the ftp data, does this requirement mean I can't use the appengine at all ? I sincerely hope not.

So has anyone else here found a way to make ftp requests, perhaps with a paid account ? And if not what have other people chosen to do ? does azure or ec2 allow outbound ftp requests ?

Answer 1

You're correct. Google App Engine does not allow you to make FTP requests. Not even with a paid account.

I had to use a LAMP instance on EC2 that handles FTP'ing through CURL, and make http requests to it from GAE.

Answer 2

This limitation used to drive me nuts; implementing the overhead around dynamically instantiating EC2 slave workers to relay FTP data felt like a real waste of time. Fortunately, as of April 9 this year (SDK 1.7.7) this isn't a problem any longer. Outbound sockets (eg FTP) are generally available to all billing-enabled apps.

Sockets API Overview (Python): https://developers.google.com/appengine/docs/python/sockets/

Answer 3

drivehq.com is another option. It provides both a web+ftp server. So a third party I needed to interface with (that spoke only FTP) would upload files via FTP. And then I would urlfetch them on appengine.