从Google App Engine中的FTP URL读取文件FileNotFoundException

Question

I'm writing code to read a file from an FTP URL in order to parse it and store the data in Google App Engine's Datastore. I'm able to get the code working fine when reading test files hosted on my own web server, however when I try to read the data file I need I'm getting a FileNotFoundException.

I'm able to use the same FTP URL in a browser to download the file, and can anonymously connect to the FTP URL in FileZilla, so access shouldn't be a problem, and the file is definitely there. It's a pretty big file, but I've tried to grab smaller files from the same FTP server with no luck either.

Here's the code I have at the moment:

public void doGet(HttpServletRequest req,
        HttpServletResponse resp) throws IOException, ServletException {

    // works with a URL to my own server & a test.zip, but not this one
    final URL url = new URL(
        "ftp://gisftp.metc.state.mn.us/google_transit.zip");

    // without the privileged action, I get an AccessControlException
    ZipInputStream zin = AccessController.doPrivileged(
        new PrivilegedAction<ZipInputStream>() {
            public ZipInputStream run() {
                return getZipStream(url);
            }
        }
    );

    ZipEntry zipentry = zin.getNextEntry();

    // processing files here

    zin.close();
}

// but using the privileged method, we get a FileNotFoundException
public ZipInputStream getZipStream(URL url) {
    ZipInputStream zipin = null;
    try {
        zipin = new ZipInputStream(url.openStream());
    } catch (IOException e) {
        e.printStackTrace();
    }
    return zipin;
}

At first I was getting an AccessControlException, but using a PrivilegedAction to open the stream seems to fix that.

I don't have access to the server where the file is stored, so can't change anything there.

Answer 1

I think there is a restriction on the ports that can be connected to from App Engine and FTP (21) is not in the list, this maybe causing the issue. From the URL Fetch documentation ;

An app can fetch a URL using HTTP (normal) or HTTPS (secure). The URL specifies the scheme to use: http://... or https://...

The URL to be fetched can use any port number in the following ranges: 80-90, 440-450, 1024-65535. If the port is not mentioned in the URL, the port is implied by the scheme: http://... is port 80, https://... is port 443.