Recursively scan a Windows FTP server using PHP hangs midway… why?

Question

I needed to map a FTP server (running on Windows) using PHP, so I made a function that essentially navigates through the FTP's folder structure, listing everything inside it and calls itself recursively when it encounters a subfolder.

Everything worked fine in my development environment, but things took a sour turn once I migrated everything to the client's infrastructure. Their FTP server holds 30GB of data (a lot of files, but not as many as you would expect, since many are > 100MB), so I was expecting it to be "slowish", but 90 to 120 seconds seemed a lot.

Therefore, I did some investigating with tcpdump and it is perfectly visible that the FTP server has some kind of anti-flood protection because at a given point every request gets stalled for at least 60 seconds. This behavior also happens if I attempt to map the server recursively using lftp .

I am currently using active mode because passive threw the following error:

Warning: ftp_rawlist() [function.ftp-rawlist]: php_connect_nonb() failed: Operation now in progress (115) in /home/user/public_html/sync.php on line 35

Line 35 refers to a ftp_rawlist of the current directory.

I not really sure it'll help, but here's the code I'm using anyway:

function ftp_get_recursive_paths($conn, $path, $root, $max_level = 0) {
    $files = array();
    $contents = ftp_rawlist($conn, $path);

    if(is_array($contents)) {
        foreach($contents as $line) {
            (...) Parsing the results (...)
            if ((strpos($f, ".DS_Store") === false) && (strpos($f, ".dropbox") === false) && (strpos($f, "Thumbs.db") === false)) {

                if(!strpos($f, '.')) {
                    $init = microtime(true);
                    $files[$f] = (object) array('name' => $f, 'date' => $date, 'path' => $path . $f, 'content' => ftp_get_recursive_paths($conn, $path . $f, $path));
                    $end = microtime(true);
                    echo '<p> Folder ' . $path . $f . ' scanned in '. ($end - $init), ' seconds. </p>';
                }
                else {
                    $files[] = (object) array('name' => $f, 'date' => $date, 'path' => $path .  $f);
                }
            }
        }
    }

    return $files;
}

Does anyone know how this anti-flood thing can be prevented? Is it a firewall? Some kind of configuration in the client's server?

I can confirm that the client is using Filezilla Server on Windows.

Answer 1

In the end, the client did have a firewall of some sort that blocked the requests after a certain threshold.

TL;DR: everybody lies.

Answer 2

同样在本地使用时（连接到与.php在同一服务器上的ftp），如果您使用“ sftp”也可以，但是在远程使用时，您必须使用“ ftp”或很好地配置远程客户端