为什么这个 SFTP 下载这么慢？

Question

I have a download button when clicked, it takes about 15 seconds to download a file because it has to SFTP into the server, find the right path/ files, and return the response.

<a class="btn btn-primary btn-sm text-primary btn-download-1" onclick="startDownload('1')"><i class="fa fa-download "></i></a>

This is the startDownload function:

function startDownload(interfaceId) {
    window.location = "/nodes/interface/capture/download?port=" + interfaceId;
}

The backend code in /nodes/interface/capture/download :

public function download_files()
{

    $dir = '';
    $portNumber = Request::get('port');
    $zipMe = false;

    $remotePath = "/home/john/logs/".$dir."/";

    if (!isset($dir) || $dir == null) {
        return redirect()->back()->withInput()->withFlashDanger('SFTP Could not connect.');
    }

    $acsIp =  explode('://', env('ACS_URL'));
    $acsIp =  explode(':',$acsIp[1])[0];
    $sftp = new SFTP($acsIp.':22');

    if (!$sftp->login('john', '***')) {
        return redirect()->back()->withInput()->withFlashDanger('SFTP Could not connect.');
    }

    // Get into the Specified Directory
    $sftpConn = Storage::disk('sftp');

    $SFTPFiles = $sftpConn->allFiles('/'.$dir);

    if ( count($SFTPFiles) > 0 ) {
        foreach ($SFTPFiles as $file) {
            $fileName = $file;
            break;
        }

    } else {
        \Log::info('Files Not found in the Remote!');
        return redirect()->back()->withInput()->withFlashDanger('Files Not found in the Remote!');
    }

    // Create and give 777 permission to remote-files directory
    if (!is_dir(public_path('remote-files/'.$dir))) {
        mkdir(public_path('remote-files/'.$dir), 0777, true);
    }

    $filesToZip = [];

    foreach ( $SFTPFiles as $fileName ) {
        if ( $fileName == '..' || $fileName == '.' ) {
            continue;
        } else if ( $fileName == '' ) {
            \Log::info('File not found');
            continue;
        }

        $fileName     = explode("/", $fileName);
        $onlyFileName = (!empty($fileName) && isset($fileName[1])) ? $fileName[1] : "";
        $filepath = $remotePath.$onlyFileName;

        if (strpos($onlyFileName , $portNumber) !== false) {


            // Download the remote file at specified location in Local
            if (!$sftp->get($filepath, 'remote-files/'.$dir.'/'.$onlyFileName))
            {
                die("Error downloading file ".$filepath);
            }

            $file = public_path('remote-files/'.$dir.'/').$onlyFileName;

            $headers = array(
                'Content-Description: File Transfer',
                'Content-Type: application/octet-stream',
                'Content-Disposition: attachment; filename="'.basename($file).'"',
                'Cache-Control: must-revalidate',
                'Pragma: public',
                'Content-Length: ' . filesize($file)
            );

            return Response::download($file, $onlyFileName, $headers);
        }

        // IF File is exists in Directory
        if ( file_exists( public_path('remote-files/'.$dir.'/').$onlyFileName ) ) {
            $filesToZip[] = public_path('remote-files/'.$dir.'/').$onlyFileName;
            \Log::info('File Generated '.'remote-files/'.$dir.'/'.$onlyFileName);

            // Remove Files from public/remote-files
            $this->removeDirAndFiles('', public_path('remote-files/'.$dir));
            exit;

        } else {
            \Log::info('File not Generated '.'remote-files/'.$dir.'/'.$onlyFileName);
        }
    }
}

This code indeed works but takes about 15 seconds, which is too long for the use case.

Is there a way to speed this up? Is there something wrong with my code or is this to be expected? Should I consider switching to SCP? Should I reconsider authentication?

Answer 1

I changed your function to measure the time in different parts, so you can understand which is the piece that is slowing down your application by looking at the log for this string "### [TIME] -- "

public function download_files()
{
    $start = time();
    $dir = '';
    $portNumber = Request::get('port');
    $zipMe = false;

    \Log::info("### [TIME] -- t1 =  " . (time() - $start));

    $remotePath = "/home/john/logs/".$dir."/";

    if (!isset($dir) || $dir == null) {
        return redirect()->back()->withInput()->withFlashDanger('SFTP Could not connect.');
    }

    $acsIp =  explode('://', env('ACS_URL'));
    $acsIp =  explode(':',$acsIp[1])[0];

    $t1 = time();

    $sftp = new SFTP($acsIp.':22');

    $t2 = time();
    \Log::info("### [TIME] -- SFTP Instantiation took " . ($t2 - $t1) . " secs");

    if (!$sftp->login('john', '***')) {
        return redirect()->back()->withInput()->withFlashDanger('SFTP Could not connect.');
    }

    $t1 = time();
    // Get into the Specified Directory
    $sftpConn = Storage::disk('sftp');

    $SFTPFiles = $sftpConn->allFiles('/'.$dir);

    $t2 = time();
    \Log::info("### [TIME] -- SFTPFiles list took " . ($t2 - $t1) . " secs");


    // this loop is not clear to me, you basically take the first element and
    // exit the loop
    if ( count($SFTPFiles) > 0 ) {
        foreach ($SFTPFiles as $file) {
            $fileName = $file;
            break;
        }
    } else {
        \Log::info('Files Not found in the Remote!');
        return redirect()->back()->withInput()->withFlashDanger('Files Not found in the Remote!');
}

    $t1 = time();

    // Create and give 777 permission to remote-files directory
    if (!is_dir(public_path('remote-files/'.$dir))) {
        mkdir(public_path('remote-files/'.$dir), 0777, true);
    }

    $t2 = time();
    \Log::info("### [TIME] -- Directory creation took " . ($t2 - $t1) . " secs");

    $filesToZip = [];
    $t1 = time();
    foreach ( $SFTPFiles as $fileName ) 
    {
        $start_loop_time = time();
        \Log::info("### [TIME] -- Loop for $fileName took " . (time() - $t1) . " secs");

        if ( $fileName == '..' || $fileName == '.' ) {
            continue;
        } else if ( $fileName == '' ) {
            \Log::info('File not found');
            continue;
        }

        $fileName     = explode("/", $fileName);
        $onlyFileName = (!empty($fileName) && isset($fileName[1])) ? $fileName[1] : "";
        $filepath = $remotePath.$onlyFileName;

        if (strpos($onlyFileName , $portNumber) !== false) {

             $responseCreationStart = time();
            // Download the remote file at specified location in Local
            if (!$sftp->get($filepath, 'remote-files/'.$dir.'/'.$onlyFileName))
            {
                die("Error downloading file ".$filepath);
            }

            $file = public_path('remote-files/'.$dir.'/').$onlyFileName;

            $headers = array(
                'Content-Description: File Transfer',
                'Content-Type: application/octet-stream',
                'Content-Disposition: attachment; filename="'.basename($file).'"',
                'Cache-Control: must-revalidate',
                'Pragma: public',
                'Content-Length: ' . filesize($file)
            );
            $responseCreationEnd = time();
            \Log::info("### [TIME] -- Response creation took " . ($responseCreationEnd  - $responseCreationStart ) . " secs");
            return Response::download($file, $onlyFileName, $headers);

        }

        // IF File is exists in Directory
        if ( file_exists( public_path('remote-files/'.$dir.'/').$onlyFileName ) ) {
            $t3 = time();
            $filesToZip[] = public_path('remote-files/'.$dir.'/').$onlyFileName;
            \Log::info('File Generated '.'remote-files/'.$dir.'/'.$onlyFileName);

            // Remove Files from public/remote-files
            $this->removeDirAndFiles('', public_path('remote-files/'.$dir));
            $t4 = time();
            \Log::info("### [TIME] -- Deletion took " . ($t4 - $t3) . " secs");
            exit;

        } else {
            \Log::info('File not Generated '.'remote-files/'.$dir.'/'.$onlyFileName);
        }

        \Log::info("### [TIME] -- Loop end reached in  " . (time() - $start_loop_time ) . " secs");
    }
}

Answer 2

Do you set the $dir variable anywhere in your code? Because as how I read it, the only contents of it is, and always will be, an empty string.

Does this take a long time in other browsers as well? Is there anything notable that pops up in the "Network" tab of the browser inspector after pressing the button?

Furthermore, perhaps you could try to put the function in a button element instead of a hyperlink.
Perhaps this delay is some sort of timeout that the browser has internally, since it is expecting another page to load?

So instead of using the hyperlink for the button, I would suggest giving this a try:

<button class="btn btn-primary btn-sm text-primary btn-download-1" onclick="startDownload('1')"><i class="fa fa-download "></i></button>

I'm curious to know what will happen. Keep us updated!