How to copy latest file from sftp to local directory using shell script?

Question

I have multiple file in SFTP server from which I need to copy only latest file. I have written sample code but in that I am passing filename. What logic I need to add that it identify the latest file from sftp and copy it into my local?

In SFTP server -

my_data_20220428.csv
my_data_20220504.csv
my_data_20220501.csv
my_data_20220429.csv

The code which I am running-

datadir="/script/data"
cd ${datadir}
rm -f ${datadir}/my_data*.csv
rm -f ${logfile}
lftp<<END_SCRIPT
open sftp://${sftphost}
user ${sftpuser} ${sftppassword}
cd ${sftpfolder}
lcd $datadir
mget my_data_20220504.csv
bye
END_SCRIPT

what changes I need to do it automatically pick the latest file from server without hardcoding the filename?

Answer 1

You can try this script mainly copied from your sample, so it is expected that the variables have already been created.

#!/usr/bin/env bash

datadir="/script/data"
rm -f "$datadir"/my_data*.csv
rm -f "$logfile"
new=$(echo "ls -halt $sftpfolder" | lftp -u "${sftpuser}","${sftppassword}" sftp://"${sftphost}" | sed -n '/my_data/s/.* \(.*\)/\1/p' | head -1)
lftp -u "${sftpuser}","${sftppassword}" sftp://"${sftphost}" << --EOF--
cd "$sftpfolder"
lcd "$datadir"
get "$new"
bye
--EOF--

Answer 2

You could try:

latest=$(lftp "sftp://$sftpuser:$sftppassword@$sftphost" \
         -e "cd $sftpfolder; glob rels -1t *.csv; bye" |
         head -1)
lftp "sftp://$sftpuser:$sftppassword@myhost" \
-e "cd $sftpfolder; mget $latest; bye"