Bash：如何解析包含 ls -ltr 的 output 的日志文件，使用 bash 脚本提取在特定时间之前修改的文件名

Question

I am trying to get all the files in SFTP directories that were last modified before 24 hours using Unix bash script. Please note that lftp option is not supported in my environment. So, I have created a script that will list all the sub directories of the SFTP directory to a local file, then I have used a loop to traverse all those sub directories names, printed the content of the directory using ls -ltr and directed that output to a local file.

Now I have a file with below content and let's assume that the current time is Oct 28th 22:32.

-rw------- 1 200      100             1930 Oct 25 08:31 File1
-rw------- 1 200      100              280 Oct 25 11:32 File2
-rw------- 1 200      100              280 Oct 25 12:17 File3
-rw------- 1 200      100              280 Oct 25 22:31 File4
-rw------- 1 200      100              280 Oct 26 22:32 File5
-rw------- 1 200      100              280 Oct 27 22:31 File6
-rw------- 1 200      100              280 Oct 16 09:59 File7
-rw------- 1 200      100              280 Oct 16 09:59 File8
-rw------- 1 200      100              280 Oct 16 10:00 File9

Is it possible to get all the filenames and their last modified time stamps with last modified date < sysdate - 24 hours?

Answer 1

For automation, I strongly suggest using 'lftp'. The 'sftp' is a simple interface, appropriate for interactive usage (eg, no status code for failures, etc.).

The 'lftp' has lot of options beyond 'sftp'. In particular, it has 'client side' listing (cls):

sftp
open sftp://host/path
user user password
cls -l --time-style='%Y-%m-%d %H:%M:%S'
Output:
drwxr-xr-x    2 root     root         4096 2019-10-13 16:01:46 /bin/
drwxr-xr-x    3 root     root         4096 2019-06-30 08:07:02 /boot/
drwxr-xr-x    2 root     root         4096 2019-06-09 19:36:33 /cdrom/
drwxr-xr-x   18 root     root         3960 2019-10-19 08:08:01 /dev/
drwxr-xr-x  150 root     root        12288 2019-10-20 15:49:39 /etc/
drwxr-xr-x    4 root     root         4096 2019-06-09 23:18:04 /home/

If you are looking for cutoff, you can also sort by time (I believe --sort=date , and optioanlly reverse -r ), and stop processing once you match the date range.

Side note: sftp has a limited 'find' command. Unfortunately, it does not offer filtering by date.

Most other 'sftp' clients that I've tried were using the default LS format. The exception are the SFTP modules for Perl, Python, which provide the client with the date in "Unix" format, allowing any test. However, this is significantly more work than using lftp .

Answer 2

I ended up parsing the log file line by line, extracted the last modified time to a variable and file name to another variable. Then I converted the last modified time string to Unix time format and then to epoch number. Similarly I got the epoch number for sysdate-1. Finally I compared those two values and filtered the file names matched my criteria. code snippet is given below

prev_day_time=$(date +%s -d "$(date --date='1 day ago')")
file_mod_time=$(date +%s -d "$(date -d "$(echo $(echo $files_list | awk '{print $1}') | sed -e 's/\(.....\)/\1 /')")")