Check directory daily for new files - linux bash script

Question

I'd like to monitor a directory for new files daily using a linux bash script.

New files are added to the directory every 4 hours or so. So I'd like to at the end of the day process all the files. By process I mean convert them to an alternative file type then pipe them to another folder once converted.

I've looked at inotify to monitor the directory but can't tell if you can make this a daily thing.

Using inotify I have got this code working in a sample script:

#!/bin/bash

while read line
do
        echo "close_write: $line"
done < <(inotifywait -mr -e close_write "/home/tmp/")

This does notify when new files are added and it is immediate.

I was considering using this and keeping track of the new files then processing them at all at once, at the end of the day.

I haven't done this before so I was hoping for some help.

Maybe something other than inotify will work better.

Thanks!

Answer 1

您可以使用日常cron作业： http : //linux.die.net/man/1/crontab

Answer 2

Definitely should look into using a cronjob. Edit your cronfile and put this in:

0 0 * * * /path/to/script.sh

That means run your script at midnight everyday. Then in your script.sh , all you would do is for all the files, "convert them to an alternative file type then pipe them to another folder once converted".

Answer 3

To collect the files by the end of day, just use find:

find $DIR -daystart -mtime -1 -type f

Then as others pointed out, set up a cron job to run your script.

Answer 4

Your cron job (see other answers on this page) should keep a list of the files you have already processed and then use comm -3 processed-list all-list to get the new files. man comm

Its a better alternative to
awk 'FNR==NR{a[$0];next}!($0 in a)' processed-list all-list

and probably more robust than using find since you record the ones that you have actually processed.