Autosys File watcher

Question

My Task : Need to look for any new/updated files in particular directory of Server A. Once a file new/modified file is present on this directoty, I will need to ftp that file to Server B.

Can I do this with Autosys and shell scripts ? or will a Java program be more appropriate for this task ?

If I go with Autosys, I can setup a file watcher job. On success I can trigger another job. But how will this job know the file name ? Is file watcher smart enough to pass the file name as parameter to another job ?

Please help....

Answer 1

The file watcher job is for checking the existence of a file and checking the size of it. You have to know the name of the file in advance. In your case if you can receive any number of new files whose names you don't know beforehand, the file watcher job is not going to work.

One plausible solution can be:

1. Write a shell script that will
   • Get ls -l of the directory you are watching and save it to a file
   • Compare this file with the file generated from the last run of the script
   • Find out the new files and the files that were changed and ftp them to the destination
2. Schedule the script in autosys to run periodically, say every 10 mins.

Answer 2

1. Write a shell script - a). cd to that directory b). use find command to find all the files,can choose the file extension in find, for eg. *.txt or *.csv. You can opt for time field as well, if you want to check for files which are not older than a certain period. OR In ur case u can maintain versioning of files,once the file comes and is found by the find command, u can rename the file.This way u can ensure u have the latest file to be ftp'ed. You can archive or purge the old files, by creating a logic after how many days u don't want to have them. c). Send the ouput of find to a file, test the file, if it is not empty.ftp that file to server B.
2. Schedule this script inside a job which runs every few minutes, depending on the frequency of the file arrival.
3. In the success condition, check if the previous run of the job is in Success.