Bash edit file and keep last 500 lines

Question

I am looking to create a cron job that opens a directory loops through all the logs i have created and deletes all lines but keep the last 500 for example.

I was thinking of something along the lines of

tail -n 500 filename > filename

Would this work?

I also not sure how to loop through a directory in bash.

Answer 1

If log file to be truncated is currently open by some services, then using mv as in previous answers will disrupt those services. This can be easily overcome by using cat instead:

tail -n 1000 myfile.log > myfile.tmp
cat myfile.tmp > myfile.log

Answer 2

Think about using logrotate .
It will not do what you want (delete all lines but the last 500), but it can take care of logfiles which are bigger than a certain size (normally by comressing the old ones and deleting them at some point). Should be widely available.

Answer 3

In my opinion the easiest and fastest way is using a variable:

LASTDATA=$(tail -n 500 filename)
echo "${LASTDATA}" > filename

Answer 4

DIR=/path/to/my/dir # log directory
TMP=/tmp/tmp.log # temporary file
for f in `find ${DIR} -type f -depth 1 -name \*.log` ; do
  tail -n 500 $f > /tmp/tmp.log
  mv /tmp/tmp.log $f
done

Answer 5

In bash you loop over files in a directory, eg like this:

cd target/directory

for filename in *log; do
    echo "Cutting file $filename"
    tail -n 500 $filename > $filename.cut
    mv $filename.cut $filename
done