concatenate files and remove source file

Question

I have this command to concate files matching a pattern, but I do want to remove them, and I want to prevent the case when a file that just created should be just deleted (and no concatenation)

sample files names:

start-2014-03-25-08-08.log
scheduled-2014-03-19-13-03.log
scheduled-2014-03-19-14-58.log

command used

ls -1 | sed -r "s/(.*)-[0-9]{4}(-[0-9]{2})+/cat \1* >> \1$(date  +"-%Y-%m-%d-%H-%M")/" | uniq | cat

output is:

cat start* >> start-2014-03-26-12-26.log
cat scheduled* >> scheduled-2014-03-26-12-26.log

but I do want to remove the files once they have been appended. Since the files are large, it could be a slight chance of delay that meanwhile appending a new "save pattern" file is created and I do not want to remove that one.

What would be the correct way?

Update

I have this now.

 rm -f temp.files;ls -1 *.log > temp.files; cat temp.files | sed -r "s/(.*)-[0-9]{4}(-[0-9]{2})+\.log/cat \1* >> \1$(date  +"-%Y-%m-%d-%H-%M").log/" | uniq | sh; xargs rm -rf < temp.files; rm -f temp.files

Answer 1

Since you generated that cat command using sed and later pipe it to sh , you could modify the sed expression so as to instruct sh to delete the file if it was appended successfully, ie, change the replacement expression to:

cat \1* >> \1$(date  +"-%Y-%m-%d-%H-%M").log \&\& rm -f &

from

cat \1* >> \1$(date  +"-%Y-%m-%d-%H-%M").log

Note that you need to escape the & in the replacement in order to produce the literal & and & by itself would be the entire match (the input filename in your case).

This would also obviate the need of rm -rf < temp.files in your command since every file would be removed after being appended.

Answer 2

No temporary files needed.

ts=$(date +"-%Y-%m-%d-%H-%M")
for f in *; do
    prefix=${f%%-*}
    cat "$f" >> "$prefix-$ts"
    rm "$f"
done

Since it's possible for the loop to take more than a minute to run, I set ts outside the loop so that the same minute is always used. You can move that assignment inside the loop if you want different output files depending on when the concatenation actually takes place.

Answer 3

最后我有这个：

 rm -f temp.files;ls -1 *.log > temp.files; cat temp.files | sed -r "s/(.*)-[0-9]{4}(-[0-9]{2})+\.log/cat \1* >> \1$(date  +"-%Y-%m-%d-%H-%M").log/" | uniq | sh; xargs rm -rf < temp.files; rm -f temp.files