Uploading file to S3 Bucket from /tmp/

Question

I automated my apache http logs to be stored in the tmp directory, which is running the script correctly until moving the file to the tmp directory. Now I have added few more lines of script to move those.tar files to the S3 bucket. When I perform this command manually they are getting moved to the s3 bucket but I do not want to do it everyday since its a daily job and I would want to automate them.

The.tar file is present in the /tmp/ directory when I manually go there and look but the awscl fails to locate to it.

The error I am getting is: the user provided path does not exist when I run the script.

the lines of code I added were these,

sudo apt update -y
sudo apt install apache2
sudo ufw allow 'Apache'
sudo systemctl start apache2
myname="abcd"
sudo tar -cvf $myname-httpd-logs-date +'%d%m%Y-%H%M%S'.tar /var/log/apache2/*.log
sudo mv *.tar /tmp/
sudo apt install awscli -y
s3_bucket="s3_test"    
aws s3 \
    cp /tmp/$myname-httpd-logs-$(date +'%d%m%Y-%H%M%S').tar \
    s3://$s3_bucket/$myname-httpd-logs-date +'%d%m%Y-%H%M%S'.tar

Can anyone help me out in figuring out why this error has occurred and how to fix it.

Error

Answer 1

The problem will have to be your second line.

aws s3 \
cp /tmp/$myname-httpd-logs-$(date +'%d%m%Y-%H%M%S).tar \
s3://$s3_bucket/$myname-httpd-logs-${timestamp}.tar

Change that to:

aws s3 \
cp /tmp/$myname-httpd-logs-${timestamp}.tar \
s3://$s3_bucket/$myname-httpd-logs-${timestamp}.tar

Answer 2

There is not needed ' in your command in $(date +'%d%m%Y-%H%M%S) . It should be:

aws s3 \
cp /tmp/$myname-httpd-logs-$(date +%d%m%Y-%H%M%S).tar \
s3://$s3_bucket/$myname-httpd-logs-${timestamp}.tar

Answer 3

And if you want the unneeded quote

aws s3 \
cp /tmp/$myname-httpd-logs-$(date +'%d%m%Y-%H%M%S').tar \
s3://$s3_bucket/$myname-httpd-logs-${timestamp}.tar

be sure to close it (the syntax color gives you a clue).

Answer 4

You appear to be calling datetime multiple times to construct different filenames. Rather than calling it multiple times, if you call it once, you will ensure that each time you refer to the tar filename, it uses the same name:

myname="abcd"
target_name=$myname-httpd-logs-`date +'%d%m%Y-%H%M%S'.tar`
sudo tar -cvf $target_name /var/log/apache2/*.log
sudo mv $target_name /tmp/

s3_bucket="s3_test"    
aws s3 \
    cp /tmp/$target_name \
    s3://$s3_bucket/$target_name