How can I transfer a remote file to my S3 bucket using AWS CLI?

Question

I tried to follow advice provided at https://stackoverflow.com/a/18136205/6608952 but was unsure how to share myAmazonKeypair path in a .pem file on the remote server.

scp -i yourAmazonKeypairPath.pem  fileNameThatYouWantToTransfer.php ec2-user@ec2-00-000-000-15.us-west-2.compute.amazonaws.com:

The command completed after a few minutes with this display:

ssh: connect to host
myBucketEndpointName
port 22: Connection timed out
lost connection

I have a couple of very large files to transfer and would prefer not to have to download the files to my local computer and then re-upload them to the S3 bucket.

Any suggestions?

Answer 1

There is no direct way to upload files to S3 from a remote location. ie a URL

So to achieve that, you have two options :

1. Download the file on your local machine and then upload it via AWS Console or AWS CLI .
2. Download the file in AWS EC2 Instance and upload to S3 by AWS CLI .

The first method is pretty simple, not much explanation needed.

But for the second method, you'll need to do :

1. Create an EC2 Instance in the same region as the S3 Bucket is. Or if you already have an Instance, then login/ssh to it.
2. Download the file from the source to the EC2 Instance. via wget or curl whichever is comfortable.
3. Install AWS CLI on the EC2 Instance.
4. Create IAM User and Grant him Permission for your S3 Bucket.
5. Configure your AWS CLI with your IAM Credentials.
6. Upload your file to S3 Bucket with AWS CLI S3 CP Utility .
7. Terminate the Instance, if you set up the instance only for this .