How to copy multiple files matching name pattern to AWS S3 bucket using AWS CLI?

Question

I would like to copy files matching a file name pattern from my machine to an AWS S3 bucket using AWS CLI . Using the standard unix file name wildcards does not work:

$ aws s3 cp *.csv s3://wesam-data/

Unknown options: file1.csv,file2.csv,file3.csv,s3://wesam-data/

I followed this SO answer addressing a similar problem that advises using the --exclude and --include filters as explained here as shown below without success.

$ aws s3 cp . s3://wesam-data/ --exclude "*" --include "*.csv"

Answer 1

Solution

$ aws s3 cp . s3://wesam-data/ --exclude "*" --include "*.csv" --recursive

Explanation

It turns out that I have to use the --recursive flag with the --include & --exclude flags since this is a multi-file operation.

The following commands are single file/object operations if no --recursive flag is provided.

• cp
• mv
• rm