Copy multiple files from hadoop to s3 bucket

Question

I have couple of files in Hadoop directory. I am trying to copy files from hadoop directory to s3 bucket.

List of files

sample1.txt

sample2.txt

sample3.txt

I want to copy all files at once

 aws s3 cp *.txt s3://bucket-name/samples/

It shows me error

usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]

Answer 1

You can do something like this :

aws s3 cp . s3://<bucket-name> --recursive --exclude "*" --include "*.txt"
aws s3 cp <Hadoop-Dir-Path> s3://<bucket-name> --recursive --exclude "*" --include "*.txt"

Or you can use sync as well :

aws s3 sync . s3://<bucket-name> --exclude "*" --include "file.*"

Note that, by default, all files are included. This means that providing only an --include filter will not change what files are transferred. --include will only re-include files that have been excluded from an --exclude filter. If you only want to upload files with a particular extension, you need to first exclude all files, then re-include the files with the particular extension. This command will upload only files ending with .txt.

AWS Documentation

AWS Documenation for sync