count the number of files copied recursively using aws cli commands

Question

Could someone help me with the command to count the number of files only copied from one parent object to another parent object. For example

1. src/folder1/some1.txt    
2. src/folder1/folder1_1/some1.txt
3. src/folder2/folder2_1/some2.txt
4. src/folder4/folder4_2/some3.txt
5. src/folder4/folder4_1/some4.txt

copy over them to dest folder

So, when I do

aws s3 cp s3://src/* s3://dest/ --recursive

I need a count of the number of files copied.

Answer 1

Lets say we have mp3 files in a S3 bucket with the following path: bucket-name/path/to/files Then the cmd aws s3 ls bucket-name/path/to/files --recursive | grep 'mp3' | wc -l aws s3 ls bucket-name/path/to/files --recursive | grep 'mp3' | wc -l aws s3 ls bucket-name/path/to/files --recursive | grep 'mp3' | wc -l will output the count of all mp3 files in that path even if they are present within directories.

Answer 2

The simplest way I can think of is to rely on the fact that cp -v prints each file that it copies in a format like `source' -> `dest' :

cp --recursive --verbose s3://src/* s3://dest/ | grep -F '->' | wc -l

grep will output something for each line where -> is found, and wc will count the lines, which will give you the total number of copies made.