I'd like to sort files in S3 folders and then check if files contain a certain string.
When I usually want to grep a file I do the following:
aws s3 cp s3://s3bucket/location/file.csv.gz - | zcat | grep 'string_to_find'
I see I can sort files like this:
aws s3api list-objects-v2 \
--bucket s3bucket \
--prefix location \
--query 'reverse(sort_by(Contents,&LastModified))'
Tried something like this so far but got broken pipe:
aws s3api list-objects-v2 \
--bucket s3bucket \
--prefix location \
--query 'reverse(sort_by(Contents,&LastModified))' | cp - | zcat | grep 'string_to_find'
You can specify which fields to output and force them into text-only:
aws s3api list-objects-v2 \
--bucket s3bucket \
--prefix location \
--query 'reverse(sort_by(Contents,&LastModified))[].[Key]' \
--output text
Basically, the sort_by
and reverse
output the Contents
array, and this extracts the Key
element. I put [Key]
in square brackets to force each result onto its own line.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.