Combine all csv files into one single csv inside AWS CLI

Question

I have a S3 bucket with a folder called batches. Inside the batches folder, I have 20 CSV files. Using AWS ClI (or a bash file to be exact), how can I combine all these csv files into a single CSV file and move it up one folder.

Normally in terminal this is how I do it:

cd batches && cat ./*csv > combined.csv

What would be a comparable way to do this for an S3 bucket inside AWS CLI?

Answer 1

If you only have 20 CSV files in total, then Marcin's suggestion is probably the right way to go. If you have more bigger-sized CSV, then I would suggest taking advantage of multipart upload in S3 and processing them in aws lambda.