Writing to a CSV file in an S3 bucket using boto 3

Question

I'm working on a project that needs to update a CSV file with user info periodically. The CSV is stored in an S3 bucket so I'm assuming I would use boto3 to do this. However, I'm not exactly sure how to go about this- would I need to download the CSV from S3 and then append to it, or is there a way to do it directly? Any code samples would be appreciated.

Answer 1

Ideally this would be something where DynamoDB would work pretty well (as long as you can create a hash key). Your solution would require the following.

1. Download the CSV
2. Append new values to the CSV Files
3. Upload the CSV.

A big issue here is the possibility (not sure how this is planned) that the CSV file is updated multiple times before being uploaded, which would lead to data loss.

Using something like DynamoDB, you could have a table, and just use the put_item api call to add new values as you see fit. Then, whenever you wish, you could write a python script to scan for all the values and then write a CSV file however you wish