How to keep track of data uploaded by user with AWS?

Question

I'm building a platform whereby users upload data to us. Users will upload data to us, whereby the amount of data is appropriately 750 MB for transaction.

What is the standard way to keep track of the data uploaded by user? How do you organize the data by user on S3 buckets?

A simple scheme could be to tag/prefix each item uploaded to the S3 buckets by the username, and use this logic in our application to allow users to work with these files. We can then keep track of username + data uploaded within a database (like AWS Dynamo).

Things get a bit complicated when I start thinking about features allowing groups to access these files of course....

Is there a better approach for this task on AWS? It feels like a standard problem.

Answer 1

AWS does not have build-in tools for keeping track of uploads per "user" nor any upload limits. This is what you, as a developer on AWS, need to design and implement. DynamoDB is a popular choice to keep track of S3 uploads and limits per user in your application.

Regarding organization. Well it depends. If your users login through Cognito to your application, each user will have IAM federated identity associated with them. Thus, you can organize the bucket and control user access using this feature, as shown for instance in the following link:

• Amazon S3: Allows Amazon Cognito users to access objects in their bucket

User groups, could also be managed through Congito.