Best approach serializing large array object from c# to CosmoDB

Question

So i have a large c# object, a collection of data. I was initially simply writing it to a documentdb doc but realised that it would exceed the size of a single document. So not sure what the best approach is, i could:

1) Write it out as separate documents but then would have 10,000s of small documents and i suspect that would be super expensive to work with

2) Find a better way of serialising it but then i would still hit the limit

3) Find a way to break it into pages of data

4) Serialize to binary and use table storage or similar

5) Serialize to binary and use the attachments feature.

I have no idea what the best approach would be, i'm just finding my feet with CosmoDB - so any pointers would be hugely useful.

There's an argument that says this data should be in a SQL style rows and columns database but these days i don't even know if that's a modern approach...

cheers Paul

Answer 1

You can either split the documents as mentioned in your option #1 or store the document as remote attachments. I mentioned remote as the built-in attachments storage of DocumentDB is limited to 2 GB only.

To use remote storage, you need to store the file first then get the link and create an attachment in DocumentDB then use the MediaLink property of Attachment to set where the binary resides.

Here is the reference look for MediaLink .

Answer 2

Looking in more detail.

I think splitting my data between documentdb and table storage makes more sense - some of the data would fit into the table concept more, hundreds of thousands of rows that could be paged and read in with a core data structure in documentdb.

The only fly in the ointment is that there's no NuGet library for Cosmo Table service and dot net core... Can't quite believe that.

So i'm left looking at DynamoDB on amazon as what appears to be a table/document store...

thanks Paul