How i can process my payload to insert bulk data in multiple tables with atomicity/consistency in cassandra?

Question

I have to design the database for customers having prices for millions of materials they acquire through multiple suppliers for the next 24 months. So the database will store prices on a daily basis for every material supplied by a specific supplier for the next 24 months. Now I have multiple use cases to solve so I created multiple tables to solve each use case in the best possible way. Now the insertion of data into these tables will happen on a regular basis in a big chunk (let's say for 1k items), which should ensure the data consistency as well ie the data should be inserted into all the tables or in none of them. Failure in doing so should be flagged as a "failure" with no inserts for further action. How can I solve this in Cassandra effectively?

On option I can think of is to use small BATCH processes (1K in number for 1k items for example). I might hit multiple partitions during insertion in different tables having a different set of primary keys;

Any Thoughts? Thanks

Answer 1

If you are talking about with respect of Database(Cassandra) then you should consider many things for data modelling point. You need to go through the data modeling detail on below link with batch. https://docs.datastax.com/en/dse/6.0/cql/cql/ddl/dataModelingCQLTOC.html https://docs.datastax.com/en/dse/6.0/cql/cql/cql_reference/cql_commands/cqlBatch.html

Also, based on application nature you should think about compaction strategy for processing of high writes or reads.