how to have transactions across main thread and entry listener in hazelcast?

Question

I have a network of nodes represented with a graph ( or more specifically a dag). The vertex and edges are just ids pointing to large objects in the cache.

I am using hazelcast and have 2 caches:
1. ReferenceObject for the graph
2. IMap for the large objects
3. IMap for the edges in the graph

When I insert a large object, I have an entry listener that will update this graph in the cache. Similarly when I add an edge data, there is also an entry listener that will update the graph.

However I have 1 problem where if I create an edge and it creates cycle, it fails (as it's a dag) but the IMap retains the records.

Any ideas how I can have transactions across the main thread and the entry listener?

Answer 1

@Pilo, the problem is EntryListener listens to events fired after data already populated in the map. So when you insert the data to your first map & listen to an update event, data is already in the first map.

You can either

1. Manually remove the record from the first map if the operation fails on the second one.
2. Use transactions & make sure either all or none of the maps updated, instead of using listeners.

Answer 2

Though it's a completely different approach, have you looked at Hazelcast Jet? It's a DAG based event stream processing engine build on top of Hazelcast IMDG. It might fit your use case better and take care of the lower level stuff for you.

https://jet.hazelcast.org

You would have a Jet cluster, which is also a Hazelcast cluster, but you get all the processing stuff with it. It extends the Java Streams programming model so you just write your app as if it was a Java stream and run it on the cluster. Something to think about anyway.