在Wildfly 9 AS中实现Infinispan缓存（考虑集群）

Question

The situation:

I have a clearing table with multiple thousands of records. They are split into packages of eg 500 records. Then each packet is sent to the AS via Message Driven Beans. The AS calculates a key depending on the contents (eg currency, validStart, validEnd) of each record and needs to store this key in the database (together withe the combination of the contents).

The request:

To avoid duplicates i want a centralized "tool" which calculates the key and stores them and thus reduces communication with the database by caching those keys with the records.

Now I tried to use a local Infinispan cache accessed in a Utility-class-implementation for each package-processing-thread. This resulted in the fact, that multiple packages calculated the same key and thus duplicates were inserted in the database. Or sometimes I got deadlocks.

I tried to implement a "lock" via a static variable to block access for the cache during a database insert, but without success. Next attempt was to use a replicated- respectively distributed-Infinispan cache. This did not change the results in AS behavior.

My last idea would be to implement as a bean managed singleton session bean to acquire a transaction lock during inserting into the database.

The AS currently runs in standalone mode, but will be moved to a cluster in near future, so a High Availability solution is preferred.

Resuming:

What's the correct way to lock Infinispan cache access during creation of (Key, Value) pairs to avoid duplicates?

Update:

@cruftex: My Request is: I have a set of (Key, Value) pairs, which shall be cached. If an insert of a new record should happen, then an algorithm is applied to it and the Key is calculated. Then the cache shall be checked if the key already exists and the Value will be appended to the new record. But if the Value does not exist, it shall be created and stored in the database.

The cache needs to be realized using Infinispan because the AS shall run in a cluster. The algorithm for creating the Keys exists. Inserting the Value in the database too (via JDBC or Entities). But i have the problem, that using Message Driven Beans (and thus multithreading in the AS) the same (Key, Value) Pair is calculated in different threads and thus each thread tries to insert the Values in the database (which i want to avoid!).

@Dave:

public class Cache {
     private static final Logger log = Logger.getLogger(Cache.class);
     private final Cache<Key, FullValueViewer> fullCache;
     private HomeCache homes;        // wraps EntityManager
     private final Session session;

     public Cache(Session session, EmbeddedCacheManager cacheContainer, HomeCache homes) {
         this.session = session;
         this.homes = homes;
         fullCache = cacheContainer.getCache(Const.CACHE_CONDCOMBI);
     } 

     public Long getId(FullValueViewer viewerWithoutId) {
         Long result = null;

         final Key key = new Key(viewerWithoutId);
         FullValueViewer view = fullCache.get(key);

         if(view == null) {
             view = checkDatabase(viewerWithoutId);
             if(view != null) {
                 fullCache.put(key, view);
             }
         }

         if(view == null) {
             view = createValue(viewerWithoutId);

             // 1. Try
             fullCache.put(key, view);

             // 2. Try
             //      if(!fullCache.containsKey(key)) {
             //           fullCache.put(key, view);
             //       } else {
             //           try {
             //               homes.condCombi().remove(view.idnr);
             //           } catch (Exception e) {
             //               log.error("remove", e);
             //           }
             //       }

             // 3. Try
             //       synchronized(fullCache) {
             //           view = createValue(viewerWithoutId);
             //           fullCache.put(key, view);
             //       }
         }
         result = view.idnr;
         return result;
     }

     private FullValueViewer checkDatabase(FullValueViewer newView) {
         FullValueViewer result = null;
         try {
             CondCombiBean bean = homes.condCombi().findByTypeAndKeys(_parameters_);
             result = bean.getAsView();
         } catch (FinderException e) {
         }
         return result;
     }

     private FullValueViewer createValue(FullValueViewer newView) {
         FullValueViewer result = null;
         try {
             CondCombiBean bean = homes.condCombi().create(session.subpk);
             bean.setFromView(newView);
             result = bean.getAsView();
         } catch (Exception e) {
             log.error("createValue", e);
         }
         return result;
     }

     private class Key {

         private final FullValueViewer view;

         public Key(FullValueViewer v) {
            this.view = v;
         }

         @Override
         public int hashCode() {
             _omitted_
         }

         @Override
         public boolean equals(Object obj) {
             _omitted_
         }
     }
 }

The cache configurations i tried with Wildfly:

<cache-container name="server" default-cache="default" module="org.wildfly.clustering.server">
   <local-cache name="default">
      <transaction mode="BATCH"/>
   </local-cache>
</cache-container>

<cache-container name="server" default-cache="default" module="org.wildfly.clustering.server">
   <transport lock-timeout="60000"/>
   <distributed-cache name="default" mode="ASYNC"/>
</cache-container>

Answer 1

I'll react only to the resume question:

You can't lock whole cache; that wouldn't scale. The best way would be to use cache.putIfAbsent(key, value) operation, and generate different key if the entry is already there (or use list as value and replace it using conditional cache.replace(key, oldValue, newValue) ).

If you want to really prohibit writes to some key, you can use transactional cache with pessimistic locking strategy, and issue cache.getAdvancedCache().lock(key) . Note that there's no unlock: all locks are released when the transaction is committed/rolled back through transaction manager.

Answer 2

You cannot generate your own key and use it to detect duplicates at the same time.

Either each data row is guaranteed to arrive only once, or it needs embodied a unique identifier from the external system that generates it.

If there is a unique identifier in the data, which, if all goes wrong, and no id is in there, is just all properties concatenated, then you need to use this to check for duplicates.

Now you can go with that unique identifier directly, or generate an own internal identifier. If you do the latter, you need a translation from the external id to the internal id.

If duplicates arrive, you need to lock based on the external id when you generate the internal id, and then record what internal id you assigned.

To generate a unique sequence of long values, in a cluster, you can use the CAS-operations of the cache. For example something like this:

@NotThreadSafe
class KeyGeneratorForOneThread {

  final String KEY = "keySequenceForXyRecords";
  final int INTERVAL = 100;
  Cache<String,Long> cache = ...;
  long nextKey = 0;
  long upperBound = -1;

  void requestNewInterval() {
    do {
      nextKey = cache.get(KEY);
      upperBound = nextKey + INTERVAL;
    } while (!cache.replace(KEY, nextKey, upperBound));
  } 

  long generateKey() {
    if (nextKey >= upperBound) {
     requestNewInterval();
    }
    return nextKey++;
  }
}

Every thread has its own key generator and would generate 100 keys without needing coordination.

You may need separate caches for:

• locking by external id
• lookup from external to internal id
• sequence number, attention that is actually not a cache, since it must know the last number after a restart
• internal id to data

Answer 3

We found a solution that works in our case and might be helpful for somebody else out there:

We have two main components, a cache-class and a singleton bean.
The cache contains a copy of all records currently present in the database and a lot of logic.
The singleton bean has access to the infinispan-cache and is used for creating new records.

Initialy the cache fetches a copy of the infinispan-cache from the singleton bean. Then, if we search a record in the cache, we first apply a kind of hash-method, which calculates a unqiue key for the record. Using this key we can identify, if the record needs to be added to the database. If so, then the cache calls the singleton bean using a create-method with a @Lock(WRITE) Annotation. The create method first checks, if the value is contained in the infinispan-cache and if not, it creates a new record.

Using this approach we can guarantee, that even if the cache is used in multiple threads and each thread sends a request to create the same record in the database, the create process is locked and all following requests won't be proceeded because the value was already created in a previous request.