Pivotal GemFire：Spring Data GemFire中的PDX序列化器配置

Question

I have created a GemFire cluster with 2 Locators, 2 cache servers and a "Customer" REPLICATE Region. (Domain object class is placed in classpath during server startup).

I am able to run a Java program (Peer) to load the "Customer" Region in the cluster. Now we want to move to Spring Data GemFire where I am not sure how to configure PDX serialization and getting...

com.gemstone.gemfire.InternalGemFireException: java.io.NotSerializableException: com.gemfire.poc.DomainObjects.Customer

cache.xml in simple Java program...

<?xml version="1.0" encoding="UTF-8"?><cache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schema.pivotal.io/gemfire/cache" xsi:schemaLocation="http://schema.pivotal.io/gemfire/cache http://schema.pivotal.io/gemfire/cache/cache-8.1.xsd" version="8.1" lock-lease="120" lock-timeout="60" search-timeout="300" is-server="false" copy-on-read="false">
 <pdx>
    <pdx-serializer>
      <class-name>
       com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer
      </class-name>
    <parameter name="classes">
      <string>com.gemfire.poc.DomainObjects.*</string>
    </parameter>
  </pdx-serializer>
 </pdx>
    <region name="Customer" refid="REPLICATE">
    <region-attributes refid="REPLICATE" scope="distributed-no-ack"> 
     <cache-loader>
        <class-name>com.citigroup.pulse.pt.gemfire.poc.clientserver.SimpleCacheLoader</class-name>
    </cache-loader>
</region-attributes>
    </region>
</cache>

spring-context.xml in Spring Boot app...

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:gfe="http://www.springframework.org/schema/gemfire"
       xmlns:util="http://www.springframework.org/schema/util"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
                http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/gemfire http://www.springframework.org/schema/gemfire/spring-gemfire.xsd
                http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
                http://www.springframework.org/schema/data/gemfire 
                           http://www.springframework.org/schema/data/gemfire/spring-data-gemfire.xsd">

    <util:properties id="gemfireProperties">
        <prop key="log-level">config</prop>
        <prop key="locators">hostA[10334],hostB[10334]</prop>
    </util:properties>


    <bean id="mappingPdxSerializer" class="com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer"/>

    <gfe:cache use-bean-factory-locator="false" properties-ref="gemfireProperties" use-cluster-configuration="true" pdx-serializer-ref="mappingPdxSerializer" />

    <gfe:replicated-region id="Customer" ignore-if-exists="true">       
    </gfe:replicated-region>


</beans>

Can someone help me fix the serialization issue?

Caused by: com.gemstone.gemfire.InternalGemFireException: java.io.NotSerializableException: com.gemfire.poc.DomainObjects.Customer
    at com.gemstone.gemfire.distributed.internal.DistributionManager.putOutgoing(DistributionManager.java:1954)
    at com.gemstone.gemfire.internal.cache.DistributedCacheOperation.distribute(DistributedCacheOperation.java:476)
    at com.gemstone.gemfire.internal.cache.AbstractUpdateOperation.distribute(AbstractUpdateOperation.java:65)
    at com.gemstone.gemfire.internal.cache.DistributedRegion.distributeUpdate(DistributedRegion.java:519)
    at com.gemstone.gemfire.internal.cache.DistributedRegion.basicPutPart3(DistributedRegion.java:500)
    at com.gemstone.gemfire.internal.cache.AbstractRegionMap.basicPut(AbstractRegionMap.java:3052)
    at com.gemstone.gemfire.internal.cache.LocalRegion.virtualPut(LocalRegion.java:5838)

Precisely, how to add "classes" parameter of ReflectionBasedAutoSerializer in spring-data-gemfire tags?

PDX deserialization exception while retrieving value from Region:

com.gemstone.gemfire.ToDataException: PdxSerializer failed when calling toData on class javax.management.Notification
    at com.gemstone.gemfire.internal.InternalDataSerializer.writePdx(InternalDataSerializer.java:3130)
    at com.gemstone.gemfire.internal.InternalDataSerializer.writeUserObject(InternalDataSerializer.java:1520)
    at com.gemstone.gemfire.internal.InternalDataSerializer.writeWellKnownObject(InternalDataSerializer.java:1416)
    at com.gemstone.gemfire.internal.InternalDataSerializer.basicWriteObject(InternalDataSerializer.java:2208)
    at com.gemstone.gemfire.DataSerializer.writeObject(DataSerializer.java:3181)
    at com.gemstone.gemfire.internal.util.BlobHelper.serializeToBlob(BlobHelper.java:50)
    at com.gemstone.gemfire.internal.util.BlobHelper.serializeToBlob(BlobHelper.java:38)
    at com.gemstone.gemfire.internal.cache.UpdateOperation$UpdateMessage.toData(UpdateOperation.java:492)
    at com.gemstone.gemfire.internal.InternalDataSerializer.invokeToData(InternalDataSerializer.java:2407)
    at com.gemstone.gemfire.internal.InternalDataSerializer.writeDSFID(InternalDataSerializer.java:1378)
    at com.gemstone.gemfire.internal.tcp.MsgStreamer.writeMessage(MsgStreamer.java:239)
    at com.gemstone.gemfire.distributed.internal.direct.DirectChannel.sendToMany(DirectChannel.java:458)
    at com.gemstone.gemfire.distributed.internal.direct.DirectChannel.sendToOne(DirectChannel.java:310)
    at com.gemstone.gemfire.distributed.internal.direct.DirectChannel.send(DirectChannel.java:696)
    at com.gemstone.gemfire.distributed.internal.membership.jgroup.JGroupMembershipManager.directChannelSend(JGroupMembershipManager.java:2929)
    at com.gemstone.gemfire.distributed.internal.membership.jgroup.JGroupMembershipManager.send(JGroupMembershipManager.java:3163)
    at com.gemstone.gemfire.distributed.internal.DistributionChannel.send(DistributionChannel.java:79)
    at com.gemstone.gemfire.distributed.internal.DistributionManager.sendOutgoing(DistributionManager.java:3907)
    at com.gemstone.gemfire.distributed.internal.DistributionManager.sendMessage(DistributionManager.java:3948)
    at com.gemstone.gemfire.distributed.internal.DistributionManager.putOutgoing(DistributionManager.java:1951)
    at com.gemstone.gemfire.internal.cache.DistributedCacheOperation.distribute(DistributedCacheOperation.java:476)
    at com.gemstone.gemfire.internal.cache.AbstractUpdateOperation.distribute(AbstractUpdateOperation.java:65)
    at com.gemstone.gemfire.internal.cache.DistributedRegion.distributeUpdate(DistributedRegion.java:519)
    at com.gemstone.gemfire.internal.cache.DistributedRegion.basicPutPart3(DistributedRegion.java:500)
    at com.gemstone.gemfire.internal.cache.ProxyRegionMap.basicPut(ProxyRegionMap.java:242)
    at com.gemstone.gemfire.internal.cache.LocalRegion.virtualPut(LocalRegion.java:5838)
    at com.gemstone.gemfire.internal.cache.DistributedRegion.virtualPut(DistributedRegion.java:387)
    at com.gemstone.gemfire.internal.cache.LocalRegionDataView.putEntry(LocalRegionDataView.java:118)
    at com.gemstone.gemfire.internal.cache.LocalRegion.basicPut(LocalRegion.java:5228)
    at com.gemstone.gemfire.internal.cache.LocalRegion.validatedPut(LocalRegion.java:1599)
    at com.gemstone.gemfire.internal.cache.LocalRegion.put(LocalRegion.java:1582)
    at com.gemstone.gemfire.internal.cache.AbstractRegion.put(AbstractRegion.java:327)
    at com.gemstone.gemfire.management.internal.ManagementResourceRepo.putEntryInLocalNotificationRegion(ManagementResourceRepo.java:169)
    at com.gemstone.gemfire.management.internal.NotificationHub$NotificationHubListener.handleNotification(NotificationHub.java:193)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor$ListenerWrapper.handleNotification(DefaultMBeanServerInterceptor.java:1754)
    at javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:275)
    at javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:352)
    at javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:337)
    at javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:248)
    at com.gemstone.gemfire.management.internal.beans.ManagementAdapter.handleRegionRemoval(ManagementAdapter.java:879)
    at com.gemstone.gemfire.management.internal.beans.ManagementListener.handleEvent(ManagementListener.java:123)
    at com.gemstone.gemfire.distributed.internal.InternalDistributedSystem.notifyResourceEventListeners(InternalDistributedSystem.java:2252)
    at com.gemstone.gemfire.distributed.internal.InternalDistributedSystem.handleResourceEvent(InternalDistributedSystem.java:506)
    at com.gemstone.gemfire.internal.cache.LocalRegion.basicDestroyRegion(LocalRegion.java:6642)
    at com.gemstone.gemfire.internal.cache.DistributedRegion.basicDestroyRegion(DistributedRegion.java:1957)
    at com.gemstone.gemfire.internal.cache.LocalRegion.close(LocalRegion.java:2219)
    at org.springframework.data.gemfire.RegionFactoryBean.destroy(RegionFactoryBean.java:529)
    at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:272)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:961)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523)
    at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.destroySingletons(FactoryBeanRegistrySupport.java:230)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:968)
    at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1032)
    at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1008)
    at org.springframework.context.support.AbstractApplicationContext$2.run(AbstractApplicationContext.java:929)
Caused by: org.springframework.data.mapping.model.MappingException: Could not write value for property protected transient java.lang.Object java.util.EventObject.source
    at org.springframework.data.gemfire.mapping.MappingPdxSerializer$2.doWithPersistentProperty(MappingPdxSerializer.java:188)
    at org.springframework.data.gemfire.mapping.MappingPdxSerializer$2.doWithPersistentProperty(MappingPdxSerializer.java:173)
    at org.springframework.data.mapping.model.BasicPersistentEntity.doWithProperties(BasicPersistentEntity.java:309)
    at org.springframework.data.gemfire.mapping.MappingPdxSerializer.toData(MappingPdxSerializer.java:173)
    at com.gemstone.gemfire.internal.InternalDataSerializer.writePdx(InternalDataSerializer.java:3075)
    ... 56 more
Caused by: com.gemstone.gemfire.pdx.PdxFieldAlreadyExistsException: The field "source" already exists.
    at com.gemstone.gemfire.pdx.internal.PdxType.addField(PdxType.java:262)
    at com.gemstone.gemfire.pdx.internal.PdxWriterImpl.updateMetaData(PdxWriterImpl.java:858)
    at com.gemstone.gemfire.pdx.internal.PdxWriterImpl.updateMetaData(PdxWriterImpl.java:851)
    at com.gemstone.gemfire.pdx.internal.PdxWriterImpl.writeObject(PdxWriterImpl.java:303)
    at com.gemstone.gemfire.pdx.internal.PdxWriterImpl.writeField(PdxWriterImpl.java:705)
    at com.gemstone.gemfire.pdx.internal.PdxWriterImpl.writeField(PdxWriterImpl.java:625)
    at org.springframework.data.gemfire.mapping.MappingPdxSerializer$2.doWithPersistentProperty(MappingPdxSerializer.java:184)
    ... 60 more

Answer 1

You have a couple of options here, along with a few suggested recommendations.

1) First, I would not use Pivotal GemFire's oagpdx.ReflectionBasedAutoSerializer . Rather SDG has a much more robust PdxSerializer implementation based on Spring Data's Mapping Infrastructure (ie the osdgmapping.MappingPdxSerializer ).

In addition, SDG's MappingPdxSerializer allows you to register custom PdxSerializer's on an entity field/property case-by-case basis. Imagine if your Customer class has a reference to a complex Address class and that class has special serialization needs.

Furthermore, SDG's MappingPdxSerializer can handle transient and read-only properties.

Finally, you don't have to mess with any fussy/complex Regex to properly identify the application domain model types that need to be serialized.

2) Second, you can leverage Spring's JavaConfig along with SDG's new Annotation-based configuration model to configure Pivotal GemFire PDX Serialization as simply as this...

@SpringBootApplication
@PeerCacheApplication
@EnablePdx(..)
class MySpringBootApacheGeodeApplication {
  ...
}

That is, using the SDG @EnablePdx annotation.

More details on #1 and #2 above are available here and here .

Of course, the later is more applicable when using Pivotal GemFire 9.x+ with Spring Data GemFire Kay ( 2.0+ ). Judging by the package in your configuration of the com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer from your XML config (ie the com.gemstone.gemfire package) it would appear you are using Pivotal GemFire 8.2.x with Spring Data GemFire Ingalls (or 1.9.x.RELEASE ), perhaps?

However, if you insist on, or are required to use XML for your configuration, then you can do the following...

<beans ...>

  <bean id="mappingPdxSerializer" class="org.springframework.data.gemfire.mapping.MappingPdxSerializer"/>

  <gfe:cache pdx-serializer-ref="mappingPdxSerializer" .../>

</beans>

And, if you really want to use Pivotal GemFire's ReflectionBasedAutoSerializer , then you can find examples of it's use in the SDG test suite. For instance .

I also have a few more examples in my spring-gemfire-test project/repo (which is quite a mess and I don't maintain this repo much anymore, as a warning). Examples here , using Java configuration with GemFire's API here , here , here which also shows the use of SDG's MappingPdxSerializer as well (by comparison), and so on and so forth. Many examples riddle throughout my repos.

Hope this helps!

Cheers, -John