JMS隊列連接的性能

Question

在春季批處理中，我正在使用Oracle DB和IBM WMQ，並實現了Atomikos事務管理和Spring分區。

要求是，需要從數據庫中讀取200萬條記錄，並經過處理，將相同的消息（1條消息的大小= 2kb）發布到WMQ中，並更新數據庫中的200萬條記錄。

以下是我的代碼配置。 隊列配置：

<bean id="atomikosJmsConnectionFactory" class="com.atomikos.jms.AtomikosConnectionFactoryBean" init-method="init" destroy-method="close">
        <property name="uniqueResourceName" value="WMQ_MQSeries_XA_RMI" />
        <property name="xaConnectionFactory">
            <bean class="com.ibm.mq.jms.MQXAConnectionFactory">
                <property name="connectionNameList" value="127.0.0.1(1414)" />
                <property name="queueManager" value="LWMQ" />
                <property name="channel" value="LCHNL" />
                <property name="transportType">
                  <util:constant static-field="com.ibm.msg.client.wmq.WMQConstants.WMQ_CM_CLIENT" />
                </property>
                <property name="clientReconnectOptions">
                  <util:constant static-field="com.ibm.msg.client.wmq.WMQConstants.WMQ_CLIENT_RECONNECT" />
                </property>
            </bean>
        </property>
        <property name="ignoreSessionTransactedFlag" value="${mq.ignoreSessionTransactedFlag:false}"></property>
        <property name="borrowConnectionTimeout" value="300"></property>
        <property name="maxPoolSize" value="50"></property>
    </bean>

     <bean id="jmsTemplate"  class="org.springframework.jms.core.JmsTemplate">
        <property name="connectionFactory" ref="atomikosJmsConnectionFactory" />
        <property name="sessionTransacted" value="true"/>
        <property name="sessionAcknowledgeMode" value="0" />
        <property name="receiveTimeout" value="2000" />
    </bean>

`

Spring Job配置：

<batch:step id="masterStep">
      <batch:partition step="slaveStep" partitioner="customPartitioner">
        <batch:handler task-executor="taskExecutor" />
      </batch:partition>
    </batch:step>
    <batch:listeners>
      <batch:listener ref="loadJobListener" />
    </batch:listeners>
  </batch:job>

  <batch:step id="slaveStep">
    <batch:tasklet transaction-manager="atomikosTransactionManager">
      <batch:chunk chunk-completion-policy="chunkCompletionPolicy" />
    </batch:tasklet>
  </batch:step>

<bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor" >
    <property name="threadNamePrefix" value="L-Thread-"></property>
    <property name="concurrencyLimit" value="20"></property>
  </bean>

 <bean id="chunkTimeout"
    class="org.springframework.batch.repeat.policy.TimeoutTerminationPolicy">
    <constructor-arg value="36000" />
  </bean>

  <bean id="commitCount"
    class="org.springframework.batch.repeat.policy.SimpleCompletionPolicy">
    <property name="chunkSize" value="100" />
  </bean>

<bean id="chunkCompletionPolicy"
    class="org.springframework.batch.repeat.policy.CompositeCompletionPolicy">
    <property name="policies">
      <list>
        <ref bean="commitCount" />
        <ref bean="chunkTimeout" />
      </list>
    </property>
  </bean>

這是我的Linux服務器配置：型號：Intel（R）Xeon（R）CPU v4 @ 2.60GHz體系結構：x86_64 CPU數量：16 RAM：100GB

目前，這項工作需要120分鍾才能完成任務（包括200萬條消息發布到MQ中並更新到數據庫中）

我的觀察：大多數時候，線程處於等待MQ連接的狀態。 以下是threaddump的Visual-VM屏幕快照。

"L-Thread-4" #101 prio=5 os_prio=0 tid=0x000000002349e000 nid=0xcabc waiting on condition [0x00000000324cd000]
   java.lang.Thread.State: TIMED_WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x0000000771afc540> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at com.ibm.ws.xs.xio.actor.impl.FutureImpl.await(FutureImpl.java:268)
    at com.ibm.ws.xs.xio.actor.impl.FutureImpl.get(FutureImpl.java:316)
    at com.ibm.ws.xsspi.xio.dispatch.MessageDispatcher.askAndGet(MessageDispatcher.java:373)
    at com.ibm.ws.xs.xio.actor.impl.ActorRefImpl.askAndGet(ActorRefImpl.java:106)
    at com.ibm.ws.objectgrid.client.XIOClientCoreMessageHandler.sendMessageInternal(XIOClientCoreMessageHandler.java:372)
    at com.ibm.ws.objectgrid.client.XIOClientCoreMessageHandler.sendMessage(XIOClientCoreMessageHandler.java:142)
    at com.ibm.ws.objectgrid.client.CommonClientCoreMessageHandler.sendGetRequest(CommonClientCoreMessageHandler.java:229)
    at com.ibm.ws.objectgrid.client.RemoteTransactionCallbackImpl.mapGet(RemoteTransactionCallbackImpl.java:1953)
    at com.ibm.ws.objectgrid.client.RemoteCacheLoader.get(RemoteCacheLoader.java:497)
    at com.ibm.ws.objectgrid.map.BaseMap.getRaw(BaseMap.java:4846)
    at com.ibm.ws.objectgrid.DiffMap.lookForKey(DiffMap.java:1391)
    at com.ibm.ws.objectgrid.DiffMap.get(DiffMap.java:981)
    at com.ibm.ws.objectgrid.ObjectMapImpl.get(ObjectMapImpl.java:451)
    at com.ibm.ws.objectgrid.ObjectMapImpl.get(ObjectMapImpl.java:411)
    at com.ibm.websphere.objectgrid.spring.ObjectGridCache.get(ObjectGridCache.java:290)

我該如何解決此問題以加快流程速度？

我需要在1小時內完成此工作流程。

Answer 1

您應該使用CachingConnectionFactory以避免為每個發送打開新的連接。