堆栈内存溢出
  简体   繁体   English

activemq jdbc性能/延迟

[英]activemq jdbc performance / delays

 原文  2014-07-16 06:18:07  5   1    java/ performance/ jdbc/ jms/ activemq

问题描述

I'm testing activemq with jdbc store and while I'm aware that it's not the most efficient approach, I see very weird delays. 我正在用jdbc store测试activemq，虽然我知道这不是最有效的方法，但我看到了很奇怪的延迟。

The code basically fills a queue with 3k messages and then single consumer consumes all of them. 该代码基本上将3k消息填充到队列中，然后单个使用者将所有消息消耗掉。 Messages are persistent, acknowledge mode is AUTO_ACKNOWLEDGE. 消息是持久的，确认模式为AUTO_ACKNOWLEDGE。 I also disabled alwaysSessionAsync and dispatchAsync. 我还禁用了alwaysSessionAsync和dispatchAsync。

The issue is that consumption speed varies a lot. 问题是消费速度变化很大。 Here's how long it takes to consume 100 messages (30x to consume it all): 这是消耗100条消息所需的时间（全部消耗30倍）： 

2014-07-16 01:08:50,171  'consumer': 45.1 ms
2014-07-16 01:08:50,207  'consumer': 35.5 ms
2014-07-16 01:08:50,290  'consumer': 83.4 ms
2014-07-16 01:08:50,412  'consumer': 122 ms
2014-07-16 01:08:50,476  'consumer': 63.1 ms
2014-07-16 01:08:50,552  'consumer': 75.6 ms
2014-07-16 01:08:50,606  'consumer': 54.5 ms
2014-07-16 01:08:50,655  'consumer': 48.7 ms
2014-07-16 01:08:50,709  'consumer': 53.2 ms
2014-07-16 01:08:50,765  'consumer': 56.2 ms
2014-07-16 01:08:50,813  'consumer': 48.2 ms
2014-07-16 01:08:50,922  'consumer': 109 ms
2014-07-16 01:08:51,188  'consumer': 266 ms
2014-07-16 01:08:51,446  'consumer': 257 ms
2014-07-16 01:08:51,724  'consumer': 278 ms
2014-07-16 01:08:51,975  'consumer': 252 ms
2014-07-16 01:08:52,224  'consumer': 249 ms
2014-07-16 01:08:52,496  'consumer': 271 ms
2014-07-16 01:08:52,743  'consumer': 247 ms
2014-07-16 01:08:52,998  'consumer': 255 ms
2014-07-16 01:08:53,255  'consumer': 257 ms
2014-07-16 01:08:53,486  'consumer': 231 ms
2014-07-16 01:08:53,710  'consumer': 224 ms
2014-07-16 01:08:53,947  'consumer': 236 ms
2014-07-16 01:08:54,189  'consumer': 242 ms
2014-07-16 01:08:54,389  'consumer': 200 ms
2014-07-16 01:08:54,598  'consumer': 209 ms
2014-07-16 01:08:54,790  'consumer': 192 ms
2014-07-16 01:08:54,985  'consumer': 195 ms
2014-07-16 01:08:55,203  'consumer': 217 ms

So I enabled logging on TRACE level to find out if there's anything interesting and indeed, there are gaps: 因此，我启用了TRACE级别的登录功能，以发现是否有任何有趣的东西，并且确实存在差距： 

2014-07-16 00:34:14,117 ActiveMQ BrokerService[broker_jdbc] Task-1 DEBUG || org.apache.activemq.broker.region.Queue - QUEUE2 toPageIn: 200, Inflight: 999, pagedInMessages.size 1214, pagedInPendingDispatch.size 214, enqueueCount: 3000, dequeueCount: 386, memUsage:1459248
2014-07-16 00:34:14,117 ActiveMQ BrokerService[broker_jdbc] Task-1 TRACE || org.apache.activemq.broker.region.Queue - Subscription full QueueSubscription: consumer=ID:LAPTOP-50558-1405488840024-5:1:1:1, destinations=1, dispatched=1000, delivered=0, pending=0
2014-07-16 00:34:14,117 ActiveMQ BrokerService[broker_jdbc] Task-1 TRACE || org.apache.activemq.thread.PooledTaskRunner - Running task iteration 572 - queue://QUEUE2, subscriptions=1, memory=1%, size=5771, in flight groups=null
2014-07-16 00:34:14,117 ActiveMQ BrokerService[broker_jdbc] Task-1 DEBUG || org.apache.activemq.broker.region.Queue - QUEUE2 toPageIn: 200, Inflight: 999, pagedInMessages.size 1214, pagedInPendingDispatch.size 214, enqueueCount: 3000, dequeueCount: 386, memUsage:1459248
2014-07-16 00:34:14,117 ActiveMQ BrokerService[broker_jdbc] Task-1 TRACE || org.apache.activemq.broker.region.Queue - Subscription full QueueSubscription: consumer=ID:LAPTOP-50558-1405488840024-5:1:1:1, destinations=1, dispatched=1000, delivered=0, pending=0
2014-07-16 00:34:14,117 ActiveMQ BrokerService[broker_jdbc] Task-1 TRACE || org.apache.activemq.thread.PooledTaskRunner - Running task iteration 573 - queue://QUEUE2, subscriptions=1, memory=1%, size=5771, in flight groups=null
2014-07-16 00:34:14,117 ActiveMQ BrokerService[broker_jdbc] Task-1 DEBUG || org.apache.activemq.broker.region.Queue - QUEUE2 toPageIn: 200, Inflight: 999, pagedInMessages.size 1214, pagedInPendingDispatch.size 214, enqueueCount: 3000, dequeueCount: 386, memUsage:1459248
2014-07-16 00:34:14,117 ActiveMQ BrokerService[broker_jdbc] Task-1 TRACE || org.apache.activemq.broker.region.Queue - Subscription full QueueSubscription: consumer=ID:LAPTOP-50558-1405488840024-5:1:1:1, destinations=1, dispatched=1000, delivered=0, pending=0
2014-07-16 00:34:14,117 ActiveMQ Transport: tcp:///127.0.0.1:50562@50559 TRACE || org.apache.activemq.broker.region.PrefetchSubscription - ack: MessageAck {commandId = 392, responseRequired = false, ackType = 2, consumerId = ID:LAPTOP-50558-1405488840024-5:1:1:1, firstMessageId = ID:LAPTOP-50488-1405487850038-3:1:1:1:2017, lastMessageId = ID:LAPTOP-50488-1405487850038-3:1:1:1:2017, destination = queue://QUEUE2, transactionId = null, messageCount = 1, poisonCause = null}
2014-07-16 00:34:14,397 ActiveMQ BrokerService[broker_jdbc] Task-1 TRACE || org.apache.activemq.thread.PooledTaskRunner - Running task iteration 574 - queue://QUEUE2, subscriptions=1, memory=1%, size=5771, in flight groups=null
2014-07-16 00:34:14,397 ActiveMQ BrokerService[broker_jdbc] Task-1 DEBUG || org.apache.activemq.broker.region.Queue - QUEUE2 toPageIn: 200, Inflight: 998, pagedInMessages.size 1213, pagedInPendingDispatch.size 214, enqueueCount: 3000, dequeueCount: 387, memUsage:1458216
2014-07-16 00:34:14,397 ActiveMQ Transport: tcp:///127.0.0.1:50562@50559 TRACE || org.apache.activemq.broker.region.Queue - ack of ID:LAPTOP-50488-1405487850038-3:1:1:1:2017 with MessageAck {commandId = 392, responseRequired = false, ackType = 2, consumerId = ID:LAPTOP-50558-1405488840024-5:1:1:1, firstMessageId = ID:LAPTOP-50488-1405487850038-3:1:1:1:2017, lastMessageId = ID:LAPTOP-50488-1405487850038-3:1:1:1:2017, destination = queue://QUEUE2, transactionId = null, messageCount = 1, poisonCause = null}
2014-07-16 00:34:14,397 ActiveMQ BrokerService[broker_jdbc] Task-1 TRACE || org.apache.activemq.broker.region.PrefetchSubscription - ID:LAPTOP-50558-1405488840024-5:1:1:1 dispatched: ID:LAPTOP-50534-1405488592270-3:1:1:1:16 - queue://QUEUE2, dispatched: 1387, inflight: 999
2014-07-16 00:34:14,397 ActiveMQ BrokerService[broker_jdbc] Task-1 TRACE || org.apache.activemq.broker.region.Queue - assigned ID:LAPTOP-50534-1405488592270-3:1:1:1:16 to consumer ID:LAPTOP-50558-1405488840024-5:1:1:1
2014-07-16 00:34:14,397 ActiveMQ Transport: tcp://laptop/127.0.0.1:50559@50562 DEBUG || jmstest.JmsTest2 - consumer recv: ActiveMQTextMessage {commandId = 20, responseRequired = true, messageId = ID:LAPTOP-50534-1405488592270-3:1:1:1:16, originalDestination = null, originalTransactionId = null, producerId = ID:LAPTOP-50534-1405488592270-3:1:1:1, destination = queue://QUEUE2, transactionId = null, expiration = 0, timestamp = 1405488593001, arrival = 0, brokerInTime = 1405488593001, brokerOutTime = 1405488854397, correlationId = null, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@1bca52f3, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = null, readOnlyProperties = true, readOnlyBody = true, droppable = false, jmsXGroupFirstForConsumer = false, text = text}
2014-07-16 00:34:14,397 ActiveMQ BrokerService[broker_jdbc] Task-1 TRACE || org.apache.activemq.broker.region.PrefetchSubscription - ID:LAPTOP-50558-1405488840024-5:1:1:1 dispatched: ID:LAPTOP-50534-1405488592270-3:1:1:1:17 - queue://QUEUE2, dispatched: 1388, inflight: 1000

Note that 250ms+ delay in the middle. 注意中间有250ms +的延迟。

The questions is - why are there such delays and how to get rid of them? 问题是-为什么会有这样的延误，以及如何消除它们？ Also - why the times are so low at the beginning (below 100ms) and then they raise to ~250ms? 另外-为什么时间一开始这么短（低于100毫秒），然后又增加到250毫秒？ (changing prefetch size moves the point when the performance degrades) （更改预取大小会在性能降低时移动点）

I also noticed that changing acknowledge mode to CLIENT_ACKNOWLEDGE and not acknowledging messages makes consumer super-fast (20ms/100), but chokes it few times for 20+ seconds. 我还注意到，将确认模式更改为CLIENT_ACKNOWLEDGE而不确认消息将使消费者超快（20ms / 100），但会在20秒钟以上将其阻塞数次。

1 个解决方案

解决方案1
 1  2014-07-16 06:34:02

I had faced similar issue sometime back. 有时我也遇到过类似的问题。 After investigations I figured out we can tune Memory Limit and Prefecth Limit parameters to improve performance for such cases. 经过调查，我发现我们可以调整Memory Limit和Prefecth Limit参数以提高此类情况的性能。

By default, memoryLimit is set to 1MB in the activemq.xml file. 默认情况下，activemq.xml文件中的memoryLimit设置为1MB。 This parameter indicates the amount of data that can be hold in memory by activemq (if this limit is reached, activemq needs to make IO calls to read data from persistent storage which impacts performance). 此参数指示activemq可以在内存中保留的数据量（如果达到此限制，activemq需要进行IO调用以从持久性存储中读取数据，这会影响性能）。

I suggest you to increase this value and see if it gives any performance gains. 我建议您增加此值，看看它是否可以带来任何性能提升。

Prefetch limit is other parameter which can be tuned to achieve performance gain. 预取限制是可以调整以获得性能增益的其他参数。 The value for this parameter varies depending on queue configuration you have for activemq. 此参数的值根据您为activemq拥有的队列配置而有所不同。

Other option is to use FileBasedCursor . 另一种选择是使用FileBasedCursor 。

Refer this link and check if it helps. 请参考此链接并检查是否有帮助。 ActiveMQ stops sending messages to Queue Consumer in case of consumer not acknowledging messages 如果使用者未确认消息，ActiveMQ将停止向队列使用者发送消息

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 使用ActiveMQ Ajax客户端时出现长时间的延迟 - Lengthy delays when using ActiveMQ Ajax Client ActiveMQ 消息消费延迟大 - Large delays in message consumption of messages in ActiveMQ ActiveMQ & Java - 发布和消费之间的延迟 - ActiveMQ & Java - delays when between publish & consume jdbc性能 - jdbc performance ActiveMQ SSL性能 - ActiveMQ SSL performance 骆驼ActiveMQ性能调整 - Camel ActiveMQ Performance Tuning JMS（ActiveMQ）性能 - JMS (ActiveMQ) Performance 带有JDBC MasterSlave的ActiveMQ计划程序故障转移 - ActiveMQ Scheduler Failover with JDBC MasterSlave ActiveMQ在不同操作系统中的性能 - ActiveMQ performance in different Operating Systems ActiveMQ嵌入式与独立性能 - ActiveMQ embedded vs standalone performance
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM