ActiveMQ Consumer OutOfMemoryException

Question

Our ActiveMQ consuming process runs out of memory and dies.

We have an ActiveMQ topic with one sender and two receivers. Superficially, all works fine---messages are sent and picked up by both receivers, but eventually we exhaust all memory. Heap dump shows 1.362 million instances each of LinkedList$Node, AtomicReference, ActiveMQObjectMessage, MessageId, and MessageDispatch. Meanwhile the client side message queue is empty or nearly empty throughout. I think the 1.362M may be on a list that tracks unacknowledged messages pending ack. The topic is specified to have AUTO_ACKNOWLEDGE set, so we're trying to ack, but possibly failing. (jmsSession = jmsConnection.createSession(false, Session.AUTO_ACKNOWLEDGE);)

The heap dump shows garbage that appears to be associated with the client's incoming message buffering exists in modest numbers (a few thousand). This seems consistent with the numbers of these objects that accumulate in a toy program that we set up to send and consume similar objects. They accumulate for a while, then get GC'd and the memory never grows significantly in the toy or in the failing program.

Is it correct to surmise that the five object types are associated with ACKS, and if so, what can cause the objects to remain on this structure despite being apparently fully consumed by both consumers? Is there some way we could be canceling the AUTO_ACKNOWLEDGE that we think we have set? BTW, one consumer is synchronous, using receive() and the other is asynchronous, using onMessage().

One possibly misleading symptom is that the ActiveMQ GUI shows the objects only being dequeued once, despite the presence of two Consumers. The toy shows two dequeues for each enqueue. However, the program itself says they are read the expected number of times.

// creating the async consumer.

connAmq = createActiveMqConnection();
connAmq.start();
session = connAmq.createSession(true, Session.AUTO_ACKNOWLEDGE);
Destination topic =        session.createTopic(appProperties.getActiveMqTopicQuotesName());
MessageConsumer consumer = session.createConsumer(topic); 
consumer.setMessageListener(this);

public void onMessage(Message message) {
    ...     
    try {   
            if (message instanceof ObjectMessage) {
                    ObjectMessage msg = (ObjectMessage)message;
                    if (msg instanceof Foo) {
                            Foo quote = (Foo)msg.getObject();
                            ...
                    }       
            }
    }
    ...
}

// creating the sync consumer

jmsConnection = mActiveMQConnectionFactory.createTopicConnection();

jmsConnection.start();
jmsSession = jmsConnection.createSession(false, Session.AUTO_ACKNOWLEDGE);
jmsDestination = jmsSession.createTopic(name);
jmsMessageConsumer = jmsSession.createConsumer(jmsDestination);


//the code for consuming looks like this for the synchronous consumer
while(true)
ObjectMessage m = (ObjectMessage) jmsMessageConsumer.receive();
if (m != null)
    Process(m.getObject());
}

Answer 1

At least in the code snippet given you have created a session that is transacted for the async consumer but I see no commit call on the session. The transaction bits stay in memory on the broker and you will eventually exhaust the Broker's memory.