Put message async on WebSphere MQ queue

Question

I am trying to put persistent messages onto a WebSphere MQ queue, however these need to be async puts. The only way that I seem to be able to get async working is if the messages are non persistent (by this I mean that the putSuccessCount equals number of messages put on the MQAsyncStatus , all other times it is zero). The code below outlines what I am trying to do:

import com.ibm.mq.MQAsyncStatus;
import com.ibm.mq.MQDestination;
import com.ibm.mq.MQMessage;
import com.ibm.mq.MQPutMessageOptions;
import com.ibm.mq.MQQueueManager;
import com.ibm.mq.constants.MQConstants;

public class MQPutTest extends TestCase{

    private static Logger log = Logger.getLogger(MQPutTest.class);

    public void testPut() throws Exception{

        Hashtable<String, Object> props = new Hashtable<String, Object>();
        props.put(MQConstants.CHANNEL_PROPERTY, "my_channel");
        props.put(MQConstants.PORT_PROPERTY, 1414);
        props.put(MQConstants.HOST_NAME_PROPERTY, "localhost");

        String qManager = "my_queue_manager"; 
        MQQueueManager qMgr = new MQQueueManager(qManager, props);

        int openOptions = MQConstants.MQOO_OUTPUT | MQConstants.MQOO_INPUT_AS_Q_DEF;

        MQDestination queue = qMgr.accessQueue("my_queue", openOptions);

        MQPutMessageOptions pmo = new MQPutMessageOptions();
        pmo.options = MQConstants.MQPMO_ASYNC_RESPONSE;

        MQMessage message = new MQMessage();
        message.format = MQConstants.MQFMT_STRING;
        message.writeString("test message");
        queue.put(message, pmo);

        queue.close();
        MQAsyncStatus asyncStatus = qMgr.getAsyncStatus();
        qMgr.disconnect();
    }
}

I am attributing the performance increase I see with a large number of messages to the fact that the queue is set to non persistent rather than the messages being put async. I have set the default put response type to Asynchronous in the queue extended properties in MQ explorer bu this has no effect.

Any help would be appreciated.

Answer 1

If its ok to lose one or two messages you can add the messages to an internal send queue eg

ExecutorService service = Executors.newSingleThreadedExecutor();

// build you message here

// add the message to be sent asynchronously.
service.execute(new Runnable() {
    public void run() {
         queue.put(message, pmo);
    }
});

Any messages in the queue when the process dies is lost.

Answer 2

Following up from the comments in the question, WMQ doesn't have a notion of a "persistent queue" (hence the reason I changed the title of the post). The queue attribute for persistence does not in any way change how the QMgr treats the queue or any messages in it. All that it does is to tell a program which persistence option to use if the programmer fails to explicitly set that value. Whether any given message is persistent depends on whether the MQMD specified persistence when the message was first put to the queue. Any queue can contain persistent and non-persistent messages in any combination.

Specifically, the code snippet int he post does not specify persistence using message.setPersistence() so it will inherit whatever the queue's default value is. That in turn depends on the value inherited from the queue attribute. In no case does the queue attribute setting override an explicit setting from the app .

So the performance difference that you are seeing is indeed most likely to reflect the setting of the queue's DEFPSIST attribute, but that does not mean that async puts are not working. Remember that async puts do not in any way reduce the amount of time that it takes to put a message on the queue. The QMgr has the same code path to persist a message whether the put is synchronous or not. The difference is that your app no longer waits for the QMgr to finish its work. It is entirely possible that when you call for an update of MQAsyncStatus that WMQ has not written any messages yet. This is even more likely if they are all in a single unit of work because WMQ will not return the status for all the messages until after the COMMIT processing is complete. Unless you call COMMIT explicitly, this happens when you close the queue.

You can verify this by repeating the qMgr.getAsyncStatus() call every second for a few seconds after the COMMIT or the CLOSE . You should see that the first one returns no messages successfully put but that eventually you can account for all of them.

Incidentally, the question of whether the messages are persistent should almost always be handled in your code . Message persistence is usually derived as a business requirement that is captured in the design of the application. It is therefore the responsibility of the project manager and developer to insure that the requirement is met. The only way to reliably make sure it is met is that the application calls setPErsistence() explicitly. Otherwise, the application will use the value from the queue's DEFPSIST attribute to implicitly call setPersistence() . So why bother to allow this kind of default in the API? Because there are a few legitimate cases where persistence needs to be able to change at run time . Writing the app to intentionally take the default value and reopen the queue after each unit of work accomplishes this. In all other cases, persistence should be set in the program or in the managed objects.

Finally, if you are putting 10,000 messages under a single unit of work one other reason that you can get a response stating that no messages were successfully put is lack of log space or tunables not set to accommodate the load. For example if MAXUMSGS is set to something less than 10,000 your entire transaction will roll back. On the other hand, if MAXUMSGS is set correctly but the size and number of primary and secondary logs is insufficient to hold the amount of data in a transaction, then again the entire transaction will roll back. You should play around a bit with the COMMIT interval because the optimal value depends on the size of the messages in relation to the size of the queue and log buffers. Once you exceed the amount of data that can be optimized into a single write operation, additional messages actually decrease performance. When people put 10,000 messages into a single unit of work it is hardly ever for performance but rather because this is their appropriate unit of recovery and the corresponding performance hit is secondary to the recovery requirement.