Drools KieSession causes memory leak when inserting object to Timer based rule session

Question

We want to use Drools to do a delay job. So, I use timer(int:5s) to do that. However, after I inserted an fact object to the KieSession, the KieSession will start a separated "Thread-XX" to dealt with this delay job, but this thread will never end although the rule has been executed after 5s elapsed.

As a result, if I inserted a large number fact object, for example 100000 objects, there will be 100000 threads in this KieSession, the memory will raise to about 500MB and it will never drop down. The more inserts, the more memory consumed.

I think Drools should close those threads if the object has been evaluated and executed the rule since they're useless.

Is there some option to make those timer thread shutdown after executed? I know drools.halt() in drl file, however, it will halt the whole session rather than each thread in this KieSession.

Here are my code:

1.main class

        KieServices ks = KieServices.Factory.get();
        final KieContainer kContainer = ks.getKieClasspathContainer();
        final KieSessionConfiguration ksconf = ks.newKieSessionConfiguration();
        ksconf.setOption(TimedRuleExectionOption.YES);
        final KieSession kSession = kContainer.newKieSession("ksession-rules", ksconf);

        for (int i = 0; i < 100000; i++) {
            Thread t1 = new Thread(new Runnable() {
                public void run() {
                    Message message = new Message();
                    message.setStatus(Message.HELLO);
                    kSession.insert(message);
                    kSession.fireUntilHalt();
                }
            });

            t1.start();
        }

2. drl file

     dialect "mvel" rule "Hello World" timer(int:10s) when m : Message(m.status == Message.HELLO) then System.out.println( "no hello"+m.message ); end 

The code above is to share a KieSession. I also try to use one KieSession per fact and then halt and dispose it. However, 100000 KieSession reside in memory at same time will cause 6.7GB memory usage. It's also not acceptable.

So, anyone can help me?

Actually, my business logic is to check if the control point and feedback point are the same within 5mins, otherwise sound an alarm. And there will be about 100000 request at same time. So, is there some other better way to achieve this case by Drools?

Thanks,

Answer 1

The problem in the code above would call kSession.fireUntilHalt(); a 100000 times, this is not following the documentation guideline, not a best practice and would seems semantically incorrect if I follow the intention of your code.

If you want to use Active mode (fireUntilHalt) you call kSession.fireUntilHalt(); in a separate thread, once, at the beginning; and then, you insert the facts.

Otherwise, and accordingly to your code without necessarily the need for another thread, you could use Passive Mode by using the fireAllRules() , but the architectural choice depends on the application requirements.

A comprehensive explanation is detailed in the Drools documentation "Rule Execution Modes" which I also linked above for what concerning Active mode (fireUntilHalt)

I assume Drools version 6.x based on API usage but I might be wrong.

Edit following comments that " The unfinished thread " are not the one created by the user code and that " I also called kSession.retract(item) after sleep 1min. But the memory is still there, no drop down. "

I was really interested to follow the OP's comment, but frankly even with further investigation is not possible to highlight such claims. Using properly the API guidelines there is no generation of thousands of threads:

As can be seen number of thread does not show anything relevant - please notice for running the tests some threads are also required by JUnit runner of course.

....and following " I also called kSession.retract(item) after sleep 1min. But the memory is still there, no drop down. " memory do INDEED drops down:

Of course memory occupation while asserting a million fact depends into a stateful session relates to a number of parameters, but even following OP's assertion memory is not showing anything revelant. So I'm not able to reproduce any relevant memory leak.

If you still think there is an issue or a memory leak kindly raise a JIRA issue ticket with reproducer highlighting the problem, so it could be properly investigated, thank you.

Answer 2

I think that you should not use Drools for scheduling a large number of TimerTasks. You can set up a Timer as a global and call its schedule method as required.

It should be evident that the simple timer attribute cannot cope with the various scenarios that are possible with the API of java.util.Timer and TimerTask. Its primary purpose is to cover simple scenarios for executing something repeatedly as long as a condition prevails. Also, the timer attribute is there so Drools marketing can claim that "you can do it all in Drools, no need to know (much) Java". Bah.