Designing Orchestrator In Java using disruptor and Executors

Question

We are designing an Orchestrator System in java that will host a web service and on a request to it will invoke a flow written in XML which are nothing but steps that are executed one after another but the XML tell the user what the flow is and he can also change the flow by changing the XML. He can add a new service and add it to the XML. But while designing I am now confused with things like.

1. Should I make a service a runnable with a blocking queue and keep it alive all the time by scheduling it to the executor so when the new request arrives I will push the request in the blocking queue and the service will process it. And create a Mailbox which will carry the message passing task between different services.

2. Instead of Making service runnable I should use spring IOC that will make the class singleton thus only one instance will be there and I will send a request directly to the service methods thus there will be no hassle that I have to do as there are no threads and also didn't need to implement any mailbox.

3. I read about how event driven system is faster like nodejs and ngnix so I was thinking to use disuptor framework to push all the incoming request to the ringbuffer and then write a handler that will emit the event to the orchestrator that will start processing the request and also send a callback with the request so that when the orchestrator is done it will send back the response back to the caller using callback. But as the request is not of the same type it so I would not be able to take advantage of disruptor object allocations.

The system needs to provide maximum throughput with low latency, the system will add new services/flows to XML in future so it should adopt the new flows without hitting the performance. I am allowed to only use java 7, no Scala so I am bounded.

Answer 1

Answer #1 is a terrible idea. You will effectively tie up a thread per service. If the number of services exceeds the number of threads backing the executor service you have an instant, automatic DOS. Plus, if the services are inter-dependent on each other... all the ways in which you can dead lock. Plus, the inefficiency of using N threads if only M (< N) are actually required.

Answer #2: if the proposed flow is Request Parsing -> Dispatch -> Service Processing -> Callbacks you rely on the actual 'services' not to foul up because that will prevent callbacks from being called and/or DOS the application. Essentially: what happens if an exception occurs in a service? Will that also impact future requests to the same service and/or other services?

Also the opportunities for parallelism are limited to the framework's way of handling incoming requests. Meaning if you have X requests and the framework inherently processes them serially, you get a backlog of X requests. Your latency requirements may be hard to meet in such a scenario.

Answer #3: an event driven system is indeed the better approach: have a scheduler farm out jobs to an executor service to allow the system to distribute the total load of all services dynamically and have a mechanism to generate and react on events to handle the control flow. This scales better if the number of services become very large and each 'job' is reasonably substantial (so the overhead of scheduling/dispatch is low compared to the actual work being performed).