Netflix Conductor as a workflow engine solution

Question

Can someone share a tutorial on how to use Netflix Conductor? Is there like a code example on github or something? I've done a sample on Amazon SWF and understand how the whole thing works but need some pointers on getting started with Conductor.

Answer 1

The initial go-to resource is the official documentation of Netflix conductor. The other parts that might help anyone to come up to speed is the actual working of the conductor. Conductor server mainly manages the lifecycle of the tasks.

Assume that the conductor server is running using default configuration. To execute a workflow, you will need to

1. Create task definitions,
2. Create workflow definition that uses the task definitions in step 1,
3. Execute the workflow.

As part of the execution, conductor server schedules the next task in the workflow. It is the first task in the workflow if you have just executed. Conductor uses dyno-queues internally where the tasks are available for consumption by the clients (A client can be an internal task or can be an external worker application that interacts with conductor through REST API).

The main doubt I had initially was about how and where worker runs? The worker can run in the moon provided your worker can access the REST API exposed by the conductor and be able to update the status of the task back. ie you can run microservices anywhere it is needed.

A simple usecase could be the on-boarding employment verification. To verify the employee before on-boarding you may check criminal background, drug, education verification etc., If you are running conductor server on your premises and have workers placed in institutions that are responsible for above said areas, the workers would perform the checks and update the workflow accordingly.

Conductor is integrated with explicit components like elastic search, dynomite backed by redis for the data stores. You can have these provisioned over HA cluster.

Using contribs modules you can integrate external MQ brokers for the tasks and have your workflow updated with respective sink details so that the events are published to the configured sink.

You can also have other workflows that you can execute as sub workflow of the main workflow through event handlers in a loosely coupled manner. In the above said usecase this subworkflow could be the workflow for education verification, or a subworkflow for criminal background check etc.

Due to the stateless nature of the conductor, you can run multiple instances of the conductor through containerization.

Answer 2

Easiest way in my opinion is using docker.

Clone this repository: https://github.com/Netflix/conductor

Go to, conductor/docker and run, docker-compose up

Now, open your browser on http://localhost:5000

On the left menu you will see the workflow def item, where you can see a workflow example.

To deploy your own workflow, copy the json from this site, change the name and deploy it using this software Postman.com.

Do a post to this endpoint to deploy: http://localhost:8080/api/metadata/workflow

Do a post to this endpoint to run your workflow: http://localhost:8080/api/workflow/

Answer 3

There are a number of sample applications and codelab walkthroughs for Netflix Conductor in the following repository: https://github.com/conductor-sdk/conductor-examples

There is also a free online playground to build/test and run your workflows at https://play.orkes.io

Answer 4

I have done small POC on Netflix conductor, you can get detailed description under the below link. Also click on code in below github link to get the code and execute the same. https://github.com/manishkushwaha1412/conductor-poc/wiki