Create a Pentaho Transformation & Steps using Java

Question

I am very new to Pentaho/ETL and need expert advise to complete my assignment. Actually, we are trying to integrate Pentaho Steps/Jobs with our web based platform. We are designing a similar interface where user will get the list of steps and can pick anystep to create a transformation. We are developing a similar looking HTML interfaces to all pentaho steps dialog, We will send the transformation data (including steps / hops) in JSON format to backend API's. Backend API will parse the JSON and create the steps / transMeta for the Kettle core engine. Can anyone please help me on how to create Steps/Transformation in Java ?, You can also suggest a good design strategy to implement this scenario. Your help is much appreciated.

Answer 1

Open any ktr, kjb file in text editor you will find that it is just xml document. Study how each step is represented in xml document, how it holds variable, fields, state actually, how steps are connected to each other.

When u will be able to build xml document that correspond to what u see in browser.

Generate transformation, or job in xml format and pass it into engine using API in kettle documentation.

I think it is gonna take time. I don't think it worth to translate all steps, but at least u can implement some small subset of necessary steps.

Answer 2

I used a factory patter to write my json parser which parse the request json and create pentaho steps & hops and added to a transformation. (Using PDI Java API's).

I am running transformation as a asynch job and maintaining a cache to save the transformation log output.

I would be happy to help if anyone is looking for detailed explanation.