Fetching Result From Java API from Pentaho Kettle Job

Question

I have a Job in Pentaho. The job has many sub-jobs and many transformation. Most of the transformation writes to a table. I would like to get some stat information like below.

1. Table1 Finished processing (I=0, O=0, R=86400, W=86400, U=0, E=0)
2. Table2 Finished processing (I=0, O=0, R=86400, W=86400, U=0, E=0)
3. Table3 Finished processing (I=0, O=0, R=86400, W=86400, U=0, E=0)

My code is: With this code, I'm just getting the result of the last transformation. For Example, If i run 40 transformation, my result is just the 40th transformation result. But I would like to see all the 40 transformation result.

KettleEnvironment.init();
JobMeta jobMeta = new JobMeta("Job.kjb", null);
Job job = new Job(null, jobMeta);          
job.start();
job.waitUntilFinished() 
Result result = job.getResult();   
System.out.println("dfffdgfdg: "+result.getLogText());

Answer 1

Use the logging system . On each transformation of interest, right-click anywhere, select setting, then logging and setup the data you want to collect stat on (for example in front of the Output button select the step that writes the data on the table you want to monitor). I suggest you use the default to start with.

After that, press the SQL button, and Pentaho Data Integrator will create a table in a database, with the relevant columns. And each time you run the transformation (or anyone using the same repository) will put a row in the table. After that, just SELECT * FROM TRANSFORMATION_LOG.

In the last Pentaho Meetup , I explained why you should do that at transformation level, and at at job level (although you can automate this if you know how to navigate in a repository). You'll also have a pointer to a github with a JSP you an copy/paste in your Pentaho BA server's WEB_INF so that you get exactly what you are after in web server.

Do not hesitate to ask for more info or provide feedback.