Missing plugins found while loading a transformation on Kettle

Question

I receive this error whenever I run my extraction from the command line, not in the Spoon UI.

Missing plugins found while loading a transformation

Step : MongoDbInput

        at org.pentaho.di.job.entries.trans.JobEntryTrans.getTransMeta(JobEntryTrans.java:1200)
        at org.pentaho.di.job.entries.trans.JobEntryTrans.execute(JobEntryTrans.java:643)
        at org.pentaho.di.job.Job.execute(Job.java:714)
        at org.pentaho.di.job.Job.execute(Job.java:856)
        ... 4 more
Caused by: org.pentaho.di.core.exception.KettleMissingPluginsException:
Missing plugins found while loading a transformation

My maven dependencies is as follows.

    <dependency>
      <groupId>rhino</groupId>
      <artifactId>js</artifactId>
    </dependency>
    <dependency>
      <groupId>pentaho-kettle</groupId>
      <artifactId>kettle-core</artifactId>
    </dependency>
    <dependency>
      <groupId>pentaho-kettle</groupId>
      <artifactId>kettle-engine</artifactId>
    </dependency>
    <dependency>
      <groupId>pentaho-library</groupId>
      <artifactId>libbase</artifactId>
      <version>5.1.0.0-752</version>
    </dependency>
    <dependency>
      <groupId>pentaho</groupId>
      <artifactId>pentaho-big-data-plugin</artifactId>
      <version>5.1.0.0-751</version>
    </dependency>
    <dependency>
      <groupId>pentaho</groupId>
      <artifactId>pentaho-mongodb-plugin</artifactId>
      <version>5.1.0.0-751</version>
    </dependency>
    <dependency>
      <groupId>org.mongodb</groupId>
      <artifactId>mongo-java-driver</artifactId>
      <version>2.11.1</version>
    </dependency>
    <dependency>
      <groupId>pentaho</groupId>
      <artifactId>metastore</artifactId>
      <version>5.1.0.0-751</version>
    </dependency>
    <dependency>
      <groupId>pentaho-library</groupId>
      <artifactId>libformula</artifactId>
    </dependency>
    <dependency>
      <groupId>simple-jndi</groupId>
      <artifactId>simple-jndi</artifactId>
    </dependency>
    <dependency>
      <groupId>org.drools</groupId>
      <artifactId>drools-compiler</artifactId>
    </dependency>
    <dependency>
      <groupId>org.drools</groupId>
      <artifactId>drools-core</artifactId>
    </dependency>
    <dependency>
      <groupId>org.postgresql</groupId>
      <artifactId>postgresql</artifactId>
    </dependency>
  </dependencies>

Am I missing anything?

Answer 1

before init kettleEnviroment

KettleEnviroment.init();

you should add this code like

StepPluginType.getInstance().getPluginFolders().add(new PluginFolder("your plugins path", false, true));

Answer 2

Having the MongoDB plugin as a dependency only works for compilation. When PDI runs it looks for a plugins/ folder under the current directory, and will load all plugins from there. You can override the location by setting the KETTLE_PLUGIN_BASE_FOLDERS system property. Either way, you'll want a plugins/ folder and to unzip the MongoDB plugin package (not the JAR) into plugins/. That should put a folder called pentaho-mongodb-plugin under plugins/, and if PDI is pointing at that plugins/ folder, it should load and use the plugin successfully at runtime.

Answer 3

You must run Kitchen from Kettle installation directory.
As written here: http://wiki.pentaho.com/display/EAI/Kitchen+User+Documentation

Please make sure that you are positioned in the Kettle directory before running the samples below. If you put these scripts into a batch file or shell script, simply do a change directory to the installation directory

Answer 4

Just edit spoon.sh and add

OPT="$OPT -DKETTLE_PLUGIN_BASE_FOLDERS=$BASEDIR/plugins"

right after the line where the OPT variable is set.

This is actually the solution from mattyb. Editing spoon.sh will fix the problem for good.

Another solution is to create a symbolic link in your ${KETTLE_HOME}/.kettle directory which points to the plugins folder.

Answer 5

I had the same error message after an update from version 7 to 8.3. In my case, it was the step 'Abort' and 'Attend' that caused the error. In version 8 these two steps aren't part of the library "kettle-engine-8.3.0.0-371.jar" anymore, they are now part of the library "pdi-core-plugins-impl-8.3.0.0-371.jar" in a new plugin. I had to add the plugin in the maven-dependencies:

<dependency>
  <groupId>org.pentaho.di.plugins</groupId>
  <artifactId>pdi-core-plugins-impl</artifactId>
  <version>8.3.0.0-371</version>
</dependency>

But that was still not everything, I still got the error message. The reason: "To use non-native plugins with an embedded Pentaho Server, you must configure the server to find where the plugins reside" ( https://help.hitachivantara.com/Documentation/Pentaho/8.0/Developer_Center/PDI/Embed ). So above the line KettleEnvironment.init() in my java-code, I added this one:

System.setProperty("KETTLE_PLUGIN_CLASSES", "org.pentaho.di.trans.steps.abort.AbortMeta,org.pentaho.di.trans.steps.append.AppendMeta");
KettleEnvironment.init();

This way worked for me. There's also a suggestion using a plugin-folder (see the link or the other answers) and working with KETTLE_PLUGIN_BASE_FOLDER instead.

Unfortunately, the error message doesn't mention which plugin is missing. I deleted and added them (in my local Spoon application) until I found exactly the one that was missing.