Define Database connection for two different datasets using variables in Pentaho

Question

I have a requirement of connecting to two different datasets using a variable which compares these datasets. I'm using two different table input steps where the database connection names, hostnames are hard coded. Instead of using hardcoded I want to use a variable which defines these connections and should be able to connect to them

Answer 1

You can define variables in the kettle.properties file, located in the .kettle directory. Then you can use these variables in your database connection settings.

You can also define variables in your own .properties files and read them in using the Set Variables job entry.

set the variables like:

db_name.host=localhost
db_name.db=databasename
db_name.user=username

Then access those variables in your job/transformation by using the format ${db_name.host} etc.

Answer 2

Use JNDI to set up all your connection parameters:

(1) edit the file: data-integration/simple-jndi/jdbc.properties , and add your DB connection strings, for example:

db1/type=javax.sql.DataSource
db1/driver=com.mysql.jdbc.Driver
db1/url=jdbc:mysql://127.0.0.1:3305/mydb
db1/user=user1
db1/password=password1 
db2/type=javax.sql.DataSource
db2/driver=com.mysql.jdbc.Driver
db2/url=jdbc:mysql://mydbserver:3306/mydb
db2/user=user2
db2/password=password2 

Here we created two JNDI name db1 and db2 which we can use in PDI jobs/transformations.

(2) In your PDI job/transformation, add a parameter, ie mydb , through the menu 'Edit' -> Settings... -> Parameters Tab. You can add more such DB parameters if more than one is used.

(3) In the Table Input step, click Edit... or New.. button and in the coming dialog, switch the item in Access: box to JNDI and then add the ${mydb} in the JNDI name at the upper-right corner. you can also use plain text db1 and db2 which we defined in (1) to identify the DB connection.

Using JNDI to manage DB connections, we were able to switch between staging and production DBs simply by using the parameters. You can do the similar to PRD .