Pentaho Spoon: load SQL code from txt file and run it

Question

I'm using Execute SQL Script component to send SQL code to MSSQL to be executed.

But I'd rather save SQL code in txt files, so that Subversion can control their changes.

These files will be stored in a sql present on the same level of ktr file. I'd like some component equivament to Execute SQL Script to read a .sql file, take its content and send it to MSSQL. It would be awesome if I could provide the connection string on the sql file, or at least set it as a ktr parameter.

In the least possibility, I'd need a component to read the sql file, store it in a string variable, and pass it to another component that would take the variable and send it.

It'd also be great to be able to replace variables on the SQL code.

Sorry for being so noob, I'm trying to learn Spoon and missing features I used to have.

Answer 1

You can adapt the following proposal:

1. The first step reads the files.

   • On the file tab put the file directory as ${Internal.Transformation.Filename.Directory} (Don't type, press Ctrl-space) and on the Wildcard put .*\\.sql to get all the files ending with ".sql". Use the Show filenames button and adapt the regex and the next tab to have all the files you need.

2. The second step execute the script.

   • Use the dropdown menu to tell kettle to use the filename in the SQL field

   • And check the box to tell it is a file name.

It is not possible to set the connection dynamically. This is because kettle uses prepared JDBC statements to be able to manage parallel threads. If you do not have too many connections, you can use a Switch/Case step, which sends to copies of the Execute row SQL script with the appropriate connection.

Answer 2

I believe you want the Load file content in memory Step. Your field name and element will be "File content". From there you can "Set Variables" or feed the field "File Content" (your SQL) directly to a Table Input Step.