Running SQL scripts in folder

Question

I have a scripts folder and I want to run SQL scripts against my database. Is there a way I can iterate through all the SQL files and check if the SQL file is not executed by checking a table in my database.

Check if the script was executed, if no then it execute else just skips it. After running the SQL script, it should insert a entry in table just for keeping track that script was executed.

I am thinking of writing a ruby or python script to achieve this. Any thoughts ?

Answer 1

Before the answer starts: Your problem looks like a migration, eg the migrations from Sequel or Active Record

Please take first a look for this migrations before you try my answer.

---

With Sequel you can try something like this:

def call_sql_scripts(mask)
  DB.create_table(:executed){
    add_column :filename
  } unless DB.table_exists?(:executed)

  Dir[mask].each{|filename|
    if DB[:executed].filter(:filename => filename).first
      puts "Script %s was already executed" % filename
    else
      puts "Start %s" % filename
      begin 
        DB.run(File.read(filename))
        DB[:executed].insert(:filename => filename) #executed if run was successful
      rescue Sequel::DatabaseError
        puts "%s failed" % filename
      end
    end

  }
end

call_sql_scripts('*.sql')

The table :executed stores the already executed script files.

Attention:

• If you rename or move the files, they are not detected as changes.
• There is no defined sequence (if you need this, you must take a look for migrations.)
• I would prefer to use Sequel-commands to build the database and not loading SQL-scripts in Files.

---

A full minimal working example:

File.open('1.sql', 'w'){|f| f << 'create table tab1 (`field` add_column)'}
File.open('2.sql', 'w'){|f| f << 'create table tab2  (`field` add_column)'}
File.open('3.sql', 'w'){|f| f << 'create table tab3  (`field` add_column)'}

require 'sequel'
DB = Sequel.sqlite('test2.db')

def call_sql_scripts(mask)
  DB.create_table(:executed){
    add_column :filename
  } unless DB.table_exists?(:executed)

  Dir[mask].each{|filename|
    if DB[:executed].filter(:filename => filename).first
      puts "Script %s was already executed" % filename
    else
      puts "Start %s" % filename
      begin 
        DB.run(File.read(filename))
        DB[:executed].insert(:filename => filename) #executed if run was successful
      rescue Sequel::DatabaseError
        puts "%s failed" % filename
      end
    end

  }
end

call_sql_scripts('*.sql')

If you call it the 2nd time you see the scripts are not called again.