Run automatically generated scripts parallel in sql server

Question

I am creating a process that automates testing the consistency in database tables across servers. I have a test_master table which contains following columns:

test_id, test_name, test_status

and Job_master table which contains following columns:

jid, test_id, job_name, job_type, job_path, job_status, 
server_ip, db, error_description, op_table,  test_table,
copy_status, check_status

There can be multiple jobs for a particular test. The jobs are logical jobs (and not sql agent jobs), they can be script, procedure or ssis package. So I have made an ssis package :

In Pre-execute, it takes up tests which aren't done yet.

Each Job runs and writes the name of live table into op_table field

In post-execute, the live tables are getting copied to a test database environment and table name is put into test_table.. and testing will be performed there only.

Here the jobs are running in a loop... Is there a way to let the jobs run in parallel because they are independent of each other.... Can I write an sql procedure for this inside of this loop or is there any other way I can do this.. Any new ideas are welcome... Thank you very much.. :)

Answer 1

Very roughly , I would put the approach as below:

• SQL bits

Wrap whatever SQL code is part of "job" into a stored proc. Inside this proc, populate a variable which takes care of the SQL bit and execute it using dynamic SQL. Update the job status in the same proc and take help of TRY-CATCH-THROW construct.

• Packages

Populate the name of packages in an SSIS string variable in delimited fashion(or have an object variable, whatever suits you). Then, in a script task, iterate through the list of packages and fire them using dtExec command. To update the job status, it's best to have the update of job status taken care by the invoked packages. If that is not an option, use Try-catch construct, update the job statuses according. This is a helpful link.

Do a check on the job_type variable on top of the SSIS package(using precedence constraint) and route them into the correct 'block'.