How to export multiple select statements to different csv files in SQL Server query

Question

I have a long SQL server query, which runs on different tables, does some pre-processing and spits out different select table outputs. I want them to be stored in different csv files.

simplified eg.

-- some declare variables and other pre-processing,
-- then save the outputs to different CSV files
select * from Table1 -- save to Table1_Output.csv
select * from Table2 -- save to Table2_Output.csv

Now, I can use this to run SQLCmd and save the output for a single select statement to a single CSV. I need to be able to save the various outputs to different CSVs

The first option in the solution above is not viable as that requires manual intervention and cannot be used in a scripting environment.

Thanks

Answer 1

Personally, I'd just use PowerShell if you can't use SSIS:

$SQLServer = 'SQLSERVERNAME';
# If you have a non-default instance such as with SQL Express, use this format
# $SQLServer = 'SQLSERVERNAME\SQLINSTANCENAME';
$Database = 'MyDatabase';

$ConnectionString = "Data Source={0};Initial Catalog={1};Integrated Security=SSPI" -f $SQLServer, $Database;
$CommandText = @'
DECLARE @ID int;
SET @ID  = 123456;
SELECT ID, First_Name FROM dbo.Person WHERE ID = @ID;
SELECT ID, Last_Name FROM dbo.Person WHERE ID = @ID;
'@;

$SqlConnection = New-Object System.Data.SqlClient.SqlConnection;
$SqlConnection.ConnectionString = $ConnectionString;

$SqlCommand = New-Object System.Data.SqlClient.SqlCommand;
$SqlCommand.CommandText = $CommandText;
$SqlCommand.Connection = $SqlConnection;

$SqlDataAdapter = New-Object System.Data.SqlClient.SqlDataAdapter;
$SqlDataAdapter.SelectCommand = $SqlCommand;

$DataSet = New-Object System.Data.DataSet;

$SqlConnection.Open();
$SqlDataAdapter.Fill($DataSet) | Out-Null;
$SqlConnection.Close();
$SqlConnection.Dispose();

$DataSet.Tables[0] | Export-Csv -NoTypeInformation -Path 'U:\FirstNames.csv';
$DataSet.Tables[1] | Export-Csv -NoTypeInformation -Path 'U:\LastNames.csv';

Answer 2

When multiple results are returned, sqlcmd prints a blank line between each result set in a batch. The output can then be processed with sed '/^$/q' or csplit -s - '/^$/' '{*}' to split into individual files: How to split file on first empty line in a portable way in shell .

UPD If sygwin is not an option custom VBScript can run sqlcmd through WScript.CreateObject("WScript.Shell") .Exec and process output line-by-line with .StdOut.ReadLine() until .StdOut.AtEndOfStream : Getting command line output in VBScript (without writing to files)

Answer 3

An alternative approach is to use SSIS . SQL Server Integration Services is great tool for moving data in, around and out of databases. It's also easy to schedule SSIS packages, using the SQL Agent, should you wish to automate.

This would require a little more effort up front. You would need to break your existing batch into several smaller SQL statements.