SQL Server CSV Exports using SQL Server Agent & Powershell?

Question

I found this answer using xp_cmdshell

SQL Server Agent Job - Export results to Tab Delimited File?

That would work fine for me except I need to provide a date parameter to a sql query. The date param gets populated via another sql query. Was thinking about using powershell.

Is there someone who has worked with this scenario and is willing to share scripts + experiences?

I could write console app in C# (or a class that is invoked via power shell once I work out how that is done). Just after the simplest, most clutter free way forward.

btw: my last use of xp_cmdshell was mired by permissioning frustrations. Last time I had to teach a sys admin how to do it via email + also troubleshoot their lockdown setup via email in a different timezone. As a result I've got some inbuilt hesitancy ;-). Thankfully I've got free reign in this situation though.

Answer 1

I would suggest before worrying about SQL Agent to first solve the problem of how to generate the CSV. You can then schedule the Powershell script in SQL Agent. Here's some code I tested to generate a CSV file from a SQL query using date param from another query:

$orderDate = Invoke-Sqlcmd -ServerInstance "Win7boot\sql1" -Database AdventureWorks -Query "select max(OrderDate) As OrderDate from Sales.SalesOrderHeader" | %{$_.OrderDate} 
Invoke-Sqlcmd -ServerInstance "Win7boot\sql1" -Database AdventureWorks -Query "select * from Sales.SalesOrderHeader where OrderDate < '$orderDate'" | Export-Csv -Path "C:\Users\Public\salesorder.csv" -NoTypeInformation -Force

This should work in SQL Agent just as it does in the Powershell console. You'll need to setup a Powershell job step.

Note if the date param is being obtained from the same server instance as the main query you could combine the query into one:

"select * from Sales.SalesOrderHeader where OrderDate < (select max(OrderDate) from Sales.SalesOrderHeader)"

Answer 2

Can you try this?:

If you want it as a stored procedure where a date can be manually or programmatically passed:

CREATE PROCEDURE bcp_test_with_date_parameter
    @paramDate DATETIME
AS
BEGIN

    DECLARE @bcpCmd varchar(2000)
    SET @bcpCmd = 'bcp "SELECT * FROM dbname.schema.tablename' 
    SET @bcpCmd = @bcpCmd + 'WHERE CONVERT(VARCHAR, datecolumn, 101) = ''' + CONVERT(VARCHAR, @paramDate, 101) + ''' '
    SET @bcpCmd = @bcpCmd + ' ORDER BY columnnameofchoice" queryout "'
    SET @bcpCmd = @bcpCmd + '"C:\tblNameData.txt" -T -c'

    EXEC master..xp_cmdshell @bcpCmd

END

But if you just want to grab the date from another table you could use this:

DECLARE @bcpCmd varchar(2000)
DECLARE @dateHolder DATETIME
SELECT @dateHolder = CONVERT(VARCHAR, datecolumn, 101) 
    FROM dateparametersourcetable
    WHERE <place conditions here>

SET @bcpCmd = 'bcp "SELECT * FROM dbname.schema.tablename' 
SET @bcpCmd = @bcpCmd + 'WHERE CONVERT(VARCHAR, datecolumn, 101) = ''' + @dateHolder + ''' '
SET @bcpCmd = @bcpCmd + ' ORDER BY columnnameofchoice" queryout "'
SET @bcpCmd = @bcpCmd + '"C:\output.txt" -T -c'

EXEC master..xp_cmdshell @bcpCmd

The one right above can also be placed in a stored procedure, it's really up to how you want to do it..