Using SSIS to Export Multiple Text Files in SQL Server

Question

I run a monthly process which requires that I take a table with 4-5 million records and divide it up into .txt files of 240,000 records each. I use this code in SQL Server 2012 to create the paging of the file and then use a SSIS package to run this stored procedure and create the .txt file. I change the @PageNumber to 2, recompile the stored procedure, run the SSIS package. Increment the @PageNumber to 3 and so on until I run out of records which at 4 million records takes around 20x.

I was wondering if anyone knows a way to loop through the code and create the multiple .txt files in one pass without having to change the @PageNumber value 20x?

CREATE PROCEDURE [dbo].[p_ExportAllDMCSBorrowersPaging] 
AS
BEGIN
DECLARE @PageNumber AS INT, @RowspPage AS INT
SET @PageNumber = 1
SET @RowspPage = 240000

SELECT  [SSN]
        ,[DOB]
        ,[LastName]
        ,[FirstName]
        ,[CustomerRecordID]
        ,[ADDate]
        ,[MiddleName] FROM (
         SELECT ROW_NUMBER() OVER(ORDER BY ID) AS NUMBER,
                 [SSN]
                ,[DOB]
                ,[LastName]
                ,[FirstName]
                ,[CustomerRecordID]
                ,[ADDate]
                ,[MiddleName]
        FROM [dbo].[All Borrowers 20160919]
               ) AS TBL
        WHERE 
            NUMBER BETWEEN ((@PageNumber - 1) * @RowspPage + 1) AND (@PageNumber * @RowspPage)
        ORDER BY 
            NUMBER

Answer 1

I would use a For Loop for this.

Use package variables to hold the number of rows in the table and the current row counter.

First put a script in the loop that changes the connection string of your flat file destination to whatever you want to name it for that iteration (eg: Page1, Page2, etc)

Then put a dataflow in the For Loop that uses the counter variable to get 240k rows starting at the value of the variable.

After each iteration of the loop, increment the counter by 240k, until it exceeds the number of rows in the table, then exit the loop.