How to pass dynamically created filters as parameter of stored procedure in SQL Server and filter data?

Question

How can I pass some dynamic filters (populated from my table tblMarketingCampaignFilters in SQL Server) and compare them to a TargetColumn in tblMarketingUsers ?

Just to clarify, I have all the code to generate and populate the filters with the InputTypes assigned (it can be numbers, dates, strings, or boolean with checkboxes, only), but the page is supposed to Add as much a user wants ( n filters).

Here is an image showing how my tables are defined

I must compare the filters , with columns (to be added) in tblMarketingUsers .

Examples:

• LastLoginDate > '20150701' if it is a Date,
• Advanced Profile Status > 50(percent) if it is a number,
• InBothSystems = 0 if it is a checkbox checked

This is my filter part of the form in the page

By now, I have only 4 filters, it could be 10, I don't know.

Thanks.

PS: I think that I am missing a TableTarget in tblMarketingCampaignFilters , if the system have to look on another table, not only tblMarketingUsers , but it doesn't matter so much.

PS2: I am considering passing the filters as text, with the comma-separated style, and then insert this into transMarketingCampaignFilters , but still don't know how to compare them dynamically, my boss says "use cursors in SQL"

EDIT

Thanks for the replies, and comments.

This is an approach of what I am thinking to do. I will need to add Operator column in tblMarketingCampaignFilters (it will store something like : ' = ' , ' < ', ' > ', ' LIKE ') , and btw, this stored procedure will assume that a Campaign with the filters has already submitted on the form, and all data is stored on transMarketingCampaignFilters

CREATE PROCEDURE usp_MarketingUsersGetFiltered
    -- Add the parameters for the stored procedure here
    @CampaignId int,
    @Debug BIT = 0
AS
BEGIN
    -- SET NOCOUNT ON added to prevent extra result sets from
    -- interfering with SELECT statements.
    SET NOCOUNT ON;
    declare @sql varchar(max) = '', @Nsql nvarchar(max), @DatabaseId INT;
    -- Insert statements for procedure here


set @DatabaseId = (SELECT DatabaseId from tblMarketingCampaigns where Id=@CampaignId);
select @sql +=  ' AND '+quotename(f.TargetColumn) + f.Operator + 
            case f.InputType 
            when 'int' then cast(cf.Value as int)
            else quotename(cf.Value,'''')               
            end
from dbo.transMarketingCampaignFilters as cf
join dbo.tblMarketingCampaignFilters as f
on cf.CampaignFilterId = f.Id
where cf.CampaignId = @CampaignId

 set @sql = '
 select MarketingUserId,
        FirstName,
        LastName,
        Email,
        CompanyId
 from dbo.tblMarketingUsers 
 where DatabaseId = @DatabaseId
 '+ @sql+'
 OPTION (RECOMPILE)';

if @Debug = 1 print @sql;
set @sql = @Nsql;
exec sys.sp_executesql @Nsql,N'@DatabaseId INT',@DatabaseId;

END
GO

Answer 1

1) Pass through the filter data as a user-defined type, or as an XML parameter, into the stored procedure.

2) Examine the content of the flexible parameter-set in the stored procedure and for the most common use-cases call subsidiary stored procedures with fixed SQL content which satisfy the most common cases. This allows those procedures to be optimal for their purpose.

3) Where the filter set does not match any high frequency predicted pattern use dynamically generated SQL where-clauses inside the stored procedure layer to generate a match.

It is tempting to generate parameterised SQL in the business logic tier and execute it against SQL Server for all cases, and this might be your best approach if you cannot identify any very frequent use-cases, but its lacks the test-ability and east performance optimisation of the branching described in (2).

Answer 2

Add all the possible filters as search parameters in the stored procedure. If the filter is set, the parameter will be not null, otherwise it will be null. Then, in the where clause you only use the parameter if it is not null. Something like this:

WHERE (@LastLoginDate IS NULL OR t.LoginDate<@LastLoginDate) AND
      (@CityId IS NULL OR t.CityId=@CityId) 

Answer 3

Finally, I've written Javascript code (a little complex) to :

• Create dynamic filters and its respective inputs
  1. My actual types are only 4 (int,bit,varchar,date)
  2. For 'int' create dynamically an <input type="number">
  3. For 'date' create dynamically an <input type="text"> , with a date plugin
  4. Etc...
• I've created an array to store the id's of my filters and send to the server via AJAX call (and the complete form)

Then in the server code ( C# ) I've written code to:

• Save the campaign details.
• Loop over the parameter array.
  1. Save each filter added with the CampaignId provided by the AddCampaign stored procedure (after save the campaign), the filter id provided by each select value of the form, and the value provided on the generated input.

And then in SQL Server , I've just used the stored procedure that I've supplied on the question, a little modified:

CREATE PROCEDURE usp_MarketingUsersGetFiltered
-- Add the parameters for the stored procedure here
@CampaignId int,
@Debug BIT = 0
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
declare @sql varchar(max) = '', @Nsql nvarchar(max), @DatabaseId INT;
-- Insert statements for procedure here

set @DatabaseId = (SELECT DatabaseId from tblMarketingCampaigns where Id=@CampaignId);
select @sql +=  ' AND '+quotename(f.TargetColumn) + f.Operation + 
            case f.InputType 
            when 'int' then cf.Value
            when 'bit' then cf.Value
            else quotename(cf.Value,'''')               
            end
from dbo.transMarketingCampaignFilters as cf
join dbo.tblMarketingCampaignFilters as f
on cf.CampaignFilterId = f.Id
where cf.CampaignId = @CampaignId

 set @sql = '
 select MarketingUserId,
        FirstName,
        LastName,
        tblMarketingUsers.Email as UserEmail,
        CompanyId,
        CompanyName,
        Address1,
        Address2,
        City,
        SmartInsightProfile,
        LastLoginDate,
        AdvancedProfileStatus
 from dbo.tblMarketingUsers 
 left join dbo.tblCompanies on tblMarketingUsers.CompanyId=tblCompanies.Id
 where DatabaseId = @DatabaseId
 '+ @sql+'
 OPTION (RECOMPILE)';

if @Debug = 1 print @sql;
set @Nsql = @sql;
exec sys.sp_executesql @Nsql,N'@DatabaseId INT',@DatabaseId;

END
GO

My tables are something like this

Where tblMarketingCampaignFilters stores rows similar to :

(Id , Name , InputType ,TargetColumn ,Operator)

(1, 'Advanced Profile Status', 'int', 'AdvancedProfileStatus', '<')

(2, 'Last Login Date', 'date', 'LastLoginDate', '<')

(3, 'Geo Location (City)', 'varchar', 'City', 'LIKE')

and the final result :

 select MarketingUserId,
        FirstName,
        LastName,
        tblMarketingUsers.Email as UserEmail,
        CompanyId,
        CompanyName,
        Address1,
        Address2,
        City,
        SmartInsightProfile,
        LastLoginDate,
        AdvancedProfileStatus
 from dbo.tblMarketingUsers 
 left join dbo.tblCompanies on tblMarketingUsers.CompanyId=tblCompanies.Id
 where DatabaseId = 3
  AND [City]LIKE'%salta%' AND [SmartInsightProfile]!=1 AND [LastLoginDate]<'20150708' AND [AdvancedProfileStatus]<42
 OPTION (RECOMPILE)

Thank you very much for the help!

I know that this is not the BEST way, but it works for me now!

Hope this will help someone else.