Dynamically build SQL-Queries wit Esqueleto and Template Haskell?

Question

I'm writing a webapp wit Yesod & Persistent. I have a SQL-Database with several tables, containing characteristics of my "projects". I have a main table and for the Option with multiple values extra tables linked with the id.

The user should be able to choose for witch of those characteristics he want's to filter and specify the filter value. If the user filters for the os, the license and the coding the SQL-Query would look like this:

runquery :: (YesodPersist site, YesodPersistBackend site ~ SqlBackend) =>
             String -> String -> String
            -> HandlerT site IO [Entity Project]
runquery os license coding = runDB
  $ select $ distinct
  $ from $ \(p `InnerJoin` pl `InnerJoin` l `InnerJoin` pc
            `InnerJoin` c `InnerJoin` o `InnerJoin` po) -> do
     on $ p ^. ProjectId ==. pl ^. ProjectLicenseFkProjectId
     on $ p ^. ProjectId ==. pc ^. ProjectCodingFkProjectId
     on $ p ^. ProjectId ==. po ^. ProjectOsFkProjectId
     on $ l ^. LicenseId ==. pl ^. ProjectLicenseFkLicenseId
     on $ o ^. OsId      ==. po ^. ProjectOsFkOsId
     on $ c ^. CodingId  ==. pc ^. ProjectCodingFkCodingId
     where_ ( o ^. OsName ==. val (Just (pack os)))
     where_ ( l ^. LicenseName ==. val (Just (pack license)))
     where_ ( c ^. CodingName ==. val (Just (pack coding)))
     limit 50
     return p

but I don't want to join always all tables, since that would be very bad for performance when there are a lot of tables but the user filters only on a few. But I also don't want to write a Query for every combination of query-able features, since that would mean writing N² mostly identical queries.

The 'on' and 'where'-clauses could be done dynamically depending on whether we want to filter or not. But the joins are within the parameters of the Lambda-function. I found no way to build this dependent on outer variables.

So I thought Template Haskell might do the trick… I started learning TH and converting the core of the query to TH. But now I'm stuck and not sure whether TH can help me and whether it is the right way?

So here is my progress with Template Haskell:

foo os license coding = lamE [pat] (code)
    where
        p = mkName "p"
        po = mkName "po"
        pl = mkName "pc"
        pc = mkName "pl"
        pat = pat' [os, license, coding] [varP po, varP pl, varP pc]
        pat' []         []        = varP p
        pat' ((Just _):ws) (q:qs) = infixP q (mkName "InnerJoin") (pat' ws qs)
        pat' (Nothing:ws)  (q:qs) = pat' ws qs
        code = do
            case os of
              Just _ -> [|
                  on $ $(varE p) ^. ProjectId ==. $(varE po) ^. ProjectOsFkProjectId
                  |]
              Nothing -> [| return () |]
            case license of
              Just _ -> [|
                  on $ $(varE p) ^. ProjectId ==. $(varE pl) ^. ProjectLicenseFkProjectId
                  |]
              Nothing -> [| return () |]
            case coding of
              Just _ -> [|
                  on $ $(varE p) ^. ProjectId ==. $(varE pc) ^. ProjectCodingFkProjectId
                  |]
              Nothing -> [| return () |]
            [| do
            limit 50
            return $(varE p) |]

So I'd like you're help:

• Can/Should I do this with Template Haskell?
• If so: how can I call the function foo with arguments?
• If not: what is the right solution?

Answer 1

So I found out that using sub-queries is in my case much faster than joins anyway and you can do them if needed:

runquery os license coding = runDB
    $ select $ distinct
    $ from $ \p -> do
         case os of
           Just os' ->
             where_ $ p ^. ProjectId `in_`
               subList_select ( distinct $ from $
                 \(o `InnerJoin` po) -> do
                   on $ o ^. OsId       ==. po ^. ProjectOsOId
                   where_ $ o ^. OsName ==. val (Just $ pack os') 
                   return $ po ^. ProjectOsPId
                   )
           Nothing -> return ()
         case license of
           Just license' ->
             where_ $ p ^. ProjectId `in_`
               subList_select ( distinct $ from $
                 \(l `InnerJoin` pl) -> do
                   on $ l ^. LicenseId      ==. pl ^. ProjectLicenseLId
                   where_ $ l ^. LicenseName ==. val (Just $ pack license') 
                   return $ pl ^. ProjectLicensePId
                   )
           Nothing -> return ()
         -- ...
         limit 50
         return p

To reduce the big amount of duplicated code I then used Template-Haskell:

gencheck t = code
  where
    tableId       = mkName $ t ++ "Id"
    crosstableId  = mkName $ "Project" ++ t ++ "XId"
    crosstablePId = mkName $ "Project" ++ t ++ "PId"
    tableName     = mkName $ t ++ "Name"
    var           = mkName $ fmap toLower t
    code = [|
      case $(varE var) f of
           Just _filter ->
             where_ $ p ^. ProjectId `in_`
               subList_select ( distinct $ from $
                 \(o `InnerJoin` po) -> do
                   on     $ o  ^. $(conE tableId)   ==. po ^. $(conE crosstableId)
                   where_ $ o  ^. $(conE tableName) ==. val (Just _filter)
                   return $ po ^. $(conE crosstablePId)
                   )
           Nothing -> return ()
           |]

runquery f = runDB
    $ select $ distinct
    $ from $ \p -> do
         $(gencheck "Os")
         $(gencheck "Cat")
         $(gencheck "License")
         $(gencheck "Coding")
         $(gencheck "Gui")
         limit 50
         return p