How to write fixed point build rules in Shake, e.g. Latex

Question

Using the Shake Haskell build library, how can I write a rule using a program that needs to reach a fixed point? Imagine I have a program foo that takes a file input and produces an output file, which should have foo applied repeatedly until the output file does not change. How can I write that in Shake?

The typical example of this pattern is LaTeX.

Answer 1

Firstly, note that calling Latex repeatedly does not always produce a fixed point, so make sure you have a bound on the iterations. Also, some distributions (MikTex) provide Latex versions that automatically run as many times as they need to, so if you use those instead the problem goes away.

Write your own foo_transitive command

The easiest way to solve the problem, assuming each run of foo has the same dependencies, is to solve the problem outside the build system. Just write a foo_transitive command, either as a shell script or as a Haskell function, that when supplied an input file produces an output file by running repeatedly and checking if it has reached a fixed point. The build system can now use foo_transitive and there are no issues about dependencies.

Encode it in the build system

You need to write two rules, one which makes one step, and one which figures out which step is the right one to use:

let step i = "tempfile" <.> show i

"tempfile.*" *> \out -> do
    let i = read $ takeExtension out :: Int
    if i == 0 then
        copyFile "input" out
    else
        let prev = step (i-1)
        need [prev]
        -- perhaps require addition dependencies, depending on prev
        system' "foo" [prev,out]

"output" *> \out -> do
    let f i = do
            old <- readFile' $ step (i-1)
            new <- readFile' $ step i
            if old == new || i > 100 then copyFile (step i) out else f (i+1)
    f 1

The first rule generates tempfile.2 from tempfile.1 and so on, so we can need ["tempfile.100"] to get the 100th iteration. If the dependencies change in each step we can look at the previous result to calculate the new dependencies.

The second rule loops round checking each pair of values in the sequence, and stopping when they are equal. If you are implementing this in a production build system you may wish to avoid calling readFile' on each element twice (once as i-1 and once as i ).

Answer 2

Expanding on @Neil Mitchell 's answer, below is a sample code of foo_transitive . Having said that, for this particular case I'd just use latexmk which Does The Right Thing™.

import Control.Monad.Fix (fix, mfix)
import Control.Monad.IO.Class (MonadIO(liftIO))
import Text.Printf (printf)

type SHA = Int

data TeXCompilationStage 
  = Init
  | BibTeX
  | Recompile SHA
  deriving (Show, Eq)

data TeXResult
  = Stable SHA
  | Unstable
  deriving (Show, Eq)

f retry x budgetLaTeXCalls
  | budgetLaTeXCalls <= 0
      = do
          liftIO $ putStrLn "Budget for LaTeX depleted; result didn't converge"
          return Unstable
  | otherwise
      = case x of
          Init        -> do
            liftIO $ do 
              putStrLn "Init"
              putStrLn "  # latex"
            retry BibTeX (budgetLaTeXCalls-1)
          BibTeX      -> do
            liftIO $ do
              putStrLn "BibTeX"
              putStrLn "  # bibtex"
            retry (Recompile 0) budgetLaTeXCalls
          Recompile previousSHA -> do
            let budgetLaTeXCalls' = budgetLaTeXCalls - 1
                calculcatedSHA    = 3
            liftIO $ do
              printf "Recompile (budget: %d)\n" budgetLaTeXCalls
              printf "  Prevous SHA:%d\n  Current SHA:%d\n" previousSHA calculcatedSHA
            if calculcatedSHA == previousSHA
              then do
                liftIO $ putStrLn "  Stabilized"
                return $ Stable calculcatedSHA
              else do
                liftIO $ putStrLn "  Unstable"
                retry (Recompile (previousSHA+1)) (budgetLaTeXCalls-1)

latex :: Int -> IO TeXResult
latex = fix f Init