Building a simple compiler with OCaml

Question

I've spent a day trying to a simple compiler program but still have difficulties understanding this function.

let rec expression tokens = 
  let (trm, tokens') = term tokens in 

here so (trm, tokens') is a tuple, is term a function or a variable? How can a tuple = two variables or a function

  expTail trm tokens'

and
  expTail trm tokens = 
  match tokens with
    Tokenizer.PlusTok::tokens' ->
      let (trm', tokens'') = term tokens' in expTail (Plus(trm, trm')) tokens''

  | Tokenizer.MinusTok::tokens' ->
      let (trm', tokens'') = term tokens' in expTail (Minus(trm, trm')) tokens''

  |  _ -> (trm, tokens)

what does this line mean? it doesn't seem to call anything. And why is it now (trm, tokens) not (trm', tokens'')

and
  term tokens = let (trm, tokens') = factor tokens in termTail trm tokens'

Not really sure what this line means either.

and
  termTail trm tokens =
  match tokens with
    Tokenizer.TimesTok::tokens' -> 
      let (trm', tokens'') = factor tokens' in termTail (Times(trm, trm')) tokens''

  | Tokenizer.DivTok::tokens' -> 
      let (trm', tokens'') = factor tokens' in termTail (Div(trm, trm')) tokens''

  | Tokenizer.ModTok::tokens' -> 
      let (trm', tokens'') = factor tokens' in termTail (Mod(trm, trm')) tokens''

  | _ -> (trm, tokens)

and
  factor tokens =
  match tokens with
  Tokenizer.LParenTok::tokens' -> 
  let (expr, tokens'') = expression tokens' in
      (match tokens'' with
      Tokenizer.RParenTok::tokens''' -> (expr, tokens''')
  | _ -> raise (Syntax "Bad factor, failed to find )\n")) 

  | (Tokenizer.IntTok i)::tokens' -> (Bits i,  tokens')

  | _ -> raise (Syntax ("Bad factor."))

Thanks guys!

Answer 1

These are pretty basic questions. It might be good to start with an OCaml tutorial (maybe see previous SO question OCaml Resources? ).

Some answers:

let (trm, tokens') = term tokens

This calls a function named term passing tokens as the argument. The result is a pair. You're going to refer to the first element of the pair by the name trm and the second by the name tokens' .

and
   term tokens = let (trm, tokens') = factor tokens in termTail trm tokens'

These are the lines that define the term function called above.

  |  _ -> (trm, tokens)

This is the default case for the pattern match. If neither of the other two patterns matches, the value is given by this alternative. This is basically a value, not a function call. However, in some sense there is a function involved--you're constructing a pair. The comma ( , ) is used as nice syntax for constructing a pair. (The parentheses are actually optional, though stylistically I like to see them.)

The names tokens and tokens' are just different names. There's no inherent relation between them. In this default case, no value named tokens' is defined. So it's not possible to refer to such a value here. Conventionally, adding a "prime" to the end of a name is used to indicate that the new value is somehow derived from the old one.