Where can I learn about constructing AST's for Scala macros?

Question

Where I can learn how to construct the AST's that Scala's macros generate?

The Scaladoc isn't as helpful as I'd like. For example:

abstract def Apply(sym: Universe.Symbol, args: Universe.Tree*): Universe.Tree
A factory method for Apply nodes.

But how do I figure out what an Apply node is? Where can I find a list of the node types in AST's, and how they fit together?

Answer 1

There isn't a lot of documentation for the internals of the compiler available, but the things that are available should be enough to get started.

Mirko Stocker , has written his Master Thesis about Scala Refactoring . In Appendix D (p. 95) he describes the architecture of the AST. It includes also a graphical overview:

Another way to find information about the AST is to look directly into the sources of reflect.internal.Trees , which contains the AST.

If one needs to find out how a specific source code snippet is represented internally there is reify :

scala> import reflect.runtime.universe._
import reflect.runtime.universe._

scala> showRaw(reify{val i = 0}.tree)
res8: String = Block(List(ValDef(Modifiers(), newTermName("i"), TypeTree(),
  Literal(Constant(0)))), Literal(Constant(())))

Answer 2

You could take a look at the scaladoc ( http://docs.scala-lang.org/overviews/reflection/symbols-trees-types.html#trees ) or at the slides ( http://scalamacros.org/talks/2012-04-28-MetaprogrammingInScala210.pdf , the "Learn to learn" part).

Here's what I usually do. I wrote a simple script called parse , which takes Scala code as an argument and then compiles it with -Xprint:parser -Ystop-after:parser -Yshow-trees-stringified -Yshow-trees-compact ( parse uses another helper script: adhoc-scalac . click here to see its sources as well).

The advantage this approach has over showRaw is that it doesn't require the code to typecheck. You could write a small snippet of code, which refers to non-existent variables or classes, and it still will successfully run and show you the AST. Here's an example of output:

09:26 ~$ parse 'class C { def x = 2 }'
[[syntax trees at end of parser]]// Scala source: tmp36sVGp
package <empty> {
  class C extends scala.AnyRef {
    def <init>() = {
      super.<init>();
      ()
    };
    def x = 2
  }
}
PackageDef(Ident(TermName("<empty>")), List(ClassDef(Modifiers(), TypeName("C"), List(), Template(List(Select(Ident(scala), TypeName("AnyRef"))), emptyValDef, List(DefDef(Modifiers(), nme.CONSTRUCTOR, List(), List(List()), TypeTree(), Block(List(pendingSuperCall), Literal(Constant(())))), DefDef(Modifiers(), TermName("x"), List(), List(), TypeTree(), Literal(Constant(2))))))))

There's also a script called typecheck , which does the same, but stops after typer . That's sometimes useful to understand how exactly the typechecker transforms the parser trees. However, both toolboxes and macros work with parser trees, so I use typecheck for tree construction purposes very rarely.