What is the difference between the these two scheme macros?

Question

provided using mit-scheme

The only modification is from (cthen (make-syntactic-closure env '(it) (third exp))) to (cthen (third exp))

In brief, what difference does make-syntactic-closure make?

(define-syntax aif
  (sc-macro-transformer
   (lambda (exp env)
     (let ((test (make-syntactic-closure env '(it) (second exp)))
       (cthen (make-syntactic-closure env '(it) (third exp)))
       (celse (if (pair? (cdddr exp))
              (make-syntactic-closure env '(it) (fourth exp))
              #f)))
       `(let ((it ,test))
      (if it ,cthen ,celse))))))


(let ((i 4))
  (aif (memv i '(2 4 6 8))
       (car it)))

(define-syntax aif
  (sc-macro-transformer
   (lambda (exp env)
     (let ((test (make-syntactic-closure env '(it) (second exp)))
           (cthen (third exp))
       (celse (if (pair? (cdddr exp))
              (make-syntactic-closure env '(it) (fourth exp))
              #f)))
       `(let ((it ,test))
      (if it ,cthen ,celse))))))


(let ((i 4))
  (aif (memv i '(2 4 6 8))
       (car it)))

I tried the two version of macro, but got the same result.

Answer 1

A syntactic closure captures the syntactic environment and allows identifiers present in it to be used in the form as variables, instead of having a meaning provided by the environment of the macro transformer (It also lets you specify a list of free names like it in this case that can be set or otherwise used in the macro body and have that binding used in the expanded form; basically it removes those names from the given environment)

An example:

(let ((i 4)
      (x '(a b c)))
  (aif (memv i '(2 4 6 8))
       (car x)))

With the first version of your aif macro, this evaluates to 'a . With the second version, it generates an error Unbound variable: x , because in the macro cthen is a list, not a syntactic closure, and x is unbound in the macro transformer so it can't be found when the expanded body is evaluated.

Answer 2

Syntactic closures were introduced in a 1988 paper by Alan Bawden. The paper provides a reference implementation, with caveats that it's not the only possible implementation; however, the reference implementation can used to understand syntactic closures.

The syntactic closures paradigm processes an top-level expression in two passes.

First, the expression is recursively turned into a tree of syntactic closures. This turns out to be a bottom-up process. Subexpressions are closed over, and then recombined into expressions (subject to possible transformations), and those expressions are syntactically closed and so on.

Then, the entire transformed tree of syntactic closures is traversed to generate code. Bawden calls this step compilation, and the reference implementation specifies a compile function. The syntactic closures themselves do much of this work.

Every individual element ends up in its own syntactic closure. For instance if (let ((a 1)) (+ aa)) is syntactically closed, there end up being child syntactic closures: there will be one for each occurrence of a , one for the + , and for (+ aa) , and for the overall let .

Syntactic closures aren't code; they are objects. In Bawden's implementation, which targets a dialect of scheme which has no objects, they are implemented using closures: essentially one-method objects which holds information in the captured lexical environment. However, certain objects, when passed to make-syntactic-closure don't become syntactic closures, such as self-evaluating atoms. Therefore in the resulting representation, it is necessary to be able to distinguish ordinary Scheme syntax from a syntactic closure. Bawden achieved this by wrapping the function representation of the syntactic closure in a two-element vector, whose first element is a certain symbol:

(define (syntactic-closure? x) 
  (and (vector? x) 
       (= 2 (vector-length x)) 
       (eq? ' syntactic-closure (vector-ref x 0)))))

What's in the (vector-ref x 1) position is a function. That function is involved in the compile step: its job is to produce the code for that syntactic closure. When the compile step walks the code representation, whenever it sees an object that satisfies syntactic-closure? , it just calls the function, passing it the syntactic environment:

(define (compile-syntactic-closure syntactic-env syntactic-closure) 
  ((vector-ref syntactic-closure 1) syntactic-env))

In a piece of code that was originally (let ((a 1)) (+ aa)) , each occurrence of a will have been translated to a syntactic closure. This will have been done under the control of the let code walking case of make-syntactic-closure which understands that a is being defined as a lexical variable, and so a got classified as an identifier. The syntactic closure for an identifier has a compile function which will return the renamed version of the identifier. When making the closure, the let code walker constructs an environment. It sticks a into that environment, associating it with some generated symbol like #:g0023 (let's use Common Lisp uninterned symbol syntax). The three syntactic closures for a will then have a compile function which just returns #g:0023 . Thus, a renaming is perpetrated, and that renaming is the basis for the hygiene which is achieved.

What you don't see in this programming interface:

(define-syntax aif
  (sc-macro-transformer
    (lambda (exp env) ...)))

is that the code returned by your lambda is also subject to a hidden make-syntactic-closure . However, this is clear in Alan Bawden's paper, which doesn't use define-syntax . His example macros are just top-level expander functions which have to be explicitly entered into an environment. Eg push looks like this:

(define (push-expander syntactic-env exp) 
  (let ((obj-exp (make-syntactic-closure syntactic-env ‘() 
                                         (cadr exp))) 
        (list-var (make-syntactic-closure syntactic-env ‘()
                                          (caddr exp)))) 
    (make-syntactic-closure scheme-syntactic-environment ‘()
                            `(set! , list-var 
                                   (cons ,obj-exp ,list-var)))))

As you can see, the expander passes the result of the quasiquote to one more call to make-syntactic-closure , which we don't see in the define-syntax interface.

Thus when your macro neglects to call make-syntactic-closure on a piece of syntax and just inserts it into the template verbatim, that piece of syntax doesn't escape from the closure-making process. The code gets walked again by another make-syntactic-closure call, which uses a higher-level environment. Thus any identifiers in the code get interpreted as belonging to a different scope. Possibly, they may still be renamed!

Suppose that your aif macro neglects to close over celse . Then suppose aif is called in a piece of code like this:

(let ((a 1))
  (let ((a 2))
    (aif #f
      (* a 10)
      (* a 100))))

Note the two definitions of a in different scopes. When aif is called to expand, it will produce something like this pseudo-code, where we use #<syn-clo...> to indicate closures:

(let ((it #f))
  (if it
    (* #<syn-clo a #:g0023> 10)
    (* a 100)))

Because a wasn't treated with make-syntactic-closure , it is completely oblivious to syntactic environments; it stays as a .

This result then gets closed over. This time the let produced by the macro gets code walked, and the it identifier gets closed:

(let ((#<syn-clo it #:g0024> #f))
  (if #<syn-clo it #:g0024>
    (* #<syn-clo a #:g0023> 10)
    (* a 100)))

the recursion pops again and we are in the let construct which binds the inner a . That let expander obtained the above body as a syntactic closure. It now generates this code:

(let ((#<syn-clo a #:g0023> 2))
  (let ((#<syn-clo it #:g0024> #f))
    (if #<syn-clo it #:g0024>
      (* #<syn-clo a #:g0023> 10)
      (* #<syn-clo a #:g0022> 100)))) ;; <--??? What is this #:g0022

Whoa, what happened? Because the a in (* a 100) was not closed up, the inner let walked onto that a and closed it up. It closed it up using the env which it had been given, and that's coming from the outer let . This becomes clear when the expansion finishes:

(let ((#<syn-clo a #:g0022> 1))        ;; outer a is #:g0022
  (let ((#<syn-clo a #:g0023> 2))      ;; inner a is #:g0023
    (let ((#<syn-clo it #:g0024> #f))
      (if #<syn-clo it #:g0024>
        (* #<syn-clo a #:g0023> 10)         ;; inner a
        (* #<syn-clo a #:g0022> 100)))))    ;; incorrectly, outer a

So now when the code generation "compile" step is invoked to unwrap and get rid of all the closures to get executable code, we get:

(let ((#:g0022 1))
  (let ((#g:0023 2))
    (let ((#g:0024 #f))
      (if #:g0024
        (* #:g0023 10)
        (* #:g0022 100)))))

The alternative else part of ouf if is referencing the a at the wrong scope: it's referring to the #:g0022 a and not the #:g0023 a .

It is the responsibility of the expander to make sure that every free identifier it encounters is closed up against the environment it has been given. It has this responsibility because it holds the knowledge for how to walk the particular form, like aif . If that expander neglects to convert some identifier to a syntactic closure, then that identifier will fall victim to another form's code walk, which will assign it to the wrong environment. Those parts that are correctly closed do not fall victim to another code walk because syntactic closures are treated as opaque, "cooked" objects in the expansion phase; they get unwrapped in the generation phase. Once an identifier is in a closure, it is safe; the closure protects it from misinterpretation in another context. It may be acted upon by seventeen more macros or special form walkers; it doesn't matter. Those will all just propagate it as-is.

This is the essence of syntactic closures: to pin down the meaning of an identifier in the correct scope exactly once, and then protect that interpretation in a safe box, which is not opened until the expansion process is complete. The unboxing produces new names for everything that needs to be renamed, and so transparency is achieved.

---

PS

The thing to understand also is that the processing of all primmitives (special forms) perpetrates renaming with syntactic closures; it's not just macros. Code that is expanded under the syntactic closure paradigm will have its local variables renamed even if it doesn't contain any macros.

PPS

In Bawden's implementation, there is no let primitive; his compile step understands only lambda , and a few other forms, not let . Bawden constructs a Scheme environment in which let has a binding to a syntactic-closure-based expander that he provides, de facto making it a predefined macro. The part of Bawden's code which performs the environment extension that maps variables to their renamed generated symbols is the compile-lambda function. It creates a new syntactic environment, adds all the variables to it, making them available to nested make-syntactic-closure calls.