如何在递归宏规则中扩展子模式？

Question

I am writing a macro to conveniently match nested structure in an enum typed variable to a compile-time template. The idea is to leverage Rust's pattern matching to enforce specific values in certain locations of the structure, or bind variables to other interesting locations. The basic idea works in my implementation but it fails for nested patterns. I believe the problem is that once a part of the macro input has been parsed as $<name>:pat it cannot later be parsed as a $<name>:tt .

To avoid ambiguous use of the term pattern I'll use the following notation in accordance with the Rust documentation:

• A pattern is what appears in match arms, in if let statements, and is matched in macros by the fragment specifier $<name>:pat .
• A matcher is the left-hand side of a syntax rule in a macro.
• A template is the part of the input to my macro that determines how the macro will expand.

Playground MCVE

This is a simplified version of the enum type I am using:

#[derive(Debug, Clone)]
enum TaggedValue {
    Str(&'static str),
    Seq(Vec<TaggedValue>),
}

For example, the following expression

use TaggedValue::*;
let expression = Seq(vec![
    Str("define"),
    Seq(vec![Str("mul"), Str("x"), Str("y")]),
    Seq(vec![Str("*"), Str("x"), Str("y")]),
]);

could be matched by this macro invocation:

match_template!(
    &expression,                               // dynamic input structure
    { println!("fn {}: {:?}", name, body) },   // action to take after successful match
    [Str("define"), [Str(name), _, _], body]   // template to match against
);

Here, on a successful match the identifiers name and body are bound to the corresponding subelements in expression and made available as variables in the block passed as second argument to the macro.

This is my effort to write said macro:

macro_rules! match_template {
    // match sequence template with one element
    ($exp:expr, $action:block, [$single:pat]) => {
        if let Seq(seq) = $exp {
            match_template!(&seq[0], $action, $single)
        } else {
            panic!("mismatch")
        }
    };

    // match sequence template with more than one element
    ($exp:expr, $action:block, [$first:pat, $($rest:tt)*]) => {
        if let Seq(seq) = $exp {
            // match first pattern in sequence against first element of $expr
            match_template!(&seq[0], {
                // then match remaining patterns against remaining elements of $expr
                match_template!(Seq(seq[1..].into()), $action, [$($rest)*])
            }, $first)
        } else {
            panic!("mismatch")
        }
    };

    // match a non sequence template and perform $action on success
    ($exp:expr, $action:block, $atom:pat) => {
        if let $atom = $exp $action else {panic!("mismatch")}
    };
}

It works as expected for non-nested templates, and for nested templates I can manually nest macro invocations. However, directly specifying a nested template in a single macro invocation fails with a compilation error.

match_template!(
    &expression,
    {
        match_template!(
            signature,
            { println!("fn {}: {:?}", name, body) },
            [Str(name), _, _]
        )
    },
    [Str("define"), signature, body]
);
// prints:
//   fn mul: Seq([Str("*"), Str("x"), Str("y")])

match_template!(
    &expression,
    { println!("fn {}: {:?}", name, body) },
    [Str("define"), [Str(name), _, _], body]
);
// error[E0529]: expected an array or slice, found `TaggedValue`
//   --> src/main.rs:66:25
//    |
// 66 |         [Str("define"), [Str(name), _, _], body]
//    |                         ^^^^^^^^^^^^^^^^^ pattern cannot match with input type `TaggedValue`

Playground MCVE

I suspect the error is saying that [Str(name), _, _] is matched as a single slice pattern which is accepted by the third macro rule where it causes the type mismatch. However, I want it to be a token tree so that the second rule can decompose it into a succession of patterns.

I tried to change the second rule to ($exp:expr, $action:block, [$first:tt, $($rest:tt)*]) => but this only causes the error to occur at the outer level.

What modifications to the macro are required so that it can recursively expand such templates?

(I don't think token munching as in Recursive macro to parse match arms in Rust works here because I explicitly want to bind identifiers in patterns.)

This is what I expect the macro invocation to expand to (Ignoring the mismatch branches for brevity. Additionally, I simulated macro hygiene by postfixing the seq variable):

// macro invocation
match_template!(
    &expression,
    { println!("fn {}: {:?}", name, body) },
    [Str("define"), [Str(name), _, _], body]
);

// expansion
if let Seq(seq_1) = &expression {
    if let Str("define") = &seq_1[0] {
        if let Seq(seq_1a) = Seq(seq_1[1..].into()) {
            if let Seq(seq_2) = &seq_1a[0] {
                if let Str(name) = &seq_2[0] {
                    if let Seq(seq_2a) = Seq(seq_2[1..].into()) {
                        if let _ = &seq_2a[0] {
                            if let Seq(seq_2b) = Seq(seq_2a[1..].into()) {
                                if let _ = &seq_2b[0] {
                                    if let Seq(seq_1b) = Seq(seq_1a[1..].into()) {
                                        if let body = &seq_1b[0] {
                                            { println!("fn {}: {:?}", name, body) }
                                        }
                                    }
                                }
                            }
                        } 
                    } 
                } 
            } 
        } 
    } 
} 

The full expansion is a bit verbose but this slightly shortened version captures the essence of what should happen:

if let Seq(seq) = &expression {
    if let Str("define") = &seq[0] {
        if let Seq(signature) = &seq[1] {
            if let Str(name) = &signature[0] {
                if let body = &seq[2] {
                    println!("fn {}: {:?}", name, body)
                }
            }
        }
    }
}

Finally, here is another playground link that shows the individual steps of recursive expansion. It's very dense.

Answer 1

Indeed, it seems the problem is that the macro matches a comma separated list of patterns. Thus, in the input [Str("define"), [Str(name), _, _], body] the macro interprets the inner [...] as a slice pattern that cannot match an expression of type TaggedValue .

The solution is to expand the input as token trees. However, this requires a small trick because a single token tree cannot represent every pattern. In particular, a pattern of the form Variant(value) consists of two token trees: Variant and (value) . These two token can be combined back into a pattern before invoking a terminal (non-recursing) rule of the macro.

For example, the rule to match such a pattern in a single-element template starts like this:

($exp:expr, $action:block, [$single_variant:tt $single_value:tt]) =>

These tokens are passed together to another invocation of the macro with

match_template!(&seq[0], $action, $single_variant $single_value)

where they are matched as a single pattern by the terminal rule

($exp:expr, $action:block, $atom:pat) =>

The final macro definition contains two additional rules to account for Variant(value) patterns:

macro_rules! match_template {
    ($exp:expr, $action:block, [$single:tt]) => {
        if let Seq(seq) = $exp {
            match_template!(&seq[0], $action, $single)
        } else {
            panic!("mismatch")
        }
    };

    ($exp:expr, $action:block, [$single_variant:tt $single_value:tt]) => {
        if let Seq(seq) = $exp {
            match_template!(&seq[0], $action, $single_variant $single_value)
        } else {
            panic!("mismatch")
        }
    };

    ($exp:expr, $action:block, [$first:tt, $($rest:tt)*]) => {
        if let Seq(seq) = $exp {
            match_template!(&seq[0], {
                match_template!(Seq(seq[1..].into()), $action, [$($rest)*])
            }, $first)
        } else {
            panic!("mismatch")
        }
    };

    ($exp:expr, $action:block, [$first_variant:tt $first_value:tt, $($rest:tt)*]) => {
        if let Seq(seq) = $exp {
            match_template!(&seq[0], {
                match_template!(Seq(seq[1..].into()), $action, [$($rest)*])
            }, $first_variant $first_value)
        } else {
            panic!("mismatch")
        }
    };

    ($exp:expr, $action:block, $atom:pat) => {
        if let $atom = $exp $action else {panic!("mismatch")}
    };
}

Here is a link to the complete example: playground .