How to create a custom DSL in Ruby like YAML, Cucumber, Markdown, etc?

Question

I currently have a Ruby-based DSL for creating slides that uses instance eval:

# slides.rb
slide {
  title 'Ruby Programming'
  subtitle 'A simple introduction'
  bullet 'First bullet'
  bullet 'Second bullet'
}

# implementation:
class DSL
  class Slide
    def title(title)
      @title = title
    end
    # ...etc...
  end

  def slide(&block)
    @slides << Slide.new.instance_eval(&block)
  end
end

dsl = DSL.new
dsl.instance_eval(File.read('slides.rb'))

Which results in something like this:

---

Ruby Programming

A simple introduction

• First bullet
• Second bullet

---

I would like to take this to the next level by creating a DSL that does not use Ruby syntax. Maybe something more like YAML or Markdown:

title: Ruby Programming
subtitle: A simple introduction
* First bullet
* Second bullet

How can I create a DSL/parser for this type of syntax?

Answer 1

Someone already mentioned Parslet, but I thought I would demo how easy it is.

require 'parslet' 
require 'pp'

slides = <<EOS
  title: Ruby Programming
  subtitle: A simple introduction
  * First bullet
  * Second bullet
EOS

#Best to read the parser from the bottom up.

class SlidesParser < Parslet::Parser
    rule(:eol)          { str("\n") | any.absent? }
    rule(:ws?)          { match('[\s\t]').repeat(0) }
    rule(:rest_of_line) { ws? >> (str("\n").absent? >> any).repeat(1).as(:text) } 
    rule(:title)        { ws? >> str("title:")>> rest_of_line.as(:title) >> eol }
    rule(:subtitle)     { ws? >> str("subtitle:")>> rest_of_line.as(:subtitle) >> eol }
    rule(:bullet)       { ws? >> str("*") >> rest_of_line >> eol }
    rule(:bullet_list)  { bullet.repeat(1).as(:bullets) }
    rule(:slide)        { (title >> subtitle >> bullet_list).as(:slide) }
    root(:slide)
end

# Note: parts can be made optional by adding a ".maybe"  eg. => subtitle.maybe

result = SlidesParser.new.parse(slides)  
pp result
#{:slide=>
#  {:title=>{:text=>"Ruby Programming"@9},
#   :subtitle=>{:text=>"A simple introduction"@38},
#   :bullets=>[{:text=>"First bullet"@64}, {:text=>"Second bullet"@81}]}}

In Parslet, Parsers are only part of the job, as they just converting text into a ruby structure.

You then use tranforms to match/replace tree nodes to make the structure you want.

# You can do lots of things here.. I am just replacing the 'text' element with their value
# You can use transforms to build your custom AST from the raw ruby tree 
class SlidesTransform < Parslet::Transform
  rule(:text => simple(:str))        { str }
  # rule(
  #    :title => simple(:title), 
  #    :subtitle => simple(:subtitle), 
  #    :bullets => sequence(:bullets)) { Slide.new(title, subtitle, bullets) }
end

pp SlidesTransform.new.apply(result)
#{:slide=>
#  {:title=>"Ruby Programming"@9,
#   :subtitle=>"A simple introduction"@38,
#   :bullets=>["First bullet"@64, "Second bullet"@81]}}

Answer 2

Maybe its worth a look at some current open-sourced implementations.

But I have to ask - why are you making your own? why dont you use one which is already available? TOML is great.

Ruby parser implementation: https://github.com/jm/toml

Answer 3

I believe Cucumber uses Ragel for its parser, here's a decent looking intro to it using Ruby...

Treetop is also pretty common, along with Parslet .

ANTLR , Rex and Racc... All kinds of ways to handle external DSLs.

Eloquent Ruby has a chapter on external DSL creation, from basic string parsing and regexes to using Treetop...

Answer 4

You can do rudimentary parsing with regexp. Something like this:

slides = <<EOS
  title: Ruby Programming
  subtitle: A simple introduction
  * First bullet
  * Second bullet
EOS

regexp = %r{
  (title:\s+)(?<title>[^\n]*)|
  (subtitle:\s+)(?<subtitle>[^\n]*)|
  (\*\s+)(?<bullet>[^\n]*)
}x

tags = {
  'title' => 'h1',
  'subtitle' => 'h2',
  'bullet' => 'li'
}

fUL = false
slides.lines.each {|line|
  md = line.match(regexp)
  md.names.select{|k| md[k]}.each {|k|
    puts '<ul>' or fUL = true if k == 'bullet' && !fUL
    puts '</ul>' or fUL = false if k != 'bullet' && fUL
    puts "<#{tags[k]}>#{md[k]}</#{tags[k]}>"
  }
}
puts '</ul>' if fUL