Domain Specific Languages with Boo: AST Macros

For those of you who don’t know what Boo is its a statically typed CLR language with Python like syntax that lets you extend it’s compiler, and the language itself easily by giving you access to the AST (Abstract Syntax Tree) and compiler’s context directly.  This gives you very powerful tools for building your own language or Domain Specific Language or DSL from here on out. Some examples of DSL’s include rSpec and Fluent NHibernate. In fact the entire subject of what is a DSL and what types of DSL there are and how to create a proper DSL could be a book itself and a fascinating one at that. 

Which is why I’ve been reading Ayende’s book DSLs in Boo: Domain Specific Languages in .NET. To make sure I understood the concepts I’ve taken to building a toy BDD DSL called bSpec, it’s got a long way to go to be something useful and I may not care to take it that far, however I did get my brain wrapped around a really cool thing called AST Macros. 

AST macros give you FULL access to the compiler context and full AST of the code. I could describe this in more detail but a code sample goes light years to making this helpful.

Let’s say I want to have a “shouldFail” method that fails my spec immediately how would I do that?

#macro declaration
macro should_fail:
    #create code using Quasi Quotation ( ”[|" and "|]“ )
    codeblock = [| 
         raise AssertionError("failed by request")
    |#exit macro statement
    return codeblock #now replace ”should_fail” with ”raise AssertionError() ”

#client code

should_fail #throws AssertionError


In reflector in C# the code looks like so:

throw new AssertionError(“automatic failure as requested by ’should fail call’”);

So how did this happen?  Well as part of Boo’s compiler pipeline it will run the macros first and replace the macro statement itself with what the macro returns. So far this doesn’t seem particularly interesting until you try and actually manipulate and work with the AST itself or get access to compiler objects.   For a simple example lets look at what I did with the “describe” macro.

#Describe Macro 

macro describe:
        #creates a reference to the first argument
        itemtodescribe = cast(ReferenceExpression, describe.Arguments[0])

        #using a simple trick to prevent compiler errors. I just want access to the body of ”block” so
        # you can safely ignore block in my example
        #note the key is using $itemtodescribe . this is how you pass in macro variables to the AST
        logdescribe = [|
                        _spechash[$itemtodescribe] = null
        yield logdescribe   #this will become the first statement
        yield describe.Body #the specified code block after ”describe FooBart: ” will now be placed

#Unit Test Code with asserts removed

class DescribeSpecs_When_Not_Nested:

        private _spechash as Hash
        private _called as List[of string]

        public def constructor():
                _spechash = {}
                self._called = List[of string]()
                describe FooBart:
                        _called.Add(“called from FooBart spec”

#compiled output via reflector

class DescribeSpecs_When_Not_Nested:

        private _spechash as Hash
        private _called as List[of string]

        public def constructor():
                _spechash = {}
                self._spechash[typeof(FooBart)] = null;  #this is the key change
                self._called = List[of string]()

So our macro has replaced code again, but this time references a field in a class that it had no prior knowledge of, yet it safely compiles. However, it would not have if there had been no field called “_spechash”. Amazingly this simple trick is only one of many ways you can extend the Boo language in a late binding fashion yet still get all the benefits of the CLR and compile time error checking.  Follow my blog in the coming weeks for more of the Boo language.

About Ryan Svihla

I consider myself a full stack polyglot, and I have been writing a lot of JS and Ruby as of late. Currently, I'm a solutions architect at DataStax
This entry was posted in Boo, DSL. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • “the entire subject of what is a DSL and what types of DSL there are and how to create a proper DSL could be a book itself and a fascinating one at that.”

    yup! :)

  • I’m sorry, it still doesn’t strike me that these macros solve more problems than they create. In this case, it strikes me that a base class with a describe method would have been more elegant. If macros get complex, sorting out the compiler errors becomes a nightmare. If they don’t, there doesn’t seem to be a lot of point. Put it another way: there are an awful lot of well-documented problems with macro programming, and very few of them are addressed by replacing text manipulation with graph manipulation.

    I think I probably need to expand on this as well… :)

  • @Julian I think that’s a bit extreme as there are well documented problems with every approach, macro’s have their place and it’s more about tradeoffs.

    This case was a simple way to show how it works, please don’t take it as a use case, as this was a unit test.

    Think of macros as a CLR version of standalone function objects and use them accordingly. They’re nice in DSL creation and not something I’d recommend you make heavy use of in general purpose programming.

  • Fair enough, my experience of Boo Macros isn’t huge, but I’ve tried implementing my own, and came to the conclusion that it didn’t really have the power I needed (It evaluates before it’s done type resolution.) My principal experience is with Binsor, which is great when things are going well, but a nightmare when they’re not: figuring out the syntax requires you to read several thousand lines of code…

    I still think that diagnostics, in particular, are a serious issue if you ever start to heavily use it. Admittedly, my own argument here is reminding me of the people who told me that they didn’t think I should use delegates in my code because people wouldn’t understand them…