Exploring ShadeTree Features, Part 1: Static Reflection with ReflectionHelper

At work, Jeremy and I have been using — and contributing to — some of the code he’s put together over the past few years.  He’s created the ‘ShadeTree’ project which is currently housed in the Storyteller Project source repository. I’d like to do a few blog posts to share with you some of the really neat gems in this project that you can you use in your projects to help accelerate your development.

The first thing I’d like to cover in ShadeTree is the ReflectionHelper class. ReflectionHelper aids with static reflection and the elimination of many ‘magic strings’.  Static reflection is a process I first became aware of on Daniel Cazzulino’s blog post "Statically-typed Reflection with Linq."  With the new Expression Trees feature of .NET 3.5, there are some interesting things you can do that aren’t necessarily query-related.  The subject of Expression Trees is a very complicated one, but they are very powerful and worth some cursory investigation.  Fortunately, you can "dip your toe" into Expression Trees without understanding how everything works and build up from there (at least, that’s what I’m doing, so there!).

First, let’s start with a problem…

Classic Reflection with Magic Strings

Let’s pretend we had an object-to-object mapping problem. Let’s say that we have some sort of web service which returns a simple DTO (data transfer object — a flattened, simple object usually used for the purpose of communicating between two different systems with different models).  So we want to take our rich domain model object graph and flatten it into a DTO in order to send to our remote client. Let’s also say that we have a lot of these situations and that it’s going to be too much work to write one-off mapping functions for each scenario.  We’ve decided we need to build a basic object-to-object mapper that can handle common problems like nulls, string conversion, etc. Please don’t get hung up on these details, the point is: We need a scenario that involves some sort of reflection. Please use your own imagination if my scenario isn’t working for you.

Consider this simple model with the red lines representing the intended mapping from the model to the DTO:

CustomerDTODiagram 

In the past, we may have pulled out our Mighty Hammer of XML +1 for this problem and ended up with something like this:

<DtoMap dtoClass="SomeProject.CustomerDTO, SomeProject" 
        srcRootClass="SomeProject.Domain.Customer, SomeProject">

    <DtoProperty name="Name" mapsToSrcProperty="Name"/>

    <DtoProperty name="SiteName">
        <DtoSourcePropertyMap propretyOnSourceObject="Site" propertyToAccess="Name"/>
    </DtoProperty>

    <DtoPropety name="PostalCode">
        <DtoSourcePropertyMap propretyOnSourceObject="Site">
            <DtoSourcePropertyMap propretyOnSourceObject="PrimaryAddress" propertyToAccess="PostalCode"/>
        </DtoSourcePropertyMap>
    </DtoPropety>
    
</DtoMap>

This would probably work with a little tweaking, but the fundamental problem with this is: Magic strings everywhere!  Our refactoring tools are now likely defeated.  If we ever rename the ‘Site’ property on ‘Customer’, it’s unlikely most refactoring tools would know to change this XML map appropriately.  It’s possible we’d catch this and other similar problem through tests, but it’s just more work and more friction that we don’t need (in this case, at least).  One other problem with this XML is that it’s very verbose, there’s lots of  language overhead here (extra angle brackets, etc). In general, it’d be nice to stick to code if we can (with nice, safe compilers to keep us on the straight and narrow).

You could accomplish this in other ways (without XML). But the code ends up looking something like this:

Type srcType = Type.GetType("SomeProject.Domain.Customer, SomeProject");
PropertyInfo nameProp = srcType.GetProperty("Name");

This code has much of the same problem that the XML-based mapping syntax has (i.e. some refactoring tools may not pick up changes, etc).

Magic strings cause problems, let’s face it. So what can we do, instead?

Static Reflection with Expression Trees and ShadeTree.Core.ReflectionHelper

With static reflection we can build maps in code that reference the types and members directly. This is made easier using ShadeTree.Core.ReflectionHelper.

First, let me show you what our end-result mapping code might look like now that we can use Static Reflection:

DtoMap<CustomerDto> map = new DtoMap<CustomerDto>();

map.AddPropertyMap(
    new DtoMapProperty<CustomerDto, Customer>
        {
            DtoPropertyReference = (dto => dto.Name),
            SourcePropertyReference = (src => src.Name)
        }
    );

map.AddPropertyMap(
    new DtoMapProperty<CustomerDto, Customer>
        {
            DtoPropertyReference = (dto => dto.SiteName),
            SourcePropertyReference = (src => src.Site.Name)
        }
    );

map.AddPropertyMap(
    new DtoMapProperty<CustomerDto, Customer>
        {
            DtoPropertyReference = (dto => dto.PostalCode),
            SourcePropertyReference = (src => src.Site.PrimaryAddress.PostalCode)
        }
    );

 

Not bad, but pretty verbose.  Maybe we could tighten it up a bit with a Fluent API:

var map = new DtoMap<CustomerDto, Customer>()
    .Property(dto => dto.Name).ToSource(src => src.Name)
    .Property(dto => dto.SiteName).ToSource(src => src.Site.Name)
    .Property(dto => dto.PostalCode).ToSource(src => src.Site.PrimaryAddress.PostalCode);

Now, that’s pretty funky, huh? It threw me for a loop when I first saw that.  My first thought was: "But src=>src.Name isn’t a valid expression in this context! You can’t just reference a member without an assignment, for the compiler tells me so!"

You might, rightly, expect to get this compile error:

error CS0201: Only assignment, call, increment, decrement, 
and new object expressions can be used as a statement

So why don’t you?  Well, the trick is that your method or property must take a parameter of type Expression<Func<T,object>>.  Not just a System.Func<T,object>, but you have to have the Expression<> around it. This causes the C# compiler to behave strangely — namely it does not treat the code inside the expression as executable code, but as a parsed language element for LATER evaluation.  "dto.Name" is a valid expression in certain contexts, so the compiler allows it as an Expression<>. If you later try to execute/invoke that expression, THEN you’ll get an error similar to the CS0201 above.  But for right now, the compiler lets it pass, and will generate the IL to pass your method or property a type of Expression<>.

UPDATE: (HT to commenter Paul Batum for catching me on this): Lambda expressions have an implied “return” statement in them, so they will compile *and execute* without the CS0201 issue because they are not a pure member access expression they are a return statement combined with a member access expression.

What can you do with this Expression<> type?  Well, lots of things, but that’s a much larger topic. For right now, you can pass it to the ShadeTree ReflectionHelper and request to get a PropertyInfo from it.  Here, let me show you.  Remember our mapping Fluent API up above? Here’s a crude start on what the DtoMap<DTO, SRC> class might look like:

public class DtoMap<DTO, SRC>
{
    private PropertyInfo _lastDtoProperty;

    public DtoMap<DTO, SRC> Property(Expression<Func<DTO, object>> expression)
    {
        _lastDtoProperty = ReflectionHelper.GetProperty(expression);
        return this;
    }
}

So I have an Expression<> of a lambda (Func<T,object>). That lambda currently holds an expression which can be evaluated into a direct member reference (i.e. the Name property).  We can use some LINQ stuff under the covers to turn that Expression<> into a .NET reflection element: a PropertyInfo.  We’ve now accomplished the same thing as someType.GetProperty("Name"), but without any magic strings!

Now, your refactoring tool should be able to directly pick up the dto.Name reference and, if you ever change the name of the Name property, change that reference for you automatically.

Deep Access Static Reflection and the Accessor interface

We have a big problem with that last code sample, though.  GetProperty() will explode if you pass it something like src.Site.Name (n-level deep property accessors).   ReflectionHelper, fortunately, has a way of dealing with this. It will create the accessor graph for you and return to you the path it took from ‘dto’ to ‘Name’ and add some nifty convenience in there for you as well!  There’s a method on ReflectionHelper called ‘GetAccessor()’.  It’s very similar to GetProperty(), but it handles deep accessor paths.

Let’s look at the ‘ToSource()’ method of our DtoMap now.  ToSource is going to need this deep access because our second call to it uses ‘src.Site.Name’ which would cause GetProperty() to throw an exception.   Here’s the updated DtoMap source with new Accessor goodness:

public class DtoMap<DTO, SRC>
{
    private Accessor _lastDtoAccessor;
    private Accessor _lastSrcAccessor;

    public DtoMap<DTO, SRC> Property(Expression<Func<DTO, object>> expression)
    {
        _lastDtoAccessor = ReflectionHelper.GetAccessor(expression);
        return this;
    }

    public DtoMap<DTO, SRC> ToSource(Expression<Func<SRC, object>> expression)
    {
        _lastSrcAccessor = ReflectionHelper.GetAccessor(expression);
        return this;
    }
}

You should notice that I changed the Property() method to use GetAccessor and I’ve added the new ToSource() method which uses GetAccessor as well.

The ShadeTree.Core.Accessor interface gives us some extra information above and beyond just the straight PropertyInfo as well as two very important methods: SetValue() and GetValue().  These methods expect an object of the root type (in our case, Customer) and will navigate the graph to get or set the value on the property on the object in which we’re interested.  For example, if I have my _lastSrcAccessor we just built in our ToSource() method, we can call GetValue() and pass in a Customer object and get, for example, it’s Site’s Name property.  Consider this example:

var customer = _customerRepository.GetCustomer(9);
customer.Site.Name = "NewSiteName";

string ourName = (string) _lastSrcAccessor.GetValue(customer);

// the _ourName_ variable will now equal "NewSiteName"

 

So Accessor.GetValue() will walk down the path from Customer to Site and from Site to Name and return the Name property value.  If any value in the chain is null, GetValue() will return null.  This is the current behavior, if you don’t like it, please submit a patch (take a look at the ShadeTree.Core.PropertyChain implementation for starters).

Conclusion

There’s one other thing on ReflectionHelper I didn’t mention: GetMethod() which allows you to pass an Expression that references a method and get a MethodInfo back from it so you can use static reflection on more than just properties.  There are a few other things on Accessor that I didn’t mention either that I think are worth you giving a glance at.

I plan on doing a few more of these ShadeTree posts, so please check back later.

Related Articles:

    Post Footer automatically generated by Add Post Footer Plugin for wordpress.

    About Chad Myers

    Chad Myers is the Director of Development for Dovetail Software, in Austin, TX, where he leads a premiere software team building complex enterprise software products. Chad is a .NET software developer specializing in enterprise software designs and architectures. He has over 12 years of software development experience and a proven track record of Agile, test-driven project leadership using both Microsoft and open source tools. He is a community leader who speaks at the Austin .NET User's Group, the ADNUG Code Camp, and participates in various development communities and open source projects.
    This entry was posted in ShadeTree, Static Reflection. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
    • Rob M

      Hi,

      This is an interesting article and the stuff with expressions is interesting too. I’m wondering though why you wouldn’t just make your DTO map take a series of lambda statements that take DTO and SRC as parameters (Action ), and do the mapping.

      e.g.
      var map = new DtoMap()
      .Map((dto, src) => dto.Name = src.Name)
      .Map((dto, src) => dto.SiteName = src.Site.Name);

      This would seem to be a lot simpler to implement and would still support refactoring?

      Rob

    • http://chadmyers.lostechies.com Chad Myers

      @Rob:

      Yeah, something like that would probably be better. Like I said, it’s a contrived scenario, so please don’t get too hung up on the details. There are a lot of other scenarios where the ReflectionHelper thing makes more sense, but unfortunately they are more complicated and difficult to set up in a blog post :(

      One small plus for the Accessor side in this example is that it’ll help you if ‘Site’ is null. If you so src.Site.Name and Site is null, you’ll get an NRE. With Accessor, it’ll just return null.

    • Rob M

      @Chad:

      That’s cool – it was definitely interesting to see the static reflection stuff, big improvement over using strings.

      I’m a bit confused by the point in your article that you wouldn’t expect src => src.Name to compile. Obviously you wouldn’t be able to do the static reflection as there would be no way of parsing the expression tree contained within the lambda, but if the methods Property() or ToSource() just took Func then it would still compile fine though wouldn’t it, as src => src.Name is equivalent to src => { return src.Name; } ?

      Rob

    • http://chadmyers.lostechies.com Chad Myers

      @Rob:

      Yes, it would compile. I’m talking about just something like this:

      src.Name;

      On a line all by itself will not compile because it’s not a full expression.

    • http://paulbatum.blogspot.com Paul Batum

      Much like Rob, I am struggling to understand what you meant about the compiler changing its behaviour when you wrap the lambda as an Expression rather than just a normal delegate such as Func. I played around a bit with this and just can’t replicate the behaviour you are describing. For example, this refuses to compile (as I would have expected before reading this post):

      Expression> exp = s => { s.Length; };

      As Rob said, the example you gave looked like it would compile just fine with simple Func’s. So I still can’t find an example of the behaviour you described. Care to shed some light? Thanks!

    • http://chadmyers.lostechies.com Chad Myers

      @Paul:

      The example I gave would indeed compile as a simple Func, but then it would only be a simple func — a first class function if you will.

      An Expression is a different beast altogether. It REPRESENTS a language element, but is not, in fact, a language element iself. That is, it represents a lambda of a member access expression. The expression is not executable because it’s a representation of a lambda. You can call .Compile() on an expression and it will then, well, compile the language expression into actual executable code. It will then become a real lambda that you can call.

      Expression> is to Func as MethodInfo is to a real method pointer.

      So by using Expression, we can achieve reflection without using magic strings.

    • http://paulbatum.blogspot.com Paul Batum

      @Chad,

      Sorry, I probably wasn’t clear enough. I understand the difference, and why you need to use Expression instead of a plain Func in your example. You can interrogate the expression tree if its an expression, you can’t if its a plain old func. What I don’t understand is this:

      — chad wrote:
      You might, rightly, expect to get this compile error:

      error CS0201: Only assignment, call, increment, decrement,
      and new object expressions can be used as a statement

      So why don’t you? Well, the trick is that your method or property must take a parameter of type Expression>.

      Unless I’m mistaken, the answer to “why don’t you?” is that the compiler allows us to elide the return, braces and semicolon. src=>src.Name is perfectly valid, because its equivalent to src => { return src.Name; }. You get this behavior regardless of whether you are using an expression or just a normal func.

      And If this statement was true:

      — chad wrote:
      This causes the C# compiler to behave strangely — namely it does not treat the code inside the expression as executable code, but as a parsed language element for LATER evaluation.

      Then this line would compile:
      Expression> exp = s => { s.Length; };

    • http://chadmyers.lostechies.com Chad Myers

      @Paul:

      The Expression Tree magic only appears to happen when the Expression is a parameter to a method. I don’t think it works as a standard assignment or even a Property setter assignment.

      If you wish to build up expression trees manually, you can use things like:

      Expression.Lambda()
      Expression.MakeMemberAccess()
      Expression.Equal()
      Expression.AndAlso()
      etc,etc,etc

    • http://paulbatum.blogspot.com Paul Batum

      @Chad

      Sorry for being such a persistent bugger about this, but I still can’t seem to replicate the magic you are describing. Would you mind pasting an example of a lambda expression body that doesn’t compile unless you make it a parameter to a method that takes an Expression?

      This fails to compile for me:

      [Test]
      public void Test()
      {
      Magic(s => { s.Length; });
      }

      private void Magic(Expression> action) { }

    • http://chadmyers.lostechies.com Chad Myers

      @Paul:

      I think I’m starting to understand what you’re pointing at here.

      Like I said above in my reply to Rob, yes, c=>c.LastName will work as a Func parameter because the compiler will translate c.LastName as a “return c.LastName” statement, which is, of course valid. So there’s some compiler trickery happening there so MAKE an otherwise invalid statement, valid.

      One note, you’re using blocks {} around your lambda. That won’t work with the Expression stuff like what I’m talking about (for static reflection purposes). And the static reflection stuff doesn’t work with Action, it must be Func.

    • http://paulbatum.blogspot.com Paul Batum

      @Chad:

      Both you and Jimmy pointed out my mistake with trying to pass the block as type Expression – yep, I was totally wrong with that one.

      However, I’m still not satisfied :)

      — chad wrote
      If you later try to execute/invoke that expression, THEN you’ll get an error similar to the CS0201 above. But for right now, the compiler lets it pass, and will generate the IL to pass your method or property a type of Expression<>.

      I can’t figure out how to write an expression that is fine at compile time, but will result in a CS0201 style error when you try to run it. I would really really love an example of this.

      Thanks for being patient with me.

    • http://chadmyers.lostechies.com Chad Myers

      @Paul

      Yeah, I think I’m mistaken with the CS0201 thing you pointed out. I forgot that the compiler automatically adds the “return” keyword deep under the hood to lambdas like that.

      I’ll correct that later tonight (gotta run right now, unfortunately).

    • http://chadmyers.lostechies.com Chad Myers

      @Paul:

      Thanks again for the catch! Sorry I was being so dense. I updated the post accordingly. Let me know if works for you.

    • http://paulbatum.blogspot.com Paul Batum

      @Chad

      No worries at all! I really digging the new C# features and am doing my best to understand all the in’s and outs. I really like how the expression trees are helping to bridge some of the “metaprogramming gap” that exists at the moment. The NHibernate mapping stuff that you and Jeremy have been doing is a good example I think. Really looking forward to seeing what else you guys come up with!

    • http://realfiction.net Frank Quednau

      Hi there, just got the trunk of storyteller. There seems to be no single line of documentation in it. Looks like the tests also leave well-trodden paths. Are the blog posts tom come the documentation or is there something somewhere that would help me drill the first hole into that?
      Cheers

    • http://chadmyers.lostechies.com Chad Myers

      @Frank: Yeah, Jeremy’s been doing good just to do code on StoryTeller, let alone docs. ShadeTree (which happens to live in the StoryTeller tree house for the time being), at this point, is a code repo and not much more (not a lot of organization).

      The idea is that we’re doing some interesting things at work and some of those things are not related to the IP or profitability of our company so we’re trying to contribute to open source. We’re doing the best we can and trying to put as much as we can responsibly put in there in order to contribute to the community.

      We realize that it’s not terribly useful for other people if it’s a hodge-podge and they can’t discover what’s in it or how to use it. At this point, we have a list of things to do on ShadeTree to make it more friendly for use by people other than us.

      My hope with this blog post series was to try to make some of the gems that are in ShadeTree a little more widely known so you can at least grab some nuggets until we can polish it up a little.

      If you or anyone else is willing to help put some docs together, I’d be willing to spend some time on the phone/skype getting you started and walking through the important things. Sometimes, just having the bootstrapping for documentation (i.e. the outline, layout, styles, etc) helps encourage more people to write more.

      Would you (or anyone else here) be willing to get a documentation shell project going in ShadeTree (an empty HTML project with CSS layout and maybe just some basic ‘getting started’ docs? Maybe a wiki instead?)?