Query Objects with Repository Pattern Part 2

As promised in my previous post, I’m going to make our query object a little more flexible and dynamic.

First, this is what I really want to be able to do something like this:

var customers = repo.FindBy(
                new TopCustomersWithLowDiscountQuery()
                    .IncludePreferred()
                    .BelowDiscountThreshold(3)
                    .WithMoreSalesThan(500)
                    .As_Expression()
);

Better yet, maybe even something like this:

var customers = repo.FindBy(
        new TopCustomersWithLowDiscountQuery()
            .IncludePreferred()
            .BelowDiscountThreshold(3)
            .WithMoreSalesThan(500)
        .AndAlso(
            new DeliquentCustomersQuery()
                .WithDebtOver(9999))
            
);

Strongly typed query objects, completely testable outside of the repository, chain-able together with other like-typed query objects using AndAlso or OrElse.

Ok, now how do we do it?

Expression Tree Helper (Naive)

First, I started with an extension method class to make dealing with some of the Expression<Func<ENTITY, bool>> expressions (which can get old to type) easier.  What I needed was the ability to take two expressions and AndAlso or OrElse them together.  AndAlso (&&) and OrElse (||) are both binary expressions represented by the BinaryExpression class in System.Linq.Expressions.  You can combine two expressions together with any binary expression type by using the Expression.MakeBinary() method.  One problem though is that both Expressions start with a different parameter (i.e. the ‘c’ in (c=>c.AnnualSales > 999)).  So you can’t just join them together because, unfortunately, the Expression Tree compiler will get the ‘c’ parameters jumbled and it won’t work.

The way I found to deal with this problem is to basically wrap an ‘Invoke()’ expression around the other lambda using the first lambda’s parameter.  In C# parlance, it’s the difference between these two things:

c => c.AnnualSales > 99 && c.Discount < 4
    -- VS.--
c => c.AnnualSales > 99 && new Func<Customer, bool>(x=> x.Discount < 4)(c));

See how the second one actually involves wrapping the other Lambda (.Discount < 4) with a function and then invokes it?  I’m not sure if that’s EXACTLY what goes on when you use Expression.Invoke(), but that’s what I like to tell myself when I’m working with Expression Trees. It also helps to keep me from ending up in the corner in the fetal position drooling and babbling incoherently which is, unfortunately, a frequent occurrence when dealing with Expression Trees.

You may notice that I put the condition “naive” on this section. This is because my expression tree helper is very naive and doesn’t account for a lot of the crazy things you can do with Expression Trees. This means that you will probably bump against its limitations and have problems. Sorry in advance for this, but I’m stretching the limits of my knowledge here and doing well to write coherently about it. If you have problems, let me know and maybe we can work it out together.

Anyhow, once you’ve invoked the other lambda, you can join them together with whatever binary expression you want, and then you have to re-wrap them in a Lambda again in order to continue working with it.  Without further ado, here’s my expression helper extension methods:

public static class ExpressionHelpers
{
    public static Expression<Func<T, bool>> AndAlso<T>(
        this Expression<Func<T, bool>> left,
        Expression<Func<T, bool>> right)
    {
        return BinaryOnExpressions(left, ExpressionType.AndAlso, right);
    }

    public static Expression<Func<T, bool>> OrElse<T>(
        this Expression<Func<T, bool>> left,
        Expression<Func<T, bool>> right)
    {
        return BinaryOnExpressions(left, ExpressionType.OrElse, right);
    }
        

    public static Expression<Func<T, bool>> BinaryOnExpressions<T>(
        this Expression<Func<T, bool>> left,
        ExpressionType binaryType,
        Expression<Func<T, bool>> right)
    {
        // Invoke that lambda with my parameter and give me the bool back, KKTHX
        var rightInvoke = Expression.Invoke(right, left.Parameters.Cast<Expression>());

        // make a binary expression between the results (i.e. AndAlso(&&), OrElse(||), etc)
        var binExpression = Expression.MakeBinary(binaryType, left.Body, rightInvoke);

        // Wrap it in a lambda and send it back
        return Expression.Lambda<Func<T, bool>>(binExpression, left.Parameters);
    }
}

 

With that out of the way, we can get on to the less complicated stuff which is chaining all these things together.  The next thing I did was to create a simple abstract base class for my query objects (I’m sure there’s a million better ways to do this, but to get things running, this was the simplest thing that worked for right now).

Query Base Class

The query base is quite simple, actually. It just shuffles around the expressions and provides some convenience methods for you to chain them together:

public abstract class QueryBase<ENTITY>
{
    private Expression<Func<ENTITY, bool>> _curExpression;

    public Expression<Func<ENTITY, bool>> AsExpression()
    {
        return _curExpression;
    }

    public Expression<Func<ENTITY, bool>> AndAlso(QueryBase<ENTITY> otherQuery)
    {
        AsExpression().AndAlso(otherQuery.AsExpression());
    }

    public Expression<Func<ENTITY, bool>> OrElse(QueryBase<ENTITY> otherQuery)
    {
        AsExpression().OrElse(otherQuery.AsExpression());
    }

    protected void addCriteria(Expression<Func<ENTITY, bool>> nextExpression)
    {
        _curExpression = (_curExpression == null)
                            ? nextExpression
                            : _curExpression.AndAlso(nextExpression);
    }
}

You can AND and OR two queries together (must be of the same type, for now).  Sub-classes can add their own expressions in a nice, easy-to-use lambda expression style.

Query Implementation

And now, on to one of the actual query classes. Remember our ridiculously named and implemented TopCustomersWithLowDiscountQuery from my last post?  Here it is in its more simplified form complete with Fluent-API bonus material:

public class TopCustomersWithLowDiscountQuery : QueryBase<Customer>
{
    public TopCustomersWithLowDiscountQuery IncludePreferred()
    {
        addCriteria(c => c.Preferred);
        return this;
    }

    public TopCustomersWithLowDiscountQuery BelowDiscountThreshold(decimal discountThresh)
    {
        addCriteria(c => c.Discount < discountThresh);
        return this;
    }

    public TopCustomersWithLowDiscountQuery WithMoreSalesThan(int salesThresh)
    {
        addCriteria(c => c.AnnualSales > salesThresh);
        return this;
    }
}

To use it, just chain the methods together. Consider this test case:

[Test]
public void low_discount_high_sales_customers_should_be_selected()
{
    _query = new TopCustomersWithLowDiscountQuery()
        .BelowDiscountThreshold(3)
        .WithMoreSalesThan(500);

    var high = 15m;
    var low = 1m;

    _customers.AddRange(new[]
    {
        new Customer {Id = 1, Discount = low, AnnualSales = 200},
        new Customer {Id = 2, Discount = low, AnnualSales = 800},
        new Customer {Id = 3, Discount = high, AnnualSales = 1000}
    });

    Results.Count().ShouldEqual(1);
    Results.ElementAt(0).Id.ShouldEqual(2);
}

 

If you want to chain them together, just use the AndAlso or OrElse methods.  Consider this other test case which uses OrElse:

[Test]
public void preferred_customers_that_are_also_bad()
{
    _query = new TopCustomersWithLowDiscountQuery()
        .IncludePreferred();

    var otherQuery = new DeliquentCustomersQuery()
        .WithPendingLitigation()
        .WithDebtOver(999);

    _customers.AddRange(new[]
    {
        new Customer {Id = 1, Preferred = true, PendingLitigation = false, OutstandingDebts = 4000},
        new Customer {Id = 2, Preferred = false},
        new Customer {Id = 3, Preferred = false,  PendingLitigation = true, OutstandingDebts = 4000}
    });

    var results = _customers
        .AsQueryable()
        .Where(_query.OrElse(otherQuery));

    results.Count().ShouldEqual(2);
    results.ElementAt(0).Id.ShouldEqual(1);
    results.ElementAt(1).Id.ShouldEqual(3);
}

Related Articles:

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Chad Myers

Chad Myers is the Director of Development for Dovetail Software, in Austin, TX, where he leads a premiere software team building complex enterprise software products. Chad is a .NET software developer specializing in enterprise software designs and architectures. He has over 12 years of software development experience and a proven track record of Agile, test-driven project leadership using both Microsoft and open source tools. He is a community leader who speaks at the Austin .NET User's Group, the ADNUG Code Camp, and participates in various development communities and open source projects.
This entry was posted in LINQ. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://dotnetslackers.com/community/blogs/simoneb Simone Busoli

    This looks like your are rewriting linq.

  • http://codeprairie.net/blogs/chrisortman Chris Ortman

    I have been using ISpecification as my query object
    It looks something like
    interface ISpecification
    {
    bool IsSatisfiedBy(T object);
    DetachedCriteria CreateCriteria();
    }

    So far these have been pretty trivial to construct. I do also have a PropertySpecification that lets me pass an expression to it.

    The worst thing is that right now I’m just manually creating the criteria rather than figuring it out from the expression so there is some duplication.

    I tend to favor the repository per aggregate with specialized method calls for queries though because I have found those easiest to read/test

  • http://chadmyers.lostechies.com Chad Myers

    @Simone

    I might be! This is as much a learning exercise for me as anything.

  • http://chadmyers.lostechies.com Chad Myers

    @Chris

    How do you test your query-methods-on-the-repository, though?

  • http://codeprairie.net/blogs/chrisortman Chris Ortman

    User u = new User();
    u.MakeActive();

    UserRepository repository = new UserRepository();
    repository.Save(u);

    var activeUsers = repository.FindAll(new ActiveUserSpecification())
    //or var activeUsers = repository.FindActiveUsers();

    Assert.Equal(1, activeUsers.Count());

    Is that what you mean?

  • http://chadmyers.lostechies.com Chad Myers

    @Chris:

    So you have to hit the DB to test it? That’s exactly what I’m trying to avoid. I’d rather not have to hit the DB to test my query specification is doing the right thing.

    I might have an integration test that goes end to end that involves the query and the repo at some point, but I’d rather not have more than just a few things that I have to test that actually hit the DB.

  • http://johnteague.lostechies.com jcteague

    @Simone,
    He is not rewriting linq, we is putting abstractions around linq type expressions to improve readbility and increase DRY.

    Scattering linq expressions in your code is going to lead to mainainability problems when your exressions need to change.

  • http://colinjack.blogspot.com Colin Jack

    On the repository approach, in my view it should definitely be one per aggregate root if you’re doing standard DDD (I don’t think Fowler is so specific though).

    Anyway in my view the Specification pattern is the best solution so I’ve been trying to use the Linq Specification approach that Ian Cooper blogged about:

    http://iancooper.spaces.live.com/blog/cns!844BD2811F9ABE9C!451.entry

    Not that different than your approach but although the idea is sound it doesn’t currently work that well with Linq To NHibernate.

    Also I’m not sure the fluent interface here is working for me, perhaps more specifications that you combine? Dunno.

    @Chad
    On testing, there was some discussion on the DDD newsgroup about this ages ago.

    I’ve tended to run the tests against the database but ultimately you could have specific repositories with methods that have a nice interface that just wrap the creation and execution of the specifications (Evans/Fowler both talk about this).

  • http://www.codeprairie.net/blogs/chrisortman Chris Ortman

    I don’t see much point in testing queries without hitting the db though.

    If the object is more akin to a finder and has some sophisticated query generation logic then I would do something like this:

    var userRepository = new UserRespository(new InMemoryRepository())

    or

    var expectedCriteria = DetachedCriteria.For
    //… build the criteria I expect

    Assert.Equal(expectedCriteria.ToString(), new ComplexSpecification().CreateCriteria().ToString());

  • http://chadmyers.lostechies.com Chad Myers

    @Colin:

    Dang, that Ian Cooper! I knew I had seen something like this before but my googling game up empty. Spot on.

    I know my FI sucks, I was trying to make a simple point and everyone is beating me up over it, lol. Ian’s approach is very similar, but more modular.

    One point about all this is that the specification is not inherently dependent upon the repository and can be tested in isolation. There’s also a distinct possibiliy of externalizing the specifications as a DSL that might be easier to tweek outside of the main application code. This is potentially a slippery slope, though, and subject to abuse. But if you keep a tight ship and require that all changes to the spec DSL have corresponding tests and don’t get rolled out to production without having been tested, it’s just as good as having it internal to the app quality-wise, but much easier for the stakeholders to see and visualize when reading the requirements-represented-as-code.

    @Chris

    Because the queries are inherently business logic. The statement that a “Top Customer” is one that has sales over a certain amount and is ‘preferred’ is business logic. I turn the question back around on you: Why do I need a DB to test my business logic? Or better yet, Why does my business logic require a repository?

  • http://codeprairie.net/blogs/chrisortman Chris Ortman

    You don’t _need_ a DB, but I prefer it probably because I am not 100% comfortable with the NHibernate criteria API and I want to make sure I am using it correctly.

    ISpecification is perfectly testable without a DB though, and usiing an ISpecification doesn’t require a repository. That’s why on my interface I have just IsSatisfiedBy(T object).

    I think our two approaches are very similar just different naming between specification and query.
    In terms of just naming, I think query is more suggestive that there is a database than specification. Dunno if that is good or bad though

  • Akash Chopra
  • Chris Martin

    By using LINQ constructs in your Repository, are you not tying your application to LinqTo*? What if I have a client that requires plain old ADO.NET? (current situation.; ) )

    I’m not sure how I could convert an ExpressionTree to SQL.

  • http://chadmyers.lostechies.com Chad Myers

    @Chris Ortman

    Absolutely right about the naming. I should’ve used Specification instead.

    @Akash

    I’ve seen that before. That article helped me finally break through my first barrier of groking Linq. I had forgotten about it until just now. Thanks for reminding me and for dropping a good link for the viewers here.

    @Chris Martin

    Yes, I’m betting that pretty much all providers will be Linq2X based from now on (Linq2Sql, Linq2NHibernate, Linq2LLBLGen, Linq2EntityFramework, etc).

    As far as converting an expression tree to SQL, that’s what Linq2SQL is all about.

    But, back to your point, you’re correct: this would not be an appropriate solution for old-fashioned unadulterated ADO.NET usage.

  • Lucas Goodwin

    I like the idea. On the repository per root versus single repository bit. We’ve done things both ways and find the repository per model object works best. More explicit and easier to maintain (less concerns merged together).

    However, we don’t use the specification/et al patterns in our repositories. They’re specific methods for specific requests, which are mapped to data adapters internally.

    Our reasoning is thus: If I want all the active users from a user repository, I don’t care how the database is setup, so I don’t care that the underlying query could contain “business” logic in the where clause. All I care about is that the data adapter is returning to me active user objects.

    This allows the DB schema to change dramatically and only affects the SQL queries in the DataAccess layer. Not the repository in the business layer.

    Hopefully that makes sense.

    When testing the repositories, we just mock the adapters. When testing the adapters, we are hitting the DB directly, but that’s ok as they’re integration tests, not regularly run unit tests.

  • flipdoubt

    Any chance you could make the repository implementation and the tests available in a download so Community Server doesn’t get its mitts all over it?

  • http://www.bjro.de Björn Rochel

    Hey Chad,

    does this code run with Linq to NHibernate or some other Linq provider?

    I’ve tried to get it work with both NHibernate and EF. Both exited with similar exceptions ala “unknown expression” or “expression type not supported”.

    P.S: Don’t get me wrong I’m totally on your side when it comes to testing with in memory repositories (which by the way works exellent with that approach). I just wanted to see it in a action on a db . . . .