CQRS revisited

Introduction

In my opinion Command Query Responsibility Segregation (CQRS) is one of the most useful architectural patterns when used in the context of a complex line of business application (LOB). Martin Fowler writes

… At its heart is a simple notion that you can use a different model to update information than the model you use to read information.

Observations

A lot of today’s applications are unnecessarily complex or slow since they do not use CQRS. This is a high level of a typical application

All read and write operations are handled by a domain model in the backend. Often this domain model is very anemic and consists mainly of entities that are pure data containers (DTOs) and some services that manipulate those entities. Sometimes the repository pattern is used to abstract the data store from the domain’s perspective. In this case we might see repository interfaces that look similar to this

public interface ICustomerRepository
{
	Customer GetById(int customerId);
	Customer GetByEmail(string email);
	Customer GetByOrderId(string orderId);
	IEnumerable<Customer> GetByLastName(string lastName);
	IEnumerable<Customer> GetTop10Customers();
	// more GetXYZ(...) methods

	int Save(Customer customer);
	void Delete(Customer customer);
}

We can see that in the same interface we have both methods that query data and methods that change data. And this is a problem! Reading or querying data is a fundamentally different concern than changing data. As such we should keep these two types of operations separate from each other. There are various reasons for that

  • Reading data usually happens much more frequently than writing data. In many applications the frequency of reading versus writing is easily 10 to 1 or even 100 to 1.
  • When reading data we usually want to retrieve quite a bit of data. It can be a list of records or a record with associated data records, e.g. a customer with a list of all her orders and her shipping and billing addresses, etc.
  • Read operations do not change data they rather provide a glimpse into a snapshot of the current state to the observer. Repeating the same read operation over and over again should always return the same result.
  • On the other hand write operations typically only affect a very small set of data. Most often one single data record is affected by a write operation and even there only a few properties of the overall data record are changed.
  • Write operations change the state of the system thus they have side effects. The world looks different before and after a write operation.
  • Read operations need to be fast. A user is easily frustrated if a query takes more than half a second or so.
  • Users are much more tolerant when it comes to write operations. They know that something important happens in the system and thus tolerate longer response times.

Lesson learned

When looking at all these points mentioned above we can learn quite a bit and do a better job in our applications.

Reading data

Since read operations need to be fast we should make sure that

  • data can be accessed in way that needs the least amount of DB queries possible for the specific context
  • no (complex) business logic needs to be executed while retrieving the data. Business logic should only be executed when the application tries to modify data.
  • aggregated data should not be calculated/aggregated on the fly when querying the data but should be pre-calculated. This can happen whenever data is changed that affects the aggregated values, which is rather rare compared to read operations, as stated above.

Read operations can never change any data! Read operations need to be side-effect free.

Writing data

As we have discussed above write operations typically only affect a very limited set of data. Let’s say we have an operation “add product XYZ to shopping cart” in an e-commerce application. In this operation we add an item to the shopping cart object consisting of product number and quantity. The shopping cart itself is identified by a shopping cart Id. Thus the command that we send to the backend might look like this

public class AddProductToShoppingCart
{
	public int ShoppingCartId { get; set; }
	public int ProductId { get; set; }
	public int Quantity { get; set; }
}

As we can see although the operation is a very important one only a very limited set of data is sent to the back-end for processing. Note that the name of the command gives the context of the operation whilst the payload of the command only contains the minimal amount of information needed to successfully fulfill the requested operation.

Since we are now clearly distinguishing read and write operations the command does NOT return any data other than maybe a status message telling the sender whether or not the operation has succeeded and possibly some error messages (which can be displayed to the user) if the command has failed since some business rules were violated.

Implementations

In our new world where read and write operations are separated a high level diagram of an application looks like this

Classical n-tier application

Let’s assume we have a LOB application using an RDBMS (SQL Server, Oracle, MySql, etc.) to store the data and .Net as the application framework. We could then use e.g. NHibernate or Entity Framework as the ORM for write operations whilst we are using bare bone ADO.NET for the queries.

We use database views to simplify the data access when reading data. Views often provide a good means to present data in a de-normalized fashion which is ideal for queries.

If we use this approach and continue using the repository pattern from above then our resulting customer repository might look much simpler

// using CQRS
public interface ICustomerRepository
{
	Customer GetById(int customerId);
	int Save(Customer customer);
	void Delete(Customer customer);
}

We only have a get method to retrieve the Customer record or entity we want to change and a save method to add a new instance of type Customer. For completeness I also added a Delete method to remove an existing Customer from the data store although I usually tend to avoid this. IMHO data should never be physically deleted from the data store… but that is a totally different discussion.

Event sourcing and CQRS

If we are using Event Sourcing (ES) as another architectural pattern we can come up with the following high level diagram

Here events generated by the domain as a result of commands that change state are written to an event store. Queries on the other hand get their data from a read model which is separate from the event store and is eventual consistent with the write model (the event store). A process running in the background asynchronously dispatches events (as they are flowing into the event store) to observers that build up the read model. In this particular case the read model could be a document database like Mongo DB or Raven DB and/or a file based full index like Elastic Search or Solr.

Summary

CQRS is one of my preferred architectural patterns when it comes to complex LOB applications. Unfortunately a lot of existing LOB applications do not follow this rather straight forward pattern. This leads to a lot of unnecessary complexity. The execution paths of commands and queries are tightly coupled and thus cannot be individually tweaked or tuned. CQRS solves this problem quite nicely and allows us to build the best possible system for a given context. CQRS is not just limited to modern type of applications using event sourcing and/or DDD. It can be applied in any application that reads and writes data.

About Gabriel Schenker

Gabriel N. Schenker started his career as a physicist. Following his passion and interest in stars and the universe he chose to write his Ph.D. thesis in astrophysics. Soon after this he dedicated all his time to his second passion, writing and architecting software. Gabriel has since been working for over 25 years as a consultant, software architect, trainer, and mentor mainly on the .NET platform. He is currently working as senior software architect at Alien Vault in Austin, Texas. Gabriel is passionate about software development and tries to make the life of developers easier by providing guidelines and frameworks to reduce friction in the software development process. Gabriel is married and father of four children and during his spare time likes hiking in the mountains, cooking and reading.
This entry was posted in architecture, CQRS, How To, learning, patterns. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Patrick

    re: “Unfortunately a lot of existing LOB applications do not follow this rather straight forward pattern.”

    I agree that the pattern is straight forward, but do you feel like the user-experience takes on an additional layer of complexity if the design of the system should be one where the user can “read what they wrote”? The amazon shopping experience, as an example, has trained the user to expect asynchronous updates. Similarly, paying your credit card online has a similar non-immediate expectation. However, suppose you are building an SaaS accounting platform where your targeted user base has an expectation that when they pay a batch of invoices, the invoices are immediately closed and balanced out to $0. You could argue that they need to be re-trained an accept an asynchronous user experience, but for the sake of conversation, suppose the business doesn’t want to do so.

    With a single read/write model, the synchronous user experience is “free”. Of course, this comes at other costs, which your article does a good job describing. When you split the models, doesn’t UX become more complex? I’ve heard of people using client-side state in the browser to simulate read-what-you-wrote – there is some engineering required to pull this off.

    Anyhow, good write up. You definitely highlight some of the strength of this approach. It would be interesting to see a post that describes the complexities of CQRS. Everything has a trade-off, right?

    • Daniel Whittaker

      “Everything has a trade-off” – so true. I’ve been using CQRS and event sourcing for a few years now. The biggest trade off I’ve noticed is in the initial learning curve. It is a very different way of thinking and as such creates a steep learning curve. I’ve also found the infrastructure code to be more complex, however, once it is done it rarely changes.

      On the other side, in most cases, I’ve found it much easier to add new features and change the way existing things work. I really love the knock on benefits of having an event store. Being able to replay events into new features so they look like they existed from the beginning of the application. This is particularly when building reports. I’ve been blogging a lot about CQRS and event sourcing recently. Feel free to take a look to get some more insight: http://danielwhittaker.me

      Gabriel – Thanks for the great post!

      • gabrielschenker

        I agree, the learning curve is quite steep. It is a paradigm shift. It’s somewhat similar to other areas e.g. using the VI editor versus other editors. Once it makes “click” though you don’t want to go back ever :-)

    • Stefan Billiet

      Concerning the the synchronous user experience:

      It depends. There are workflows where you are changing data in one place, that is used in a completely different place. In this case async doesn’t matter because by the time you get to that other screen, the async process would have already been completed (unless it’s a long running process).
      If the changed data needs to be “immediately” visible, you can setup a system where the backend sends (domain) events back to the frondend (sort of server-sent events), through libraries like SignalR or socket.io. If you have this system, you just wait on the page after having pressed save untill you get an event back indicating the job is done; then you can refresh the page.

      Having set up something like this recently, if your backend is already eventdriven, then sending those events back to the browser is reasonably straightforward.

      • jbogard

        “There are workflows where you are changing data in one place, that is used in a completely different place” <- I find these few and far between. Even in ecommerce systems where I've worked with billions of dollars flowing through them, pricing is most assuredly synchronous with the "back end". A nightly ETL job pushes these to the read-only caches on the front-end.

        Users might not expect to see the side effects of their changes immediately, but they certainly expect to see their change immediately in whatever system they're using.

        We used event sourcing in one app. Devs loved it, users h…a…t…e…d it, because every operation introduced latency through all this back-end event publishing, denormalizing, SignalR'ing etc etc. We were promised better scaling, and users complained the app was slow.

        • Stefan Billiet

          It depends on the interface; if it’s a task based interface it’s generally possible.
          As to the situation you described, I can only assume something went wrong architecturally. If latency was such a big problem, you could always resort to handling projections in the same process as the handling of commands and only then publish the events.

          • jbogard

            Synchronous projections should be the default mode. Async projections goes against decades of previous art and millenia of how humans have interacted with each other.

            In our task-based interfaces, the users always expected that at least they could see that their task was recorded. Users expect consistency. They don’t expect “approve invoice” “view outstanding invoices” “wtf why is my approved invoice still there”. We then have to mimic consistency with tricks, either with write-through client-side models, notifications that worsen client experience, or by simply forcing consistency by waiting on the server side for projections to finish.

            It’s also why every single one of our clients has ditched RavenDB. Nobody actually wanted asynchronous read models inside the same bounded context.

            I use CQRS all the time now, just not the event-sourced async mode. Just instead of views I use LINQ projections.

          • gabrielschenker

            You have a very strong opinion and I respect it. But please don’t make it a “one has to…” global law. I can give you countless examples where it makes perfectly sense and it is even desirable to have asynchronously processed projections (eventual consistent read model). Right now I am working in a project where it is strictly required by the customer that everything happens asynchronously (we use NServiceBus). In my previous project which was a huge enterprise app we also used eventual consistent read models mixed with fully consistent ones. But one has to match that with a task based UI of course. CRUD-ish applications are not good candidates for eventual consistency.
            Please let’s not make this a religious war. As always “it depends” which approach is the “right” one.

          • jbogard

            I’m genuinely curious what kind of problem needs async projections in the same app/context boundary. I’ve seen it implemented countless times, and every time it’s been needless complexity, matched up with some sort of half-baked mimicked consistency in the front-end.

            And these were *all* task-based UIs. To me that’s orthogonal. What I never found were fire-and-forget messages. Users never wanted fire-and-forget except for very particular cases (send email). If something was a long-running process, they expected a synchronous recording of the request, and dispatching of the request. The request was recorded in a process entity with status, and back-end events updated the status of the request.

            The places where I’ve seen Event Sourcing (not CQRS) work well are places where the domain is actually around complex event processing. And not “Approve Invoice/Invoice Approved” sort of things, but where the business conceptualized their domain, thought about it, dreamed about it, represented it, as a stream of events. Like stock prices, for example.

          • gabrielschenker

            I will try to author another post about synchronous versus asynchronous. There I hope to clarify certain open questions. But once again this post was first and foremost about the CQRS pattern which is in no way related to the question of synchronous versus asynchronous. CQRS is about recognizing that changing state is fundamentally different from querying state and that as a consequence these two concerns should be treated separately if possible. Also I mentioned it in my post that I am strictly referring to (somewhat) complex LOB applications.

          • jbogard

            Oh of course – we use CQRS in just about every app we build these days. It’s the Event Sourcing part (one that often gets conflated with CQRS) that I was cautioning on. And its value has less to do with domain complexity than the business already seeing work done as processing events.

          • gabrielschenker

            I knew we will be on the same page, eventually ;-)

          • Marco

            I quite agree with all you said about synchronous/asynchronous even tho all the projects I worked on which were using CQRS were dealing with model updates using async events. There a couple of things tho that I didn’t fully understand.You update the model that lives in the same context boundary, but what about the other models outside of the context but still within your domain? And do you have a different way to dispatch those sync events (i’m guessing here that all your other events are dispatched asynchronously). Also do you use MSDT transactions when updating the models synchronously? and if so, in’t that a big issue?

          • jbogard

            Updating other models outside the context and inside the domain – sounds like domain events!

          • Marco

            Ha ok I see, within the context the changes are updated asynchronously but outside of the BC you use domain events… I just wasn’t sure, grand stuff.

          • jbogard

            Not quite – inside the bounded context changes are updated via domain events (synchronously or asynchronously, it really depends on what’s needed).

          • Marco

            “I’m genuinely curious what kind of problem needs async projections in the same app/context boundary” That’s what I was talking about, by projections you’re are talking about the read models right?

          • jbogard

            Yes. I really have yet to see an architectural need for separate read models inside the same bounded context. If you need more than data model, you likely have more than one bounded context at hand.

          • Marco

            Maybe my understanding is wrong here, but aren’t read models outside the scope of the domain? Within a domain we can have one or more bounded contexts, but I always thought that the scope of those bounded contexts were limited to that of the domain, which does not include the read models, right?

          • gabrielschenker

            It depends. Currently we mostly use NServiceBus to dispatch events that are consumed outside the bound context. I have also in the past used a more passive approach where a bound context would not expose any events publicly but only an API over which other bound contexts would query data if needed. Push vs. Pull…

          • Marco

            That’s a nice approach, surely one way of doing things doesn’t fit well in all situations. The only problem I see with that approach is the different versions of an aggregate a consumer’s bounded context might miss, since you’re not publishing events, but then again maybe fetching data once a day/week is more than enough for that particular business requirement.

          • Patrick

            The “wtf why is my approved invoice still there” comment is spot on – this is where CQRS, IMO, becomes less “straight-forward” and may itself create a lot of “unnecessary complexity”. For other parts of the system, like a dashboard with some summary-level information, separating the read-model feels much more natural. This aligns with your previous remark:

            > “Users might not expect to see the side effects of their changes immediately, but they certainly expect to see their change immediately in whatever system they’re using.”

            +1 .. good thread

          • Andrew Siemer

            The problem I have with the statement above “this is where CQRS IMO becomes less…” CQRS doesn’t assume any form of event sourcing, eventual consistency, etc. It is just a pattern to separate your read and write models/paths. You don’t have to put DDD behind it, or a messaging sub system, etc. The CQRS as a pattern conversations always gets (no offense) polluted with additional threads of conversation that reflect poorly on a very simple pattern.

          • Andrew Siemer

            The thing that I find happens all the time when folks talk about CQRS is that they quickly down grade the conversation into all the things that people like to dangle off the pattern of CQRS. The pattern doesn’t dictate synch vs asynch. Though having components that are asynch can enable scalability and having task based UI’s vs crud based UI’s are a major part of the retraining effort to drive the health of the internal model and how you maneuver changes to that model over time.

            But there are always strategies to get around this any of the shortcomings that you might bump against depending on the client/project needs. FB does a good job of this on first approach. I post something to my wall. And in my view the post is on my wall. However, if I immediately open a new window and view my wall it has not yet found its way there. Essentially they either cached my addition locally or locked me in to a server path that had my changes in its realm. I feel that thy are making head way on educating users about eventual consistency but they do a great job of ensuring that most users have no idea that this misdirection has happened. And they have a good enough durability picture in their infrastructure that they mean that it will truly eventually be consistent. If you don’t have the ability to ensure durability – yes – eventual consistency is a horrible idea.

            But in many cases I agree that this “experience” is not acceptable to business users on business apps. But oddly enough the users of business systems have been mis-trained to expect this immediate processing/consistency. I like to provide a representation of reality where we treat system transactions as though they are pages in the steps of the process. When I submit something I guarantee that I have the document with a status of “posted”, “submitted”, or “received” — not “accepted” or “processed” etc. This initial transaction may be consistent – not eventual. Posted simply means that we have it. The answer (such as for loan processing, or order submission) will come eventually – and eventually may usually be a wicked fast rate in that the consumer of the application may never see the initial status. But at least with this model the user sees the reality of the system.

            The best use case to point too for this way of doing things in most of our every day lives is when you buy something from Amazon. They take my card info and the order. They send me an order received email. Everything – for the majority of users – appears that the order is good to go. However, if for some reason the card doesn’t go through at a later time they loop me into another process where they attempt to get updated payment information. This too is an acceptable experience that does at times require some end user retraining. But most of the time it is a feature of the system that is totally invisible.

          • jbogard

            Small quibble – this isn’t “mistraining”. It’s millenia of human experience. I write down an order on a piece of paper, hand it to a human – they don’t forget about my order.

            Let’s say we have the 2 tabs issue – this DOES happen IRL. I hand a cashier a deposit, they write it down. I go to the next cashier, they don’t have my deposit yet and the world doesn’t explode.

            Why? Because the first teller *tells me* that “my deposit will show up in a couple of days”. The user experience doesn’t pretend it’s sync – it’s not, but it explicitly builds around an async process, makes it completely up-front and obvious.

            If you look at talks done by Facebook/Disqus/Twitter engineers, NONE of them built this async architecture at the beginning. It’s much more complicated. They only did it when demands of scale forced them to, years into the product and only when they absolutely had to.

            My single biggest beef with the event sourcing community is that the async architecture is somehow simpler, despite so, so much evidence to the contrary. Your Amazon example is more an example of a “Saga” or “Process manager” – every step is synchronous, but there’s an overall asynchronous flow. Very, very different than “I approved the invoice, I go to the open invoices and it’s still there”. If you wanted to have a good UX, you’d have something like dual write models on client and server, or an explicit UX that tells the user that this request is getting processed in the backend.

            Separate “asynchronous, long-running business transaction” from “asynchronous read models” – users understand the first, but the second is far more complicated to both manage from the UX and backend perspective.

          • Marco

            Nobody forgets your order if your message is sent asynchronously either (I know you know that, just basing on your example) and I can give you a fairly common scenario where we use async messaging without even realizing: mail letters. Things move a lot quicker nowadays, but messaging has almost always been an async process – I drop my letters in a mailbox, they are picked up, delivered and later on I might get a reply, I just don’t seat next to the mailbox waiting for a reply/acknowledge message to arrive and in some cases it never does, so the millenia of human experience you’re talking about needs to be taken with a pint of salt. You are very right tho that async messaging is way more complicated to implement then synchronous, but scalability always comes at a cost, but the really is that most of the people doing messaging don’t really do it because of the scalability, you can easily archive that in a cloud based hosting, they do it, because of the decoupling it provides.

          • jbogard

            Ah yes – async messaging. The problem I see is that we design fake UIs that pretend messaging is synchronous. You don’t get confused with mail because the UX is designed for that flow. But something like “I hit the button to place and order and it’s asynchronous from there”, well you ARE waiting at the mailbox. It’s your browser. The other side doesn’t “know” that the order has been placed until the message is received. You don’t keep asking the mailbox “where’s my order” – as you do in a browser hitting F5.

            Millinia of human experience still holds :) – we shouldn’t design a UX (sync or async) that ignores the realities of what’s actually going on underneath the covers.

            Let’s design a UI that actually conforms to reality. I have a UI to place an order, that is async. I click a button, the message is sent async, and I’m redirected to a “Thank You” page that says “thank you for your order your order will be processed soon you will be emailed the details of your order”. No link to see details there – since this is async there’s no guarantee my order has been saved anywhere.

            In order for no one to “forget” my order – it needs to be saved at some point during my order process. But then we’re not pure fire-and-forget commands :)

          • Marco

            A few years ago, before the times of asynchronous browser calls (ajax) most of the web applications were indeed synchronous, from top to bottom but I challenge you to tell me of a “modern” website which does’t use ajax, and as you know an UX built with ajax is asynchronous, so aren’t we doing that all the time, deceiving the user to believe that things are happening as they click the button when in fact they are being processed asyncronously? What I usually do is issue a fire&forget command and then pull to get the status of the command and depending on the status I update the UX. The example you gave, Amazon does it (or at least they used to, I haven’t order anything from them fro a while now), they tell you that you will receive an email confirming your order and the same happens to a lot of airlines. Most of them use Amadeus for all their booking and I can only guess how difficult it would be for them to handle all those requests synchronously, so at the end of the booking process they always give you a reservation number, but that number is the one the airline registered you with against the Amadeus platform, it doesn’t mean your reservation is fully booked, you always receive an email a few minutes later. Google does it all the time too, you delete an email from your browser (desktop) and if it won’t be until a few seconds that the same message is gone from your mobile device even if you keep refreshing. The real problem is that many people don’t know how the web works neither have a full understanding of the real implications of working with asynchronous systems and end up building something awful in terms of UX experience. I once worked for a company that had a peek of 3,5 billion web requests/daily, from all the different platforms and all their systems are synchronous (back-end) so it always comes down to the quality of software we’re building. I prefer to work with asyn messaging systems, but most of the time we just don’t need that extra complexity. Reality is, we like to justify the need to build systems using asynchronous patterns because of the scalability/performance it provides, and we end up with a complete horrible UX/ service-layer – I do that mistake myself quite often, thinking about async messaging before anything else, but then I read this article and it brings me back to the reality: http://martinfowler.com/articles/lmax.html

          • I don’t think Facebook or Twitter people were not building async architecture in a first place because this was their intention. Usually, when a small team starts a small campus project, the last thing they think about is how will they scale. Then, when they face the scalability issues, they start to change their architecture in a hurry and usually this does not turn well. They come to something working far later then they should have come. This is a faith of nearly any now-large-but-not-so-large-in-the-beginning system. People prefer to postpone decisions and sometimes this works well, sometimes it doesn’t. People also tend to have quite limited knowledge about how to make things and learn on the go as we all do. I listened to a Facebook talk on how they came to ReactJS and what they “invented” was some sort of event processing. When they were explaining their chat issues that they suffered for years and got loads of complains, I immediately thought about the solution. Later in the talk they explained that they did nearly the same that I was thinking about. But I won’t be able to get this solution a few years ago due to lack of knowledge and experience, neither did they. So, this is not a good argument to be honest. I believe that web systems considering their async and stateless nature can be event sourced. Also, in a well load balanced system the time that is required to project and denormalize could be made less that the UI update takes. And, to be honest, I do not buy this “it is not about CQRS but about event sourcing” and they you argue that stale data is evil, brings bad UX. Well, isn’t CQRS about stale data, about “splitting one object into two”, about having two models? This is the most essential part, it has literally nothing to do with the event sourcing. Event sourcing is just naturally comes to the surface as write model that we can comfortable work with, it plays nicely with DDD and gives us the full audit log.

          • jbogard

            I’m not developing Facebook. My users expect changes to show up immediately. If I were developing in a situation at Facebook scale, I’d modify my approach accordingly.

          • Still, CQRS means two models, meaning you will have some latency in the read models. So I don’t fully understand you statement you are pro CQRS but sort of contra event sourcing because it makes your visible data stale.

          • jbogard

            CQRS means two objects. Not two DB models. That was never a requirement for CQRS.

          • Yes, it was wrong definition from my side. Splitting one object into two. However, in Greg’s talks, designs with one storage is presented as not very clever. He always pushes the idea of two storage models and that stale data is not bad at all. There is always a fight around this topic in particular. Still, it needs to be seen case by case, there is no general approach there.

          • jbogard

            Yeah, it’s unfortunate – most of the descriptions of CQRS skip straight to 2 DB models. But that’s quite a bit more work (and quite a bit harder to do).

            Honestly I try to avoid clever these days :)

          • I posted a brownfield refactoring scenario as a reply to the root topic. I think, only with two (or more) models and event sourcing, one can move some old system like the one I told you about yesterday, from using logic in procedures and UI with database-table-oriented design, to DDD. To be honest, I do not see any other way to keep the existing database for quite some time and keep all inside it functioning, and take out some domain, model it, event source it, and populate the old db tables using projections. I completely see all the challenges with async vs sync, command handling result propagation, failure handling and so on. However, this approach will not have the “query that now” issue since all data will still be available in SQL although not in 1 ms but may be in 100 ms.

          • But indeed we all still to see some specific reference projects, based on “clever” CQRS and Greg’s ES.

          • gabrielschenker

            very good comments. Thank you

        • gabrielschenker

          I’m sorry for you Jimmy if your user hated the app. I have a completely different experience. Our users loved it. But that’s not the point here. You can use CQRS in an application that is fully synchronous all the time. By the way we should never forget that as soon as we build a web application everything is asynchronous!

          • jbogard

            Oh yes – I was referring to async read models. Users expect consistency. That’s what DBs are great at – providing transactional consistency and have spent man-centuries perfecting that model. Anything I build client-side to mimic it is a buggy, half-implemented version of a transaction. When I meet clients with async read models – in which the read model is in the same bounded context/app, I have them ditch it immediately and rip out all the plumbing they’ve put in to try and deal with it in the various layers of the app.

    • gabrielschenker

      In my first sample of a classical n-tier application using an RDBMS as back-end storage everything is synchronous. CQRS by no ways implies that we have to use event sourcing or an eventual consistent read model. The latter is just an option which scales out nicely.

  • Pingback: The Morning Brew - Chris Alcock » The Morning Brew #1835()

  • Mike Rowley

    In your opinion with regard to CQRS (no es) + DDD, would domain services and domain factories make use of the read model (i.e. IQuery) or would they use a IQueryRepository? I struggle with this because factories and services often need more than GetById and need to do querying but if they use IQuery then Dtos are exposed in the domain which doesn’t feel right.

    • gabrielschenker

      Good question. This is more of a DDD question and not so much about CQRS. But my rule of thumb is that the domain should never rely on an eventual consistent read model. Also most often if you need something other than GetById in the domain that “could” be an indication of a design flaw. Normally an aggregate encompasses all that is usually needed for processing. If that is not sufficient then maybe different instances of aggregates need to collaborate. But then again those instances are retrieved by their respective ID. I am sorry but the topic is way too complex to discuss it in a comment. I will try to author a post specifically about how I/we do DDD.

      • Andrew Siemer

        Gabriel – this is fun!

  • w1ngnut

    Hello Gabriel,
    I’ve been searching for approaches to handling eventual consistency in the front-end apart from “Thanks we are working on your request”. Any thoughts on this?

    Regards.

    • gabrielschenker

      Eventual consistency can only really work well in conjunction with a task based UI. First of all we want to make sure that each command that is sent to the back-end is very specific and if possible limited in what is changed. Secondly we should make sure that the command is highly likely to succeed. Then you shouldn’t reload data without need. If a command succeeds usually the corresponding view has already all necessary data displayed. Also eventual consistency means that in general the read model is in sync in a few milliseconds. That is nothing compared to the normal reaction time of a user. If the user navigates away from the current view to another view that needs the updated data then this happens in a matter of 100-eds of milliseconds.

      • w1ngnut

        Thanks for the reply! Yes, I’m very aware of those requirements. But my question is more about scenarios where the system is not capable of processing requests in less than a few seconds (let’s say 5s) for whatever reason (latency, performance, load etc).

        Any suggestions in dealing with this?

        • gabrielschenker

          I need to have a specific scenario to try to answer it. As always “it depends”. Can you give a concrete sample?

          • w1ngnut

            A very simple example could be: user submits information on (a task-based) screen A and navigates to screen B expecting to see that result in a grid. If that happened before the 5s mentioned above, he wouldn’t see his record and would be very frustrated. How to approach this? Thanks.

          • gabrielschenker

            As I said above, normally the eventual consistent read model is in sync with the write model in a matter of milliseconds. That is way more than the user needs to navigate to another screen. If for some reason the time is not sufficient and the read model is stale then one could use push notification technologies such as signal-r to push updates to the screen or the screen can poll for updates. If you are using auto save then you do not need to trigger a command only when the user clicks e.g. Next to navigate to the next screen and you will never run into the problem of out of sync…

          • w1ngnut

            Makes sense! Thanks for the input.

  • Blake H

    Excellent Post Gabriel. I’m a huge fan of CQRS/ES and I believe, if applied correctly, it solves a lot of issues that the standard relational CRUD model cannot. Chief among these issues is scalability, partitionability, elimination of transaction deadlocks from reading where you write, and providing fast targeted reads from an appropriate data store (graph, relational, document db) for different data sets, etc,etc.

    I think the biggest benefit for developers is that it keeps the code clean and maintainable… the biggest benefit for business is that it allows for scale without a complete rewrite, and it can provide insights on data that other methods just can’t.

    That being said, eventual consistency does require some new ways of thinking and is not without it’s own “baggage”. But this article brings it home for me: http://www.xaprb.com/blog/2014/12/08/eventual-consistency-simpler-than-mvcc/

    If you are looking to produce an MVP, then maybe CRUD is the way to go. If you are looking to build a system for a lot of users, and needs to survive for the long haul, I’d recommend you use a different pattern.

  • I found several handy uses of CQRS and event sourcing used together:

    1) System behaviour audit log. We have loads of situations when the data comes to invalid state and since the database is just frozen state, we cannot figure out how did it get there and it is hell to find and fix the reason so we are forced to deal with consequences. Users tend to forget what they did a second ago and if you ask them they will tell you something completely different from what actually happened. Event sourcing solves this issue completely. Even more, after you fix the problem, you can revert an aggregate to the last known valid state and replay all events to recover it to a valid state and update your projected read models accordingly.

    2) Adding additional read models. If you want to add some behaviour-based analytical model or just a new data storage to test and later use it in production, you will value stored events. It allows you not only to start putting data from now on, but put all historical data there as well. In no way this can be achieved by holding just a system state in any sort of database.

    3) Surprisingly, brownfield system refactoring. Usually you have a huge SQL database with level three normalization (or the other way around) and complex database logic to handle the data (procedures and lots of UI code). If you want to do DDD there, it is extremely complex because you are in big risk of modelling your domain to close to the data model since this is most obvious and developers all speak in database terms. However, this is not what you want to do, you want to have a proper model with aggregates, their entities and value objects and you don’t want to look at the existing data model when doing your new model. CQRS allows you to make this old database your read model and event sourcing gives you a new data model. Projections will put the domain behaviours to the old database data and old stuff will still work. In time, you can attach new storages as you go, replay all events with new projections and may be even get rid of the old database.

  • Dariusz Lenartowicz

    “aggregated data should not be calculated/aggregated on the fly when
    querying the data but should be pre-calculated. This can happen whenever
    data is changed that affects the aggregated values, which is rather
    rare compared to read operations, as stated above.”

    Yeah: http://stackoverflow.com/questions/6441644/dealing-with-utc-timezone-and-making-group-by-on-that-with-localtime