Lecture
The Command and Query Responsibility Segregation (CQRS) - a pattern that separates responsibility into commands and queries. It separates read and update operations for a data store. Implementing CQRS in an application can maximize its performance, scalability, and security. The flexibility introduced by moving to CQRS allows the system to evolve better over time and prevents update commands from causing merge conflicts at the domain level.
The CQRS pattern applies the imperative programming principle of command-query separation - command-query separation (CQS), using separate query and command objects to retrieve and modify data respectively . CQS was introduced by Bertrand Meyer during his work on the Eiffel programming language. The principle states that a method should be either a command that performs some action or a query that returns data, but not both at once. In other words, asking a question should not change the answer. More formally, a value may be returned only by a pure (i.e., deterministic and side-effect-free) method. It should be noted that strict adherence to this principle makes it impossible to track the number of query calls.
CQRS fits especially well with the design-by-contract methodology, which uses assertions embedded in the source code that describe the state of the program at certain important moments. In design by contract, assertions relate to design rather than execution logic, so their execution should have no effect on the program's state. CQRS is beneficial for design by contract, since any value-returning method (any query) can be called in assertions without worrying about a possible change in the program's state.
In theory, this makes it possible to learn the program's state without changing it. In practice, CQRS makes it possible to skip all assertion checks in a live system in order to improve its performance, without fear that this will change its behavior. CQRS also prevents certain heisenbugs from occurring.
Even beyond design by contract, its proponents see the application of CQRS as having a simplifying effect on a program, making access to its state (through queries) and modification of its state (through commands) more understandable, similar to how avoiding the use of goto simplifies understanding of a program's flow of execution.
CQRS is well suited to the object-oriented programming methodology, but it can also be applied outside of OOP, since separating side effects from returning values does not require OOP. Therefore, CQRS can be usefully applied in any programming paradigm that requires care about side effects.
CQS can complicate the creation of reentrant and multithreaded software. This problem usually arises when using a non-thread-safe pattern to implement CQS.
A simple example of a pattern that violates CQS but is useful in multithreaded software:
private int x;
public int increment_and_return_x()
{
lock x; // some locking mechanism
x = x + 1;
int x_copy = x;
unlock x; // some locking mechanism
return x_copy;
}
A common CQS pattern applicable only in single-threaded applications:
private int x;
public int value()
{
return x;
}
void increment_x()
{
x = x + 1;
}
Even in the case of single-threaded programs, one can sometimes argue that it is significantly more convenient to have a method that combines a query and a command. Martin Fowler cites the stack pop() method as an example.
I was told right away that the project I would be working on uses CQRS, Event Sourcing, and MongoDB as the database. Out of all of these, I had only heard of MongoDB. When I tried to get into CQRS, I did not immediately grasp all the subtleties of this approach, but for some reason I liked the idea of splitting the data-interaction model into two — read and write. Perhaps because it somehow echoed the “separation of concerns” programming paradigm, perhaps because it was very much in the spirit of DDD.
Many people talk about CQRS as a design pattern. In my opinion, it affects the overall architecture of an application too much to be called just a “design pattern,” so I prefer to call it a principle or an approach. The use of CQRS permeates almost every corner of an application.
I want to clarify right away that I have only worked with the combination of CQRS + Event Sourcing, and I have never tried plain CQRS, because it seems to me that without Event Sourcing it loses a great many of its benefits. As the CQRS framework I will use our corporate Paralect.Domain. In some ways it is better than others, in some ways worse. In any case, I recommend that you familiarize yourself with the others as well. I will mention here only a few frameworks for .NET. The most popular are NCQRS, Lokad CQRS, SimpleCQRS. You can also take a look at Jonathan Oliver's Event Store, which supports a huge number of different databases.
So what is CQRS?
CQRS stands for Command Query Responsibility Segregation. It is a design pattern that I first heard about from Greg Young. At its core lies the simple notion that you can use different models to update and read information. However, this simple notion leads to serious consequences in the design of information systems. (c) Martin Fowler
Not exactly an exhaustive definition, but I will now try to explain what exactly Fowler had in mind.
By now, the situation has developed such that practically everyone works with a data model as with a CRUD store. CQRS offers an alternative approach, but it affects more than just the data model. If you use CQRS, it strongly impacts the architecture of your application.
Here is how I depicted the diagram of how CQRS works
The first thing that catches your eye is that you already have two data models, one for reading (Queries) and one for writing (Commands). And this usually means that you also have two databases. And since we use CQRS + Event Sourcing, the write database (write model) — is the Event Store, something like a log of all user actions (actually not all of them, but only those that are important from the business model's point of view and affect the construction of the read database). And the read database — is, in the general case, a denormalized store of the data that you need to display to the user. Why did I say that the read database is denormalized? You can of course use any data structure as the read model, but I believe that when using CQRS + Event Sourcing you should not fuss too much over normalizing the read database, since it can be fully rebuilt at any time. And this is a big plus, especially if you do not want to use relational databases and are looking toward NoSQL.
The write database in general is a single collection of events. That is, here too there is no point in using a relational database.
The idea of Event Sourcing is to record every event that changes the application's state into the database. This way, it turns out that we store not the state of our entities, but all the events that relate to them. However, we are used to manipulating exactly the state; it is stored in our database and we can always look at it.
In the case of Event Sourcing, we also operate with the state of an entity. But unlike the ordinary model, we do not store that state; instead, we reproduce it each time it is accessed.
If you look at the code that loads an aggregate from the database, you might not even notice any difference from the traditional approach.
var user = Repository.Get(userId);
In reality, though, the repository does not take the ready-made state of the UserAR aggregate (AR = Aggregate Root) from the database; it selects from the database all the events associated with this user, and then reproduces them in order, passing them into the aggregate's On() method.
For example, the UserAR aggregate class should have the following method in order to restore the user's ID in the user's state
protected void On(User_CreatedEvent created)
{
_id = created.UserId;
}
Out of the aggregate's entire state, I only need the user's _id; I could likewise restore the state of the password, name, etc. However, these fields can be modified by other events as well, not only by User_CreatedEvent, so I will need to handle them all. Since all events are reproduced in order, I am confident that I am always working with the latest current state of the aggregate, provided of course that I have written On() handlers for all the events that change this state.
Let's look at the example of creating a user to see how CQRS + Event Sourcing works.
The first thing I will do — is compose and send a command. For simplicity, let the user-creation command have only the most essential set of fields and look as follows.
public class User_CreateCommand: Command
{
public string UserId { get; set; }
public string Password { get; set; }
public string Email { get; set; }
}
Don't let the command class name confuse you; it does not conform to generally accepted guidelines, but it lets you immediately understand which aggregate it relates to and what action is being performed.
var command = new User_CreateCommand
{
UserId = “1”,
Password = “password”,
Email = “test@test.com”,
};
command.Metadata.UserId = command.UserId;
_commandService.Send(command);
Then I need a handler for this command. The command handler must be passed the ID of the required aggregate; by this ID it will get the aggregate from the repository. The repository builds the aggregate object as follows: it takes from the database all the events that relate to this aggregate, creates a new empty aggregate object, and reproduces the retrieved events in order on the aggregate object.
But since we have a creation command — there is nothing to load from the database, which means we create the aggregate ourselves and pass it the command metadata.
public class User_CreateCommandHandler: CommandHandler
{
public override void Handle(User_CreateCommand message)
{
var ar = new UserAR(message.UserId, message.Email, message.Password, message.Metadata);
Repository.Save(ar);
}
}
Let's look at what the aggregate's constructor looks like.
public UserAR(string userId, string email, string password, ICommandMetadata metadata): this()
{
_id = userId;
SetCommandMetadata(metadata);
Apply(new User_CreatedEvent
{
UserId = userId,
Password = password,
Email = email
});
}
The aggregate must also have a parameterless constructor, since when the repository reproduces the aggregate's state, it first needs to create an empty instance and then pass events into the projection methods (the method On(User_CreatedEvent created) is one of the projection methods).
Let me clarify a bit about projection. A projection — is the reproduction of an aggregate's state based on events from the Event Store that relate to that aggregate. In the user example — these are all the events for that specific user. In the aggregate, all those same events that are saved via the Apply method can be handled during the reproduction of its state. In our framework, to do this it is enough to write a method On(/*EventType arg*/), where EventType — is the type of event you want to handle.
The aggregate's Apply method triggers sending the events to all handlers. In fact, the events are sent only when the aggregate is saved to the repository, while Apply simply adds them to the aggregate's internal list.
Here is the handler for the user-creation event(!), which writes the user itself into the read database.
public void Handle(User_CreatedEvent message)
{
var doc = new UserDocument
{
Id = message.UserId,
Email = message.Email,
Password = message.Password
};
_users.Save(doc);
}
An event can have several handlers. Such an architecture helps maintain data integrity if your data is heavily denormalized. Suppose I need to frequently display the total number of users. But I have too many of them, and a count operation over all of them is very costly for my database. Then I can write another event handler, which will already relate to statistics and every time a user is added will increment the total user counter by 1. And I will be confident that no one will create a user without also updating the counter. If I did not use CQRS, and I had an ordinary ORM instead, I would have to watch in every place where a user is added or removed to make sure the counter is updated too.
And the use of Event Sourcing gives me additional advantages. Even if I made a mistake in some EventHandler or did not handle an event everywhere I needed to, I can easily fix this simply by regenerating the read database with the now-correct business logic.
Creation is clear. How does modifying an aggregate and validating a command happen? Let's look at the example of changing a password.
I will provide only the command handler and the aggregate's ChangePassword() method, since in the other places the difference is generally not great.
public class User_ChangePasswordCommandHandler: IMessageHandler
{
// Constructor omitted
public void Handle(User_ChangePasswordCommand message)
{
// take the aggregate from the repository
var user = _repository.GetById(message.UserId);
// set the metadata
user.SetCommandMetadata(message.Metadata);
// change the password
user.ChangePassword(message.OldPassword, message.NewPassword);
// save the aggregate
_repository.Save(user);
}
}
public class UserAR : BaseAR
{
//...
public void ChangePassword(string oldPassword, string newPassword)
{
// If the password does not match the old one, throw an error
if (_password != oldPassword)
{
throw new AuthenticationException();
}
// If everything is OK - accept the event
Apply(new User_Password_ChangedEvent
{
UserId = _id,
NewPassword = newPassword,
OldPassword = oldPassword
});
}
// Projection method for restoring the password state
protected void On(User_Password_ChangedEvent passwordChanged)
{
_password = passwordChanged.NewPassword;
}
// We also add restoring the password state on the user-creation event
protected void On(User_CreatedEvent created)
{
_id = created.UserId;
_password = created.Password;
}
}
}
I want to note that it is highly undesirable for an invalid event to be passed into the Apply() method. Of course you will be able to handle it later in an event handler, but it is better not to save it at all if it is not important to you, since this will only clutter the Event Store.
In the case of a password change, there is no point at all in saving this event, unless of course you are collecting statistics on failed password changes. And even in that case you should think carefully about whether you need this event in the write model or whether it makes sense to record it in some temporary store. If you assume that the business logic of the event validation may change, then save it.
That is basically all I wanted to talk about in this article. Of course it does not cover all the aspects and capabilities of CQRS + Event Sourcing; I plan to talk about that in future articles. Also left out of frame are the problems that arise when using this approach. And we will talk about that too.
If you have any questions, ask them in the comments. I will gladly answer them. Also, if you have any suggestions for future articles — I would very much like to hear them.
A fully working example on ASP.NET MVC can be found here.
There is no database there; everything is stored in memory. If desired, it is simple enough to hook one up. There is also an out-of-the-box implementation of Event Store on MongoDB for storing events.
To hook it up, it is enough to replace InMemoryTransitionRepository with MongoTransitionRepository in the Global.asax file.
As the read model I have a static collection, so with each restart the data is destroyed.
I have several ideas for articles on this topic. Suggest more. Tell me what is most interesting.
Recently, event-driven architectures and, in particular, Event Sourcing have been gaining traction. This is driven by the desire to create resilient and scalable modular systems. In this context, the term “microservices” is used quite often. In my opinion, microservices — are just one way of implementing a “Bounded Context.” It is very important to correctly define the boundaries of modules, and Strategic Design, described by Eric Evans in Domain Driven Design, helps with this. It helps you identify / discover modules, boundaries (“bounded context”) and describe how these contexts relate to each other (a context map, ContextMap).
Although this is not explicitly stated in Eric Evans's book, domain events greatly support DDD concepts. Practices such as Alberto Brandolini's Event Storming shift the emphasis of events from the technical to the organizational and business level. Here we are not talking about user-interface events such as a button click (ButtonClickedEvent), but about domain events that are part of the domain. Domain experts talk about them and understand them. These events represent first-class concepts and help form a ubiquitous language that everyone (domain experts, developers, etc.) will agree on.
Domain events can be used for interaction between bounded contexts. Suppose we have an online store with three contexts: Order, Delivery, Invoice.
Consider the “Order accepted” event in the Order context. The Invoice context, as well as the Delivery context, are interested in tracking this event, since this event triggers certain internal processes in these contexts.
Using domain events helps develop loosely coupled modules. Individual modules may be temporarily unavailable. But to a domain event it does not matter at all whether they are available or not, since an event only describes what happened in the past. Other modules decide for themselves when to handle the event. By default, you end up with a flexible system.
In addition to decoupling in time, domain events give you one more advantage: the order context does not need to know that the invoice and delivery contexts are listening to its events. In fact, it does not even need to know that these contexts exist.
That's great, but the difficulty lies in deciding what data to store in the event.
Events are useful, so why not use them to the fullest. This is the core idea of Event Sourcing. You store the aggregate's state not by updating its data (CRUD), but by applying a stream of events.
In addition to being able to reproduce the events and obtain the state, Event Sourcing has another feature: you get a complete audit log for free. Therefore, when such a log is required, be sure to consider Event Sourcing when choosing a storage strategy.
The core idea of event sourcing is that you should store the state of the system as an ordered, immutable log of events that occur in the system, instead of storing it as snapshots of object states.
6. From state snapshots to events
When the system's state is stored as state snapshots — an object's state in the system is represented by a mutable row or rows in the database. When updating an object's state, we overwrite the row with new values.
When event sourcing is applied — the state is stored as a sequence of immutable events. When updating an object's state, you need to save the new events without changing the old ones.
An event in event sourcing is a set of data describing some fact of the real world; once an event has already occurred, then, just as in the real world, it can no longer change.
In code it is some immutable object,
It may seem strange to you that I jumped straight from domain events to storage, since it is obvious that these are concepts of different levels.
… and here is my point of view: Event Sourcing — is a local solution used only within a single bounded context! Event Sourcing events should not be exposed outward to the external world! Other bounded contexts should not know about each other's ways of storing data, and therefore it does not matter to them whether some context uses Event Sourcing.
If you use Event Sourcing globally, then you expose your storage layer.
The way data is stored becomes your public API. Every time you make changes to the storage layer, you will have to deal with a change to the public API.
I am sure everyone will agree that it is bad when different bounded contexts jointly use data in a (relational) database because of the coupling that arises. But how is this different from Event Sourcing? It is not. It does not matter whether you share common events or common tables in a database. In both cases you are sharing storage details.
I still maintain that domain events are ideally suited for interaction between bounded contexts, but these events should not be tied to the events that are used for Event Sourcing.
The proposed solution is very simple: regardless of which approach you use for storing data (CRUD or Event Sourcing), you publish domain events to a global event store. These events represent the public API of your context. When using Event Sourcing, you store the Event Sourcing events in your local store, accessible only to that bounded context.
Having separate domain events in the public API lets you model them flexibly. You are not limited to a model that is predefined by the Event Sourcing events.
There are two options for working with ”real-world events”: an Open Host Service with a Published Language, or Customer / Supplier.
Only a single domain event is published, containing all the data that other bounded contexts might need. In DDD terminology, this can be called an Open Host Service with a Published Language.
The occurrence of the real-world event “Order accepted” leads to a single domain event, OrderAccepted, being published. The payload of this event contains all the order data that other bounded contexts might need… so, hopefully, the Invoice and Delivery contexts will find all the information they need.
Separate events are published for each consumer. It is necessary to agree on the model of each event with only one consumer; there is no need to define a common shared model. DDD calls these relationships Customer / Supplier.
The occurrence of the real-world event “Order accepted” leads to the publication of separate events for each of the consumers: InvoiceOrderAccepted and DeliveryOrderAccepted. Each domain event contains only the data that the receiving context needs.
I do not want to discuss the pros and cons of these approaches right now. I just want to draw attention to the fact that you can choose the number of domain events and the data to store in them.
This is an advantage that you should not underestimate, because you can decide how to evolve the API of your bounded context without being tied to the Event Sourcing events.
Exposing storage details outward — is a well-known anti-pattern. When we talk about storage, we first think of database tables, but we have seen that the events used for Event Sourcing are just another way of storing data. Therefore, exposing them outward is also an anti-pattern.
Translation: “a good developer is like a werewolf — afraid of silver bullets.”
Event Sourcing — is a powerful approach if used correctly (locally). At first glance it seems to be a silver bullet for event-driven architectures, but if you look more closely, you can see that this approach can lead to strong coupling … which, of course, you do not want.
Comments