A better way to handle authorization in ASP.NET Core

Last Updated: December 16, 2018 | Created: December 14, 2018

I was asked by one of my clients to help build a fairly large web application, and their authentication (i.e. checking who is logging in) and authorization (i.e. what pages/feature the logged in user can access) is very complex. From my experience a knew that using ASP.NET’s Role-based approach wouldn’t cut it, and I found the new ASP.NET Core policy-based approach really clever but it needed me to write lots of (boring) policies.

In the end I created a solution for my client and this article describes the authorization part – I call it Roles-to-Permissions (the name will make more sense as you read the article). I have also build an example ASP.NET Core application, with all new code to support this article. This example application is quite different from my client’s system as I tap into ASP.NET Core built-in Identity system (the client’s system needed OAuth2). The example application contains about 60 lines that I copied (with my client’s permission) from the original implementation to create an open-source version (MIT licence) you can use.

NOTE: you can Clone the GitHub repo and run locally – it uses in-memory databases so it will run anywhere. The application was written using ASP.NET Core 2.1: parts of the ASP.NET Identity is changing, but the overall concept will work with any version of ASP.NET Core.

This is a long article, so here are links to the major parts:

TL;DR; – summary

  • NET Role-based authorization system works for systems with simple authorization rules, but it has limitations, like the fact that you have to republish your code if you change the authorization rules.
  • Roles, with names like “Manager” or “ExternalBuyer” makes sense for users (human or external services) as they define a “Use Case” of what the user should be able to do.
  • But Roles don’t work well when applied to ASP.NET Core actions or Razor Pages. Here you need a much more fine-grained solution, with names like “CanRequestHoliday”, “CanApproveHoliday” – I call these Permissions.
  • The solution is to map the user’s Roles to a group of Permissions and store these in the User’s Claims.
  • Then I use ASP.NET Core’s new policy-based authorization system to check that the User’s Permissions Claims contains the Permission placed on the action/page they want to access.
  • There is an open-source example ASP.NET Core application to go with this article.

Setting the Scene – a look at different application security needs

If you understand ASP.NET’s authorization and authentication features then you can skip this section.

There are billions of web applications and the control of what you can do ranges for “anyone can do anything”, e.g. Google search, up to some military systems where access needs keys, biometrics etc. When you need to prove you are a valid user of the system, say by logging in, that is referred to as authentication. Once you are logged in then what you can do is controlled by what is called authorization.

Authorization breaks down into two parts:

  1. What data can I access? For instance, you can see your personal information, but not other people’s personal information.
  2. What features you can use? For instance, are you allowed to change the title of a book that you can see?

NOTE: This article only describes a way to manage the second part, what features can you use.

ASP.NET MVC and now ASP.NET Core have various systems to help with authorization and authentication. Some systems only need a simple authorization – I could imagine a very simple e-commerce system could get away with: a) No logged in – browsing, b) Logged in – buying, and c) Admin – Add/Remove items for sale. This could be done using ASP.NET Role-based authentication.

But many business-to-business (B-to-B) systems have more complex authorization needs. For instance, think of a human resources (HR) system where people can request holiday leave and their manager has to approve those requests – there is lot going on inside to ensure only the correct users can use these features.

Systems like my example HR B-to-B system often end up with lots of complex authorization rules. My experience is that the ASP.NET Role-based authentication starts to have problems implementing this type of system, which is why I created the Roles-to-Permissions code.

Another type of application that could benefit from the Roles-to-Permissions approach are subscription systems, where the features a user can access depend on what subscription they paid for. The Roles-to-Permissions approach can control the features that as user can access based on the subscription they bought.

Role authorization: what is it and what are its limitations?

Roles authorization has been around for years in the ASP.NET MVC application, and I have used it in a number of applications. Here is an example of a ASP.NET Core controller that can only be accessed by logged in users with either the “Staff” Role or the “Manger” role.

[Authorize(Roles = "Staff,Manager")]
public ActionResult Index()
{
    return View(MyData);
}

This works for applications that have fairly simple and well-defined Roles, like User/Admin or Staff/Manager/Admin, then Roles is a good choice. But here are some of the problems I have found:

  1. If you have lots of roles you can end up with long Authorize attributes, e.g. [Authorize(Roles = “Staff,HrManager,BizManager,DevManage,Admin,SuperAdmin”)].
  2. Because Authorize is an attribute then the string has to be a constant, e.g. you can’t have $”{RoleConstants.StaffRole}”. This means if things change you are editing strings, and you could misspell something really easily.
  3. The big one for me is your authorization rules are hard-coded into your code. So, if you want to change who can access a certain action you have to edit the appropriate Authorize attributes and redeploy your application.

My experience from previous applications using Roles-based authorization is me constantly having to go back and edit the authorize Roles part as I develop or refine the application. I have been looking for a better way for some time, and my client’s requirements spurred me on to find something better than Roles authorization.

The architecture of the Roles-to-Permissions system

1. Introducing Roles, Permissions and Modules

It turns out that there is nothing wrong with the idea of a user having Roles. A user (human or an external service) can typically can be described by their function or department, like “Developer” or “HR”, maybe with side Roles like “HolidayAdmin”. Think of Roles as “Use Cases for users.

NOTE: In the example application I have Roles of “Staff”, “Manager”, and “Admin.

But the problem is that Roles aren’t a good fit for the actions in the Controllers. Each Controller action has a little part to play in a Role, or to turn it around, a Role is made up of a series of Controller actions that the Role allows you to access.  I decided I would call the authorization feature on each action a “Permission”, and I used an Enum to define them. A permission Enum member might be called “CanReadHoliday”, “CanRequestHoliday”, “CanApproveHoliday”, etc.

NOTE: In the example application I have Permissions on my ColorController of “ColorRead”, “ColorCreate”, “ColorUpdate”, and “ColorDelete”.

Now that we have permissions we can provide another feature that of controls access to optional features, e.g. features that a user only has based on their subscription to the service. There are many ways of handling features but by combining optional features into the permissions makes it simpler to setup and control.

NOTE:  In the example application I have Permissions called “Feature1” and “Feature2” which are mapped to Modules with the same name.

2. How this is implemented

Having defined my terms, I’m going to give you an overview of the process. It consists of two parts: the login stage and the normal accesses to the web site. The login stage is the most complex with lots of magic goes on in the background. Its basic job is to convert the user’s Roles into Permissions and add it to the User’s information.

I have set up my example application to store the user’s claims in a cookie which is read in with every HTTP request and turned into a ClaimsPrincipal, which can be accessed in ASP.NET Core by the HttpContext property called “User”.

Here is a diagram of that login stage. It might not make a lot of sense yet, but I describe each part in the rest of the article. This diagram is to give you an overview.

The second part is simpler and covers what happens every time the logged-in user accesses a protected Controller action. Basically, I have a policy-based authorization with dynamic rules that checks the current User has the permission needed to execute the ASP.NET action/razor page.

NOTE: Don’t forget there is example application if you want to look at the actual code.

Now I’m going to build up the Roles-to-Permissions in stages and explain what each part does.

Why I used Enums for the Permissions

One of the down-sides of using Roles is it used strings, and I’m a little bit dyslexic. That means I can type/spell things incorrectly and not notice. Therefore, I wanted something where intellisence would prompt me and if I still typed it incorrectly it would be a compile error. But it turns out there are a couple of other reasons that make using an Enum for the permissions a good idea. Let me explain.

In a big application there could be hundreds of Permissions. This lead to two problems:

  1. If I use Cookie Authorization there is a maximum size of 4096 bytes for the Cookie. If I had hundreds of long strings I might start to fill up the Cookie, and I want some room for other things like my data authorization. If I can store the Enums permissions as a series of integers it’s going to be much smaller than a series of strings.
  2. Secondly, I want to help the Admin person who needs to build the mapping from Roles to permissions. If they need to scroll through hundreds of permission names it could be hard to work out which ones are needed. It turns out Enum members can have attributes, so I can add extra information to help the Admin person.

So, here is part of my Permissions Enum code

public enum Permissions
{
    //Here is an example of very detailed control over something
    [Display(GroupName = "Color", Name = "Read", Description = "Can read colors")]
    ColorRead = 0x10,
    [Display(GroupName = "Color", Name = "Create", Description = "Can create a color entry")]
    ColorCreate = 0x11,
    [Display(GroupName = "Color", Name = "Update", Description = "Can update a color entry")]
    ColorUpdate = 0x12,
    [Display(GroupName = "Color", Name = "Delete", Description = "Can delete a color entry")]
    ColorDelete = 0x13,

    [Display(GroupName = "UserAdmin", Name = "Read users", Description = "Can list User")]
    UserRead = 0x20,
    //This is an example of grouping multiple actions under one permission
    [Display(GroupName = "UserAdmin", Name = "Alter user", Description = "Can do anything to the User")]
    UserChange = 0x21,

    [Obsolete]
    [Display(GroupName = "Old", Name = "Not used", Description = "example of old permission"
    OldPermissionNotUsed = 0x40,

The things to note are:

  • I show two types of permissions.
    • First four (lines 4 to 1) are fine-grained permissions, almost one per action.
    • Next two (lines 13 to 17) are more generic, e.g. I have a specific “UserRead”, but then one permission called “UserChange” which allows create, update, delete, lock, change password etc.
  • Line 5, 7, etc. Notice that I give each enum a specific number. If you are operating a 24/7 application with a new version seamlessly replacing the old version, then the Permission numbed mustn’t change otherwise user’s Claims be wrong. That is why I give each enum a specific number.
  • Line 19. I also support the Obsolete attribute, which stops the Permission appearing in the listing. There are plenty of scary stories about reusing a number with unintended consequences. (Also, it you try to use something marked as Obsolete you get a warning).
  • Line 4 etc. I add a Display Attribute to each Permission Enum. This has useful information that I can show lots of useful information to help the person who is building a Role.
  • Line 4, 6, 8, 10. I “Group” permissions that are used in the same place. This makes it easier for the Admin person to find the things they want. I also number in Hex, which gives me 16 possible permissions in a Group (I tried 10 and you could go over that, so 16 is better).

Here is a list of some of the Permissions in my example application listed via the Users->List Permissions nav dropdown.

And the code that produced that output (link to PermissionDisplay class for the whole thing)

public static List<PermissionDisplay> GetPermissionsToDisplay(Type enumType) 
{
    var result = new List<PermissionDisplay>();
    foreach (var permissionName in Enum.GetNames(enumType))
    {
        var member = enumType.GetMember(permissionName);
        //This allows you to obsolete a permission and it won't be shown as a
        //possible option, but is still there so you won't reuse the number
        var obsoleteAttribute = member[0].GetCustomAttribute<ObsoleteAttribute>();
        if (obsoleteAttribute != null)
            continue;
        //If there is no DisplayAttribute then the Enum is not used
        var displayAttribute = member[0].GetCustomAttribute<DisplayAttribute>();
        if (displayAttribute == null)
            continue;

        //Gets the optional PaidForModule that a permission can be linked to
        var moduleAttribute = member[0].GetCustomAttribute<PermissionLinkedToModuleAttribute>();

        var permission = (Permissions)Enum.Parse(enumType, permissionName, false);

        result.Add(new PermissionDisplay(displayAttribute.GroupName, displayAttribute.Name, 
                displayAttribute.Description, permission, moduleAttribute?.PaidForModule.ToString()));
    }

    return result;
}

How to handle optional/paid-for features?

My client provides a Business-to-Business application and plans to add new features that customers can subscribe to. One way to handle this would be create different Roles, like “Manager”, “ManagerWithFeature1”, “ManagerWithFeature2” or add separate Feature Roles that you have to manually apply to a user. That works but is pretty horrible to manage, and human error could cause problems. My preferred system is mark Permissions linked to a paid-for feature filter them based on the User’s subscriptions.

Marking Permissions as linked to a module is easy to do with the Enums – I just add another attribute. Here an example of Permissions linked to a Module (see line 5).

public enum Permissions
{
    //… other Permissions removed for clarity

    [LinkedToModule(PaidForModules.Feature1)]
    [Display(GroupName = "Features", Name = "Feature1", Description = "Can access feature1")]
    Feature1Access = 0x30,
    [LinkedToModule(PaidForModules.Feature2)]
    [Display(GroupName = "Features", Name = "Feature2", Description = "Can access feature2")]
    Feature2Access = 0x31
}

The paid-for modules are again represented by an Enum, but one marked the [Flags] attribute because a user can have multiple modules that they have subscribed to. Here is my PaidForModules Enum code

[Flags]
public enum PaidForModules : long
{
    None = 0,
    Feature1 = 1,
    Feature2 = 2,
    Feature3 = 4
} 

NOTE I add “: long” to the Enum which gives me up to 64 different modules in my system.

What happens is that Permissions linked to a Module that the user hasn’t subscribe to are filtered out during the login stage (I show how later). This makes the setting up the Roles much simpler, as you build each Role with all the Permissions that make sense for that role, including features mapped to a paid-for module. Then, at login time, the system will remove any Permissions the current user doesn’t have access to. That is simpler for the Admin person and more secure for the application.

How do I turn the Roles into a Permissions Claim?

I my client’s system we uses 0Auth2 authentication, but for this example I used ASP.NET Core IdentityRole to hold the Roles that a user has. That means I can use all of the ASP.NET Core built-in Identity code to set up the Users and Roles. But how do I convert the User’s Roles to a Permissions Claim?

Again there are few ways to do it, but in the end I tapped into an event in the Authorization Cookie called ‘OnValidatePrincipal’ (here is a link to the lines in the example application startup class). This calls the code below, but be warned it’s pretty complex so here is a summary of the steps it goes through:

  1. If the Claims already have the Permissions claim type then nothing to do so return quickly.
  2. Then we get the Roles the user has from the Role Claim
  3. I need to access my part of the database. I can’t use dependency injection, so I use the extraAuthDbContextOptions, which is a singleton that I can provide at startup.
  4. Then I get all the permissions for all of the roles, with a Distinct to remove unnecessary duplicates.
  5. Then I filter out any permissions that are linked to a Module that the user doesn’t have access to.
  6. Then I add a permissions Claim containing all the Permissions the user is allowed, but packed as hex numbers in a single string so that it doesn’t take up so much room (I used Hex format as it made debugging easier).
  7. Finally I have to create a new ClaimsPrincipal and tell ASP.NET Core to replace the current ClaimsPrincipal, plus set the all-important ShouldRenew to true, which updates the Cookie, otherwise this complex (slow) method on every HTTP request!
public async Task ValidateAsync(CookieValidatePrincipalContext context)
{
    if (context.Principal.Claims.Any(x => 
        x.Type == PermissionConstants.PackedPermissionClaimType))
        return;

    //No permissions in the claims so we need to add it
    //This is only happens once after the user has logged in
    var claims = new List<Claim>();
    foreach (var claim in context.Principal.Claims)
    {
        claims.Add(claim);
    }

    var usersRoles = context.Principal.Claims
        .Where(x => x.Type == ClaimTypes.Role)
        .Select(x => x.Value)
        .ToList();
    //I can't inject the DbContext here because that is dynamic, 
    //but I can pass in the database options because that is a 
    //From that I can create a valid dbContext to access the database
    using (var dbContext = new ExtraAuthorizeDbContext(_extraAuthDbContextOptions))
    {
        //This gets all the permissions, with a distinct to remove duplicates
        var permissionsForUser = await dbContext.RolesToPermissions
            .Where(x => usersRoles.Contains(x.RoleName))
            .SelectMany(x => x.PermissionsInRole)
            .Distinct()
            .ToListAsync();

        //we get the modules this user is allows to see
        var userModules =
            dbContext.ModulesForUsers
                .Find(context.Principal.Claims
                     .SingleOrDefault(x => x.Type == ClaimTypes.Name).Value)
                ?.AllowedPaidForModules ?? PaidForModules.None;
        //Now we remove permissions that are linked to modules that the user has no access to
        var filteredPermissions =
            from permission in permissionsForUser
            let moduleAttr = typeof(Permissions).GetMember(permission.ToString())[0]
                .GetCustomAttribute<LinkedToModuleAttribute>()
            where moduleAttr == null || userModules.HasFlag(moduleAttr.PaidForModule)
            select permission;

          //Now add it to the claim
          claims.Add(new Claim(PermissionConstants.PackedPermissionClaimType,
              filteredPermissions.PackPermissionsIntoString()));    }

    var identity = new ClaimsIdentity(claims, "Cookie");
    var newPrincipal = new ClaimsPrincipal(identity);

    context.ReplacePrincipal(newPrincipal);
    context.ShouldRenew = true;
}

How do I convert my Permissions into Policy-based authorization?

OK, I now have access to the Permissions via the User’ Claims, but how do I get this turned into something that ASP.NET Core can use for authorization. This is where a .NET developer and friend, Jerrie Pelser helped me.

When I started this project, I emailed Jerrie Pelser, who runs the ASP.NET Weekly newsletter (great newsletter! Do sign up) as I know Jerrie is an expert in authentication & authorization.  He pointed me at a few architectural things and I also found his own article “Creating Authorization Policies dynamically with ASP.NET Core” really helpful.  Jerris’s article showed me how to build policies dynamically, which is exactly what I need.

I’m not going to repeat Jerrie article here (use the link above), but I will show you my PermissionHandler that is used inside the policy to check that the current User’s Permissions claim exists and contains the Permission on the action/Razor Page. It uses an extension method called ThisPermissionIsAllowed which does the check.

public class PermissionHandler : 
    AuthorizationHandler<PermissionRequirement>
{
    protected override Task HandleRequirementAsync(
        AuthorizationHandlerContext context, 
        PermissionRequirement requirement)
    {
        var permissionsClaim = context.User.Claims
            .SingleOrDefault(c => 
                 c.Type == PermissionConstants
                     .PackedPermissionClaimType);
        // If user does not have the scope claim, get out of here
        if (permissionsClaim == null)
            return Task.CompletedTask;

        if (permissionsClaim.Value
            .ThisPermissionIsAllowed(
                 requirement.PermissionName))
        {
            context.Succeed(requirement);
        }

        return Task.CompletedTask;
    }
}

There are two other classes that are involved in making dynamic policy-based authorisation work. Here are the links to them:

Policies are defined by strings, but as I said I hate strings as I can make a mistake. I therefore created this very simple HasPermission attribute which allows me to apply an Authorize attribute, but using a Permissions Enum

[AttributeUsage(AttributeTargets.Method 
    | AttributeTargets.Class, Inherited = false)]
public class HasPermissionAttribute : AuthorizeAttribute
{
    public HasPermissionAttribute(Permissions permission) 
       : base(permission.ToString()) { }
}

That’s pretty simple, but it means I get intellisence when I am adding the Permission.

Putting it all together

So, we have the Permissions in the code and we can apply them using our HasPermissionAttribute to each action we want to protect via authorization. Here is one action taken from the ColorController in my example application.

[HasPermission(Permissions.ColorRead)]
public ActionResult Index()
{
    return View(MyData);
}

We also need to add two tables to whatever database your application uses. The two EF Core entity classes are:

Once the application is up and running an Admin-type user has to:

  1. Create some roles, e.g. “Staff”, “Manager”, etc. using ASP.NET Core Identity code.
  2. Create matching RoleToPermissions for each of the roles, specifying what Permissions map to each Role.

Then, for every new user an Admin person (or some automatic subscription code) has to:

  1. Create the user (if an invite-only type application)
  2. Add the correct Roles and ModulesForUser for that new user.

Once that is done all the code I have shown you takes over. The user logs in, gets the Permissions and what they can access is managed by ASP.NET Core’s policy-based authentication.

Things I didn’t cover elsewhere

There are a few things I didn’t cover in detail, but here are links to the items:

  • The important parts about registering things are shown in highlighted lines in this link to the Startup class. (NOTE: I built the application with ASP.NET Core 2.1, but I know the Identity parts are changing in 2.2, so you might have to update the code I put in the Startup class for newer versions of ASP.NET Core).
  • You don’t need to use ASP.NET Core Identity system at all – I said the client’s version uses an external authentication system. You just have to create a Roles-to-User table so you can assign Roles to each user.
  • I didn’t cover how I Packed/Unpacked the Permissions. You can find the Extension methods for doing that in PermissionPackers.
  • You might want to check for a permission in your razor pages to show/hide links. I created a simple method in the PermissionsExtension class, and used it in the _layout.cshtml Razor page.

Conclusion

Well that is a long article so well done by getting to the end. I have described an authentication I have built that handles complex authentication rules while being (relatively) easy to understand and manage via the Admin staff. Sure, if you have hundreds of Permissions it’s not to be hard setting up the initial RolesToPermissions, but the Admin has a lot of information to help them.

For me the Roles-to-Permissions approach solves a lot of problems I had in older systems I built using ASP.NET MVC Roles. I have to write some more code, but it makes it a) easier to change the authorization rules and b) helps the Admin person manage applications with lots of Roles/Permissions. I hope it helps you think of better ways of building better authentication systems for your projects.

Further reading

Happy coding.

Don’t forget to sign up to Jerrie Pelser’s, ASP.NET Weekly newsletter if you are interested in ASP.NET or Entity Framework. It is the most important newsletter I get.

Building high performance database queries using Entity Framework Core and AutoMapper

Last Updated: December 7, 2018 | Created: November 30, 2018

When you are writing Entity Framework Core (EF Core) queries (i.e. reading data from the database) you want them to be quick to run (high performance) and quick to write. It turns out that it’s possible to do that in EF, but the “quick to run” part isn’t that obvious. I make a living around writing EF Core code quickly, and that performs well (I wrote the book, “Entity Framework Core in Action”) and my approach is to use the LINQ Select method (quick to run) and the AutoMapper library (quick to write).

This article is about both “quick to run” and “quick to write”, but majors on AutoMapper, as the Select method is easy to understand, but AutoMapper does take a bit of getting used to. I also mention my EfCore.GenericServices library that uses AutoMapper, which speeds up the writing of database accesses even more.

TR;DR; – summary

  • LINQ Select method normally produces the fastest performing query because they only load what you need, and some calculations can be run in the database (see Posts.Count later) .
  • Writing Select queries contains more lines of code than other query approaches because you have to select each property you want to load.
  • AutoMapper’s ProjectTo method can build the Select query for you, but you need to configure the mapping.
  • Setting up the AutoMapper configuration isn’t total obvious (to me at least) so I give you some examples.
  • My GenericServices library uses AutoMapper and does some of that configuring for you.

This article is aimed at developers who use Microsoft’s Entity Framework library, so I assume you are familiar with C# code and either Entity Framework 6 (EF6.x) or Entity Framework Core library. I don’t assume you know the AutoMapper library, but I don’t detail all the features of AutoMapper and give links to its documentation. The examples in this article use EF Core, but all of the ideas are transferable to EF6.x.

Setting the scene – A quick intro to Entity Framework Core

Database accesses are normally referred to as CRUD operation (Create, Read, Update and Delete), with the read referred to in EF as a query. Queries are normally the most common operation, and often the one you want to be quick (Google and Amazon searches have conditioned us to expect very fast searches). Sometimes the query is a list of things, like products you can buy or trains you could catch, and sometimes you need a single item, like a calendar event you want to change. When the data is stored in a relational database, like SQL Server, a query might need data from multiple parts of the database, called tables, to form the exact data the user wants to see.

The EF Core library makes it easy to access a database by mapping the database tables to .NET classes (known as entity classes), and uses the database pointers (known as foreign keys) between tables to create references (known as navigational properties) to other entity classes. It also turns your LINQ commands into the correct language/format used by the database you want to access. The figure below shows four entity classes, which represent a set of Posts, each with a Blogger who wrote it, and some Tags (via a many-to-many PostTag linking entity class).  The relational properties are represented in red.

In general, the LINQ Select operator in Entity Framework Core (EF Core) can produce database queries that are often perform better than any other approach. This is because the Select method allows you to pick exactly the properties you want, including looking into related classes. This performance gain is especially good when part of the query can be done inside the database, e.g. like counting how many posts a blogger has written (I show that later).

The problem is that writing a Select query, with an assignment for each property, is very repetitious and a bit boring (and boring is bad, as I can make mistakes when I am bored). In this article I show how you can use a library called AutoMapper to automatically build Select queries, which saves you time (and you don’t need to be bored anymore). I also cover one of my libraries that uses AutoMapper to make Create, Read, Update and Delete (known as CRUD) database accesses easier.

In the next section I am going to build some queries to show this database in different ways.

Part 1. Building a simple database query – a list Bloggers

The first display I want to build a list of each blogger, with the number of posts they have written. Here is the type of display I want to show.

This must pick out information from two of entity classes. Now, let’s see the different ways of obtaining that data.

1a. Using Eager loading to load the Post classes

One way to do that would be to use Eager Loading. The code would look like this.

var list = context.Bloggers
    .Include(x => x.Posts)
    .ToList();

That gets the data we need, but it pulls in ALL of the post’s content, when I only want to count them. That is very inefficient, because the Posts can be very big. If you need good performance then you don’t want to write your queries this way.

NOTE: You get EVEN WORSE performance if you load the Posts via Lazy Loading, e.g. having a virtual public ICollection<Post> of Posts. That will cause one extra trip to the database for every entry. I get paid to performance tune EF systems and the first thing I look for is Lazy Loading – it’s a major performance drag.

See the Summary of the performance table for detailed timings.

1b. Using a hand-coded Select query

Over the years I have found the LINQ Select queries to be more efficient, where I only select the properties I need into a class I call a Data Transfer Object (DTO), also known as a ViewModel in ASP.NET. Here is my ListBloggersDto class that I am going to copy the data into

 
public class ListBloggersDto 
{
    public string Name { get; set; }
    public string EmailAddress { get; set; }
    public int PostsCount { get; set; }
} 

And here is a hand-coded Select query that will only read in the data I actually need.

 
var list = context.Bloggers.Select(x =>
    new ListBloggersDto
{
    Name = x.Name,
    EmailAddress = x.EmailAddress,
    PostsCount = x.Posts.Count
}).ToList(); 

This is more efficient as it only loads the data it needs. This is especially true for the count of the Blogger’s Posts – this is done in the database, which stops us loading all the Posts just to count them. The SQL that EF Core would create from this query looks like this (don’t worry if you don’t know SQL – take it from me that is very efficient).

 
SELECT "x"."Name", "x"."EmailAddress", (
    SELECT COUNT(*)
    FROM "Posts" AS "p"
    WHERE "x"."BloggerId" = "p"."BloggerId"
) AS "PostsCount"
FROM "Bloggers" AS "x"

Summary of the performance

In section 12.5.1 on my book “Entity Framework Core in Action” I built a query to load the data to create the book display list (see http://efcoreinaction.com/ for live example site) using three approaches. Here is the result of the three different ways of loading the data. As you can see the Select query was the fastest.

NOTE: I didn’t test Lazy loading as EF Core 2.1 wasn’t released when I was writing that chapter, but I can tell you it would be equal (or maybe worse) to the explicit loading timing.

Now, for our much simpler example the Select method would have a sub-millisecond timing because the query is so simple, and would be much quicker than any of the other loading approaches. But the table above gives you a better comparative of a complex query (see my article “Entity Framework Core performance tuning – a worked example” for more on performance)

Quick to write – AutoMapper

Having looked at the “quick to run” part lets move onto the “quick to write” part. The query examples so far are pretty easy to write, but in bigger Select queries you would have to write many assignments, and that gets tedious. This is where AutoMapper comes in, so lets now recreate the same query using AutoMapper.

1c. Using AutoMapper to build the Select query automatically

AutoMapper is what is known as an object-to-object mapper, and can selectively map from one class to another, handling any relationships. AutoMapper can also produce LINQ code via its ProjectTo<T> method (see Queryable Extensions). This allows you to let AutoMapper build your Select queries for you by matching up names. There are three stages to building a Select query using AuthoMapper:

  1. Create your DTO
  2. Add an AutoMapper configuration,
  3. Then use that AutoMapper configuration in your query.

Let’s go through each part in turn

a) Create my DTO

I need to create my DTO, which is the same as the hand-coded one.

 
public class ListBloggersDto
{
    public string Name { get; set; }
    public string EmailAddress { get; set; }
    public int PostsCount { get; set; }
}

The first two properties have the same name and type as properties in the Blogger class, so they are copied. The interesting property is the PostsCount – this is translated as Posts.Count(). That’s a AutoMapper feature which is helpful.

NOTE: AutoMapper can also ‘flatten’ relationships’: what that means it can pick out a one-to-one relationship by joining together the two names without the dot in the middle. You will see this in part 2, where I list each Post, but access the Blogger’s Name by using the name of the relationship, Blogger, followed by the property’s name, Name, i.e. BloggerName.

b) Configure the AutoMapper mapping via dependency injection

There are lots of ways to configure AutoMapper mappings. For NET Core the recommends using AutoMapper’s Profile and assembly scanning with NET Core’s Dependency Injection (DI).

For this example, I use a DI approach to configuring your AutoMapper mapping in an ASP.NET Core web application. First you add a class that inherits AutoMapper’s Profile class, most likely near to where you define your DTOs. In class’s constructor you add one or more CreateMap methods to set up the mapping. Here is a simple example.

 
public class BloggerDtosProfile : AutoMapper.Profile
{
    public BloggerDtosProfile()
    {
        CreateMap<Blogger, ListBloggersDto>();
        // Add other CreateMap’s for any other configs
    }
}

Then you need to call AutoMapper’s AddAutoMapper() method. In ASP.NET Core that would be in the ConfigureServices method in the Startup class, e.g.

 
public IServiceProvider ConfigureServices(
    IServiceCollection services)
{
    // … other setups removed for clarity
    services.AddAutoMapper();
}

The AddAutoMapper() method scans all the assemblies looking for all the classes that inherit AutoMapper.Profile class. It then produces a IMapper instance containing the all the Mappings found.

NOTE: I find the configuring AutoMapper is the hard part, as it relies on me remembering to set up the AutoMapper mapping. I have tried a number of ways to make this better and I think in my EfCore.GenericServices I have got it right. In that I write my DTO and then you need to add an empty interface called ILinkToEntity<TEntity>, where TEntity is your class that EF Core maps to the database. Then the setup of EfCore.GenericServices a) finds the DTOs and then uses the TEntity part of the ILinkToEntity<TEntity> interface to, among other things, form the correct AutoMapper mappings.

c) Use AutoMapper in your query

I’m now going to show you a ASP.NET Core controller so that you can see how the DI works, but you can use AutoMapper in any application you like.

 
public class HomeController : Controller
{
    private readonly BlogContext _context;
    private readonly IMapper _mapper;

    public HomeController(BlogContext context, IMapper mapper)   
    {                                              
        _context = context;  
        _mapper = mapper;                      
    }                                              

    public IActionResult Index()            
    {
        var dtos = _context.Bloggers
            .ProjectTo<ListBloggersDto>(_mapper)     
            .ToList();   

        return View(dtos);         
    }
}

I have injected by EF Core content and the AutoMapper’s IMapper instance into the controller via the constructor. Then any of my methods in the ASP.NET Core controller. And the SQL code produced is the same as the hand-coded Select earlier in 1b.

For this small DTO then it isn’t worth using AutoMapper, but most applications have hundreds of mappings like this, and classes that contains a lot more properties. This is where AutoMapper comes into its own.

NOTE: If you are generally doing CRUD operations then you should look at my article “GenericServices: A library to provide CRUD front-end services from a EF Core database” that uses AutoMapper and EF Core to make CRUD very simple to do.

AutoMapper is clever and can work out some relationships for you, but for more complex relationships you need to do some more configuring, which I cover in the next part.

Part 2: Handling more complex database queries – list of posts

The next example is a summery list of all the articles, showing the bogger (author), the post title and the tags the post has.

This is a bit more complex, especially around getting the tag names.

2a. List posts using hand-coded select query

Let’s start with the hand-coded LINQ Select query. First, I need to show you the DTO I use to hold the resulting query.

 
public class ListPostsDto 
{
    public string BloggerName { get; set; }
    public string Title { get; set; }
    public DateTime LastUpdated { get; set; }
    public List<string> TagNames { get; set; }
}

Now the hand-coded Select query.

 
var dtos = context.Posts.Select(x => 
    new ListPostsDto
{
    BloggerName = x.Blogger.Name,
    Title = x.Title,
    LastUpdated = x.LastUpdated,
    TagNames = x.TagLinks.Select(y => y.Tag.Name).ToList()
}).ToList();

Notice the following:

  • Line 4: I can select the Name of the Blogger by using the navigational property. EF can handle any depth of selection, say the Bogger class had a property called Address, which has its own table holding address information, then I could access the Blogger’s Country via x.Blogger.Address.Country. EF produces efficient SQL (an INNER JOIN) to extract the data from the related tables.
  • Line 7: I want a list of all the Tag Name’s associated with this post. I therefore need to use the PostTag linking table to access the Tag names.

The produces fairly efficient (see note below) SQL query by only extracting the data you need.

NOTE: Handling “many” relationships, i.e. ones that return multiple results like the TagNames, is an area where EF Core can have performance issues. If I left off the .ToList() on the TagNames then it would query the database for each ListPostsDt0 (this is known as the N+1 database queries issue). But in EF Core version 2.1, adding the .ToList() turns the load of all the TagNames into one final database query. This is OK, but can still have problems – see my article “Entity Framework Core performance tuning – a worked example” to see an even better way to handle comma separated strings.

2b. List of posts using AutoMapper and special configuration

AutoMapper is clever enough to automatically map the first three properties in the ListPostsDto – in particular it maps the BloggerName property to Blogger.Name. What it can’t do is work out the mapping for the TagNames, so we need to add some configuration the CreateMap method. I use the original ListPostsDto I used in the hand-coded version, but add the AutoMapper configuration code:

 
public class ListPostsDto
{
    public string BloggerName { get; set; }
    public string Title { get; set; }
    public DateTime LastUpdated { get; set; }
    public List<string> TagNames { get; set; }
}

Now the AutoMapper configuration code.

 
public class PostsDtosProfile : AutoMapper.Profile
{
    public PostsDtosProfile()
    {
        CreateMap<Post, ListPostsDto>()
            .ForMember(
                p => p.TagNames, 
                opt => opt.MapFrom(x => 
                    x.TagLinks.Select(y => y.Tag.Name).ToList()));
    }
}

The ForMember method (lines 6 to 9) takes two parts: the first part defines the property in the DTO that you are mapping (TagNames, on line 7), and the second part (lines 8 to 9) tells AutoMapper how to generate the data to go into the TagNames property from the input class, which is the Post class.

NOTE: AutoMapper automatically turns the BloggerName into Blogger.Name – this is the “flattening” I referred to earlier. This is a very useful feature and makes handling references to a single relationship really easy.

Let’s see the list articles query using AutoMapper’s ProjectTo method. In case I’ll show you a unit test as you might need this in your system. I also show how to create the IMapper variable using a helper method I created, as that is useful too (and checks you created the Profile class correctly).

 
[Fact]
public void TestProjectionMappingPosts()
{
    //SETUP
    var config = AutoMapperHelpers
        .CreateMapperConfig<PostsDtosProfile>();

    //ATTEMPT
    var input = EfTestData.CreateBloggersWithPosts()
        .AsQueryable();
    var list = input.ProjectTo<ListPostsDto>().ToList();

    //VERIFY
    list.First().BloggerName.ShouldEqual("Ada Lovelace");
}

Where my helper method CreateMapperConfig, looks like this.

NOTE: Technical point about unit testing. ProjectTo<T> need MapperConfiguration when testing, while Map<T> needs IMapper. Have a look at my unit tests of AutoMapper here for a good example of how to write tests that include AutoMapper.

 
public static MapperConfiguration CreateMapperConfig<T>() 
    where T : Profile, new()
{
    var config = new MapperConfiguration(cfg =>
    {
        cfg.AddProfile(new T());
    });
    return config ;
}

This produces the same SQL as the hand-codes example.

NOTE: In EfCore.GenericServices you can configure the AutoMapper mappings by adding a class which inherits from PerDtoConfig<TDto, TEntity>. When EfCore.GenericServices is scanning for DTOs it also looks for matching PerDtoConfig<TDto, TEntity> classes uses that data to alter the AutoMapper’s MapperConfiguration.

Conclusion

Firstly, I showed you how to create high performance queries by using the LINQ Select method. The Select query isn’t obvious – in fact the EF Core documentation on loading relationships doesn’t even list it even though you can load relationships in a Select method. But the ASP.NET Core EF tutorial lists Select as an option, as I was asked to help develop that tutorial 😊. If you want fast and effective queries then think about using the LINQ Select method.

The problem with the Select method is it takes some work to write it, e.g. when Selects have lots of properties in them then they get long and repetitious (i.e. boring). I showed that AutoMapper can help by automatically build the Select query for you. While AutoMapper is a great tool, but it takes a bit of getting used to, especially around the configuration. But when you have lots of queries then it’s worth the effort.

Along the way I talk about my EfCore.GenericServices library, which uses AutoMapper inside. This handles the configuration of AutoMapper for you, which makes it a bit easier. But on the other hand, you need to learn how to set up EfCore.GenericServices. I have written an introductory article, some documentation, and an example application in the repo that you can run locally (it uses an in-memory database, so it runs anywhere), so don’t be afraid to try it.

Happy coding.

Three approaches to Domain-Driven Design with Entity Framework Core

Last Updated: September 25, 2018 | Created: September 24, 2018

On my article “Creating Domain-Driven Design entity classes with Entity Framework Core@ardalis commented that “your entities all are tightly coupled to EF Core. That’s not good…”. Then I did a podcast with Bryan Hogan where we discussed Domain-Driven Design (DDD) and he goes further than my CRUD-only (Create, Read, Update, and Delete) approach – he says that the entity classes is the perfect place for business logic too.

NOTE: My discussion with Bryan Hogan is now out. You can find the PodCast here.

With such diverging views on the best way to implement DDD in Entity Framework Core (EF Core) I decided to write an article that a) compares normal approach with a DDD approach, and b) compare three different ways to implement DDD in EF Core. This is a detailed look at the issues, hence it is very long! But hopefully useful to those looking to use DDD, or developers that want to consider all the options available to them.

TL;DR; – summary

NOTE: DDD is a massive topic, with many facets. In this article I only look at the entity class issues, which is a tiny part of what DDD is about. I really recommend Eric Evan’s book Domain-Driven Design for a full coverage.

The DDD approach to writing entity classes in EF Core makes every property read-only. The only way to create or update entity data is constructors (ctors), factories or methods in the entity class. This article introduces the DDD entity style by comparing the standard, non-DDD approach, against a basic DDD-styled entity class. This gives us a clear starting point from which I can go on to compare and contrast three difference DDD approaches:

  1. A DDD-styled entity, but not including any references to EF Core commands.
  2. A DDD-styled entity which has access to EF Core’s DbContext and implements CRUD methods.
  3. A DDD-styled entity which has access to EF Core’s DbContext and contains all code that interacts with the entity, i.e. CRUD and more complex business logic.

Comparing a standard entity class with a DDD approach

Let’s start by comparing an implementing something using a standard entity class and a DDD-styled entity class. The aim of this section is to describe the major differences and sets the scene for a look at the subtler differences between the three DDD approaches.

The diagram below shows a standard entity class (i.e. a class that EF Core maps to the database) with read-write access.

And this diagram shows the same entity class but using a DDD approach.

The big difference is that the standard entity class can be created/changed by any external code, but in the DDD-styled entity class you can only create/change data via specific constructors/methods. But why is this a good idea? – here is a list of benefits:

  • External code now has clearly named methods/static factories to call. This makes it much clearer to developers what the entity class supports in terms of creating/changing that class.
  • The code to create/change the class is contained in the class itself, which keeps the code co-located with the data. This makes writing/refactoring of the class much simpler.
  • It stops duplication of code, and in multi-person projects it stops different developers (or even the same developer!) applying different business rules to the same feature.

DDD is about making the domain (i.e. business) rules the focus of the code, so having methods like ChangeDeliveryDate or MarkOrderAsDispatched in your entity class encapsulates business rules with a meaningful name. There is much less possibility of getting things wrong this way.

Pros and Cons of Standard & DDD approaches

Here is summary of the Pros/Cons (advantages/disadvantages) of the two approaches

Approach Pros Cons
Standard Simple.

Minimum code.

 

Big systems can become quite hard to understand.

Possibly of duplicate code happening.

DDD-styled More obvious, i.e. meaningful named methods to call.

Better control of access to data.

Puts the code next to the data.

Slightly more code to write.

 

Different views on building DDD-styled entity classes

Having introduced the DDD approach I now want to look at three different approaches to implementing DDD entity classes. Here is a diagram to show you the three DDD approaches, showing what code the entity classes contain. I use two terms that I need to define:

  • Repository: A repository pattern provides a set of methods to access the database. These method ‘hide’ the code needed to implement the various database features you need.
  • C(r)UD: This is Create, (Read), Update, and Delete – I use the term C(r)UD to separate the Read from the functions that change the database (Create, Update are Delete).
  • Query Objects: This is a design pattern for building efficient database queries for EF. See this article from Jimmy Bogard on this topic.
  • Business logic (shortened to Biz Logic): This is more complex processes that go beyond simple validation. They may have complex calculations (e.g. pricing engine) or require access to external systems (e.g. sending an email to a customer).

As you will see in the diagram below, the further to the right you go the more external code is moved inside the entity classes.

NOTE: I will use the terms POCO-only class, C(r)UD only and C(r)UD & Business Logic when referring to these three approaches.

1. The different ways of reading data

The first big difference is how the data is read from the database. When using a repository pattern, you tend to read and as well as write via the repository pattern. I personally have found (some) repository patterns can deal to poor performing code, mainly because it’s hard to build a fully featured repository with the correct adapters for the front-end. I therefore use Query Objects, normally defined in the services layer so that they can adapt the data to the needs of the front end.

I’m not going to cover this here as I have an article called “Is the repository pattern useful with Entity Framework Core?” which goes into it in detail, with this section on query objects.

2. The different ways of updating relationships

The big difference between the POCO-only class approach and the other DDD versions is that the POCO-only version doesn’t have any access to the DbContext, so it can’t load relationships. To show why this matter, let’s introduce a Book entity class a collection of Reviews, e.g. (5 stars – “It’s a great book” says Jill). Now we want to implement a method to allow a user to add a new Review entity class to an existing Book entity instance. The steps are:

  1. Get the new review information, with the Book’s primary key.
  2. Load the Book entity instance using the primary key.
  3. Call a method in the loaded Book instance to create a new Review and add it to the Book’s Reviews collection.

The problem comes if the Reviews collection hasn’t been loaded when the Book entity instance was loaded. If we assume the Reviews collection was loaded and its wasn’t, then adding a new review will either a) fail with a null collection or b) silently delete all the other reviews! Therefore, something has to make sure the collection is loaded. Here are some possible implementations for the POCO-only version, and the other two DDD versions:

1.a Handling adding a Review to POCO-only version

In a POCO-only entity class there is no references to EF Core. This means you need something else to handle the correct loading of the Book with its Review collection. The standard way to do this is via a repository pattern. The repository would contain an AddReview method that would load the Book entity instance, along with loading the Reviews collection before it calls the Book’s AddReview method to update the Books _reviews backing field. Here is some example code:

1.b Handling adding a Review within the DDD-styled entity class

The limitation of the POCO-only entity class is you need to rely on the caller to pre-load the Reviews collection. Because the POCO-only entity class can’t access any of EF Core’s classes or methods. In the other two types you can access EF Core, so you can move loading of the Reviews collection inside the entity class. Here is the code:

NOTE: When using the C(r)UD only approach the code in the ASP.NET controller has a repetitive pattern:  load the entity and call a named method, which makes it a candidate for building library to handle this. This is exactly what I have done with the EfCore.GenericServices library, which can work with normal and DDD-styled entity classes. This the article “GenericServices: A library to provide CRUD front-end services from a EF Core database” for more on this.

Comparing the two approaches to adding a review

The differences are subtle, but important. In the POCO-only class the update relies on an external piece of code, in this case a repository pattern, to correctly update the reviews. In the C(r)UD only case the DDD access method, AddReview, handles the whole process.

Now, if you are using a repository pattern everywhere then this difference is small. However, it does allow a small loophole in which someone could bypass the repository and calls AddReview directly, which with the code I have written would cause a null reference exception (other designs I have seen silently delete existing reviews if you don’t preload the collection).

In this case the C(r)UD only approach is better because it encapsulates the whole of the process so that calling methods don’t have to know anything about the process. Therefore, the C(r)UD only (and the C(r)UD & Business Logic) approach follows the Single Responsibility Principle, i.e. the AddReview is responsible for adding a new review to a Book, and entirely encapsulates the code to add that review.

However, the C(r)UD only approach has one big down side – it refers to EF Core. Why is this a problem? The entity classes, like Book and Review, are core parts of your domain design and Eric Evans says in chapter 6 of his book that repositories should be used to “decouple application and domain design from persistence technology”. There are many people who like having entity classes that contain no database access logic so that they the class is focused on domain (i.e. business) logic.

You have to remember that the repository pattern was the recommended approach when Eric Evan’s book was published in 2004. Personally I, and others, don’t think the repository pattern appropriate for EF Core (see my article “Is the repository pattern useful with Entity Framework Core?”).

In the next section we look more carefully at the business logic and how that is handled, but before I leave this comparison, I should talk about performance. In the two examples the POCO-only class approach is quicker, as it loads the reviews at the same time as the book, while the C(r)UD only approach needs two trips to the database. I used similar code two examples to make it easier for you to compare the two approaches, but in my actual code (shown below) I use a much more efficient approach if the _reviews collection is not loaded/initialised: It just creates a new review with the correct foreign key, which is much more efficient.

public void AddReview(int numStars, string comment, 
    string voterName, DbContext context) 
{
    if (_reviews != null)    
    {
        //Create a new Book: the _reviews HashTable is set to empty
        _reviews.Add(new Review(numStars, comment, voterName));   
    }
    else if (context.Entry(this).IsKeySet)  
    {
        //Existing Book: create a new review directly
        context.Add(new Review(numStars, comment, voterName, BookId));
    }
    else                                     
    {                                        
        throw new InvalidOperationException(
             "Could not add a new review.");  
    }
}

3. Different ways of handling business logic

The first example was a Create, i.e. adding a new Review (known as an aggregate entity in DDD) to the Book entity (known as the root entity in DDD). These CRUD operations might include some form of validation, such as making sure the numStars value in the Review is between 1 and five. But when the rules get more complicated than validating properties then we tend to call the code business logic.

When someone wants an application written there are rules on how things work. For instance, placing an order might require certain checks on stock, payment etc. and kick off other tasks such as processing the order and delivery. The steps in the example order are called business rules, or in DDD-speak, domain problems.

In Eric Evans book he says “When the domain is complex, this is a difficult task, calling for the concentrated effort of talented and skilled people”. In fact, most of Eric Evan’s book is about domain problems: how to describe them (ubiquitous language), how to group domain problem (bounded context), etc. But in this section, I’m just going to look at the three DDD approaches and how they handle business logic.

For this I am going to use the example of processing a customer’s order for books. To make it easier to understand I am only showing 5 stages (e.g. I left out payment processing, shipment, etc.):

  1. Load the Books ordered using the primary keys provided by the higher layers
  2. Get the address that the order should be sent to
  3. Create the Order
  4. Save the Order to the database
  5. Send an email to the user saying the Order was successfully

I am also going to show the layering of the assemblies, i.e. in what order the assemblies reference each other. This matters because in .NET you can’t have circular references to assemblies (that causes problems at compile time), which constraints how you put the assemblies together. In most cases the entity classes are the lowest assembly, because everything else needs to access these key classes. (NOTE: I also add a red oval on the assembly that references EF Core).

2.a. POCO-only class approach

The POCO-only class approach uses a repository for database accesses so when the business logic needs data it relies on the repository. Because you can’t have circular references to assemblies this means part of the business logic moves into the repository. Typically, this results in repository taking on a coordination of the building of the order (Command Pattern). The figure to the right shows the assembly references.

And below is a diagram of one possible implementation of handling a customer order:

2.b The C(r)UD only approach

Over the years I have developed a series of rules to help me in implement business logic. And one of them is “Business logic should think it’s working on in-memory data”. Having decided that generic repository pattern doesn’t work well with EF Core (see my article “Is the repository pattern useful with Entity Framework Core?” on why that is) I had to come up with another way to do this. My solution is to have two new assemblies, which are:

  1. BizLayer: This contains classes that are pure business logic, i.e. they don’t access EF Core directly, but rely on their own mini-repository for all database actions
  2. BizDBAccess: This is handles all database accesses for a single business logic class, i.e. it’s a dedicated repository for the business logic.

It may seem odd to say I don’t like the repository pattern, and then build a dedicated repository for each business logic. But it’s because writing generic repository is actually very difficult – for instance EF Core is a generic repository/unit of work pattern and think how much effort it’s taken to write that! But writing a dedicated repository for a specific piece of business logic is easy. (I agree with Neil Ford’s comment from his book Building evolutionary architectures –The more reusable the code is, the less usable it is.”).

Now, I’m not going to show the code here, as I have a much longer article called “Architecture of Business Layer working with Entity Framework (Core and v6) – revisited” where I covers the same example of placing an Order (NOTE: this article doesn’t include a “SendMail” part, but I think you can see that goes in the business logic).

2.c. C(r)UD and business logic

The final version has methods/factories for ALL manipulation of the data in the entity class. This means any there is no repository and no business layer because it’s all handled by the entity class itself.

In this case the business logic code that was in the repository (see 2.a) is moved completely into the entity class. The technical issue with this is the SendMail method is in an assembly that is linked to the assembly containing the entity classes, which stops you referencing the SendMail directly. It’s pretty simple to fix this by defining an interface (e.g. ISendMail) in the entity class and using dependency injection to provide the SendMail instance at run time.

At this point every operation other that read (and maybe some deletes) will all be entity class, and those method will use EF Core to access the database directly from the entity itself.

Comparing approaches to business logic

In the end the differences are around the ‘scope’ of the code inside the entity class. The POCO-only has the simplest code, with no references to EF Core. The C(r)UD only approach uses EF Core to be a one-stop-shop for doing database changes but has the business logic in another assembly. While the C(r)UD + Biz Logic approach has everything that changes the database inside the entity class.

Here is a table where I try to list the advantages/disadvantages of each approach.

Approach Pros Cons
1. POCO-only Follows the original DDD description of handling entities. Hard to write generic repositories.

Repository pattern in data layer can cause performance problems.

The repository can be bypassed.

2. C(r)UD only Access methods follow the single responsibility principle.

Business logic is separated from database accesses.

More work writing the business logic and its dedicated repository.
3. C(r)UD + Biz Logic Everything in one place. Can suffer with the “God Object” anti-pattern.

Problems calling methods in “higher” assemblies.

 

While I think a DDD approach is useful, it’s a trade-off of features and limitations whichever way you go. In the end it depends on a) your application and b) what you feel comfortable with. For instance, if your application has some really complex business logic, with lots of calls to other parts of the system then the C(r)UD + Biz Logic approach might not work that well or you.

Conclusion

I do believe using a DDD approach with EF Core has many benefits – it can make the code clearer and more robust (e.g. no way to get it wrong). But coming up with a good DDD approach can take a while – I built my first DDD business logic approach in 2015, and it’s been through two iterations in 2016 and 2017 as I have improved it. This article is just another part of my continuing journey to learn and improve my skills around using DDD.

I am also want to be an efficient developer, so whatever approach must allow me to build quickly. But with a bit of help from some libraries, like the EfCore.GenericServices, I can build both robust, well-performing systems quickly. And when new features, like JSON Patch, become useful I try to adapt my approach to see if it can be added to my DDD style (see my article “Pragmatic Domain-Driven Design: supporting JSON Patch in Entity Framework Core” to see the outcome).

I hope this article helps you decide for yourself whether it’s worth using the DDD approach with EF Core and which approach suits your needs.

Happy coding!

Pragmatic Domain-Driven Design: supporting JSON Patch in Entity Framework Core

Created: September 20, 2018

I’m a big fan of Domain-Driven Design (DDD) and build all my Entity Framework Core (EF Core) entity classes using a DDD approach, i.e. all properties have private setters and all changes are made via method in the entity class. This makes it very clear what, and how, properties and relationships can be updated. (See the article “Creating Domain-Driven Design entity classes with Entity Framework Core” for more details on implementing DDD in EF Core).

But there are times where a fully DDD-styled entity can create more work for yourself. I’m working with a client on a large application that will have hundreds of Web APIs feeding an Angular front-end. In this case using the JSON Patch feature will save the frontend developers a lot of time, because there are libraries that will process a form and create a JSON Patch to update the properties that have changes. (see the article “JSON Patch With ASP.NET Core“ for a fuller description of JSON Patch).

In this article I talk about how I merged the use of JSON Patch approach into my DDD entity class approach. I also describe how I added support for JSON Patch in my EfCore.GenericServices library, while keeping all the good attributes of a DDD-styled entity class.

Introducing the hybrid DDD entity class

JSON Patch, with its update via the property’s setters, seems at odds with the DDD approach to entity classes where properties have private setters. But you can’t ignore the fact that JSON Patch, and associated libraries, can significantly speed up the building of single-page applications, using Angular, React, Vue etc.

DDD is all about making the entity class business-focused, where it deals in terms of the business needs rather than simple properties. So, having methods in the entity class with names like MarkOrderAsDispatched is better than setting the OrderStatus to Dispatched. Also, DDD wants to provide the developer with a clear idea of if/how a property can be updated, especially around relationships.

When building my EfCore.GenericServices library I made it capable of updating “standard” entity classes, i.e. with public setters, and also work with DDD-styled entities, i.e. where updates are done via calling public methods. In all of this I did allow what I call a hybrid DDD entity class, where some properties were updated by method calls, and some were updated via their public setters.

The typical use case for a hybrid DDD entity class is updating an address, with its various street, city, state, zip/postcode properties, inside entity class. I wanted to keep DDD’s tight control over the relationships, but in many cases updating the address properties has no business effects so it simplified things to make the properties’ setters public (the library would notice that and call AutoMapper to update the properties from a DTO).

While some might say that a hybrid DDD entity class breaks the DDD approach, but I believe it follows the spirit of what Eric Evans was getting at in his book, Domain-Driven Design. The ‘intent’ of what can be done with property is clearly communicated.

  • If a property has a public setter then there are no business logic implications (other than validation).
  • If the property has a private setter then it either has business logic implications or you wanted to provide a business-named access method, like my earlier example of MarkOrderAsDispatched. Either way you must update it via a method call because the setter is private.

Altering a DDD-styled entity to allow JSON Patch to work

JSON Patch only works if the properties have public setters – which is where the hybrid DDD entity class comes in.  Here is an example from my EfCore.GenericServices.AspNetCore library, which contains some simple entity classes and Web API to show how things work (you can clone and run this locally to try this out – it uses an in-memory which is seeded at startup to make it easier to play with).

I created a simple hybrid entity class called TodoItemHybrid, as shown below

public class TodoItemHybrid
{
    public int Id { get; private set; }

    [Required(AllowEmptyStrings = false)]
    public string Name { get; private set; }

    [Range(1, 5)]
    public int Difficulty { get; set; }

    //ctor and access methods
    public TodoItemHybrid(string name, int difficulty)
    {
        Name = name ?? throw new ArgumentNullException(nameof(name));
        Difficulty = difficulty;
    }

    public IStatusGeneric ChangeName(string name)
    {
        var status = new StatusGenericHandler();
        if (name.EndsWith("!"))
            status.AddError("Business logic says the name cannot end with !", nameof(Name));

        Name = name;
        return status;
    }
}

You can see that the Name property (line 6) has a private setter and can only be updated via the ChangeName method (lined 18 to 26). That method has some (fake) business logic connected to it (see lines 21 to 22), so that must be updated via the ChangeName method.

On the other hand, the Difficulty property (line 9) is a simple property with no business logic attached (apart from validation), so I have given it a public setter. This means I can update it via the Microsoft.AspNetCore.JsonPatch library.

The way that JSON Patch works is you can provide a list of changes. In this simple case there is only one property, Difficulty, but you can provide an array of updates. Here is simple example containing two updates to the same class:

[
	{ "op": "replace", "path": "/FirstName", "value": "Jon" }
	{ "op": "replace", "path": "/LastName", "value": "Smith" }
]

Using JSON Patch can cut down the amount of work required in the front-end code. A JavaScript library such as https://www.npmjs.com/package/rfc6902 can automate the creating of the JSON patches to send back to the server. This package has a createPatch feature that compares original data with new data (for instance, from a form) and generate a patch with all the changes to send to the server.

A look at the ASP.NET Core Web APIs

To apply such an update from a Web API controller, then you would need something like this

[HttpPatch("{id}")]
public IActionResult HandCodedUpdate(int id, 
     JsonPatchDocument<TodoItemHybrid> patch, 
     [FromServices]ExampleDbContext context)
{
    var entity = context.Find<TodoItemHybrid>(id);
    if (entity == null)
    {
        return NoContent();
    }
    patch.ApplyTo(entity);
    context.SaveChanges();
    return Ok();
}

The call sends the primary key of the entity you want to update (end of line 2), plus a JsonPatchDocument<T> patch (line 3) created by the caller. Then you have to load the entity you want to update (line 6) and if found you apply the patch and save the changes to the database (lines 11 to 12).

To make this more streamlined I have added a JSON Patch feature to my EfCore.GenericServices library. This wraps the ApplyTo with extra checks, optional validation and success/error messages. Here is the same example but using the GenericServices library:

[HttpPatch("{id}")]
public ActionResult<WebApiMessageOnly> Update(int id, 
    JsonPatchDocument<TodoItemHybrid> patch, 
    [FromServices]ICrudServices service)
{
    service.UpdateAndSave(patch, id);
    return service.Response();
}

Conclusion

DDD can provide many benefits, such as access methods which are named in a business-focused way, and these access methods are the definitive way to do something, as there is no other way in. I really like that level of isolation.

But most database tables have plenty of simple read/write columns that don’t need the level of access control. Writing access methods still works, but sometimes a simpler copy of data is sufficient. And when approaches like JSON Patch make the developers life simpler its worth adopting it in the right places.

In the end DDD is more a philosophy than a standard pattern – Eric Evans was saying “make the domain (business) problem the focus, not the tools and technology you use”. My hybrid DDD entity class might not be liked by some, but I think it still conveys the intent of the data, and it makes you a quicker developer.

Happy coding!

How to write good, testable ASP.NET Core Web API code quickly

Last Updated: September 1, 2018 | Created: August 7, 2018

This article is about ASP.NET Core 2.1 Web API code and ways to write code that is easy to understand, easy to test and quick to write. You might think that is quite a task, but I introduce a number of libraries that make that feat possible. Also there is an example application that you can clone from GihHub and run locally to see the libraries in action.

If you are new Web API I do start with a general introduction which covers the difference between Web APIs and using ASP.NET Core to directly deliver HTML pages. I then show you how to write simple Web API actions and set the correct HTTP response type back to the caller.

TL;DR; – summary

I describe a way to write Web API controllers that moves the database access code out to a library called GenericServices – this reduces the code needed in a Web API to a few lines. I also have another library called GenericBizRunner that does the same for business logic, which I cover in this article too. I describe a companion library called EfCore.GenericServices.AspNetCore that handles the creation of the correct response with a HTTP status code and json payload.

The overall effect of using the described pattern and libraries is that your Web API actions are small, easy to test and quick to develop.

NOTE: There is an example application on GitHub, which you can clone and run locally – it uses an in-memory database and includes Swagger, so its easy to run.

Introduction to ASP.NET Core Web API

Note: if you already know about ASP.NET Core Web API then you can skip this section.

There are now three main approaches to building a web application using ASP.NET Core version 2.1:

  1. NET Core MVC, which uses controllers, actions and views
  2. NET Core Razor Pages, which has pages with associated code
  3. NET Core Web API, which can be queried by a framework, like React.js, or an external system

The first two have the ASP.NET Core web app deliver HTML pages, with the help of the built-in Razor syntax. In these cases, the ASP.NET Core web app is in charge of what is shown on the screen – see diagram below:

But in the ASP.NET Core Web API case what is shown to the user is controlled by something outside ASP.NET Core. For the human-user example its either framework (like AngularJS or React.js) or mobile application makes requests to the ASP.NET Core application to either get data or execute some business logic – see the diagram below:

There are pros and cons to both approaches, but nowadays more applications are moving over some sort of front-end code running in the browser or mobile which uses Web API to access the backend system. Also, application-to-application traffic can also use the Web API approach. Therefore, being aware on how to write good, testable Web API code quickly is a useful skill.

Going beyond Microsoft’s Web API examples

Microsoft has some good examples of writing Web API applications, including one using EF Core for the database (VS Code version, VS2017/Win version and VS2017/Mac version). These are a great place to start as they clearly show what is going. Here is an example taken from Microsoft’s example (Note: I have left out the constructor of the Web API controller, where the EF Core DbContext for the database is injected and put in the private field called _context).

[HttpPut("{id}")]
public IActionResult Update(long id, TodoItem item)
{
    var todo = _context.TodoItems.Find(id);
    if (todo == null)
    {
        return NotFound();
    }

    todo.IsComplete = item.IsComplete;
    todo.Name = item.Name;

    _context.TodoItems.Update(todo);
    _context.SaveChanges();
    return NoContent();
}

The example is very clear about what is happening, as you can see each part of the process, which is why Microsoft wrote it that way. But when the databases accesses and business logic get more complex, then putting the code in the Web API action isn’t such a good idea, for a number of reasons:

  1. The code needed to achieve a particular action (in the example it’s a simple update) can become many lines of code, which means your controller is long and hard to understand.
  2. Your database or business logic is likely to have extra validation/checks which it needs to return as errors, which not covered in the Microsoft example.
  3. There is a danger of duplication of code. For instance, if another action needed to update the TodoItem you would you need to duplicate part of the code.
  4. It wouldn’t be hard to unit test the Web API action in Microsoft’s example, but in more complex applications with say extra data injected via tokens or other parts of the HttpContext then it can get more difficult.

Over the years I have learnt to keep my controller actions as small as possible. This means moving the code out of the action into a separate method. At the same time, I have noticed patterns in database accesses and business logic calls which have culminated in my building libraries to handle the common parts. Here is my version of the TodoItem update using my GenericServices library, which is designed to handle CRUD (Create, Read, Update and Delete) functions to a EF Core database.

[HttpPut()]
public ActionResult<WebApiMessageOnly> Put(
    ChangeTodoDto dto, 
    [FromServices]ICrudServices service)
{
    service.UpdateAndSave(dto);
    return service.Response();
}

The differences are:

  • Its short and to the point – my Web API controller will be small and easy to read
  • The library uses a general ‘update TodoItem’ method which is used throughout the application – no possibility of duplication of code.
  • My EfCore.GenericServices (injected via the [FromServices]ICrudServices service property on line 4) provides generic CRUD functions, including the mapping from the input class to the ToDoItem class.
  • The Response method on line 7 returns a response that is either OK (HTTP 200) with a success message, NoContent (HTTP 204) if the item wasn’t found, or if the update logic or database validation finds errors then it returns Invalid Input (HTTP 400) with the error messages (The format of these responses are shown later).

This approach also allows me to unit test the ‘update TodoItem’ on its own, which I prefer. As I said before testing Web API actions can work, but if you have complex HTTP things going on, like cookies or tokens providing data, then testing can get very hard. I find it much simpler and easier to test the service.UpdateAndSave(dto) code on its own in a unit test.

NOTE: If you are using EF Core then see my open-source library EfCore.TestSupport for which has lots of useful methods for testing code that uses EF Core. Also, consider using in-memory databases for unit testing as they are really quick – the article “Using in-memory databases for unit testing EF Core applications” to show you how.

So, by doing all this I have:

  • I wrote good code: is short, obvious and does not duplicate code.
  • I have written testable code: it’s easy to duplicate the same code in your unit tests.
  • I wrote the code quickly: The EfCore.GenericServices library handles the four CRUD patterns, including copying from a ViewModel/Dto class to the EF Core entity class.

It’s my EfCore.GenericServices open-source library that makes the code shorter and quicker to write. This library is a second-generation version of my original  GenericServices library built for EF6.x, and is significantly improvement over the original. My experience with the original GenericServices library was it has saved me months-per-year of development time. I expect my new library to give me a similar, if not better, productively as it now supports DDD-styled entity classes which centralises the create/update code.

I’m not going to describe the EfCore.GenericServices library in detail here, because I have already done that in the article “GenericServices: A library to provide CRUD front-end services from a EF Core database” (see also NuGet, GitHub, and documentation pages). But that article talks about MVC/Razor Pages usage.

In this article I am going to describe how to use my libraries with Web API, and to help I have updated the companion EfCore.GenericServices.AspNetCore library to convert the status output of my libraries library into the correctly formed HTTP responses.

What about business logic?

I have handled CRUD accesses to the database, but complex applications often have complex business logic. As I mentioned before I have another open-source library called EfCore.GenericBizRunner which runs business logic working with EF Core, which the companion EfCore.GenericServices.AspNetCore library also supports. You can find out more about the EfCore.GenericBizRunner library via the article called “A library to run your business logic when using Entity Framework Core”, and the GenericBizRunner’s documentation pages.

How I turn my libraries’ statuses into HTTP responses

My EfCore.GenericServices and EfCore.GenericBizRunner libraies are linked to EF Core, but could be used in any sort of application – web, desktop etc. But I mainly use them with ASP.NET Core, so I have a library called EfCore.GenericServices.AspNetCore which contains code to change the IGenericStatus that both these libraries use into either ASP.NET ModelState (for MVC and Razor Pages) into either

  • ActionResult<WebApiMessageOnly>, where it returns just a success message (or errors if incorrect).
  • ActionResult<WebApiMessageAndResult<T>>, if the Web API returns a success message and some results (or errors if incorrect).

In this article I want to look at the CreateResponse class that contain extension methods to turn my IGenericStatus into the two ActionResult forms.

NOTE: The EfCore.GenericServices.AspNetCore contains an ExampleWebApi project that contains an ASP.NET Core 2.1 Web API controller as an example of how the CreateResponse extension methods work. It uses an in-memory database and Swagger so that it’s easy to try out the Web API commands – just start the ExampleWebApi project and it should got directly to the swagger pages which lists each Web API command and allows you to test them.

Now I will describe the three main response types and what they look like. I start with the handling errors, as ASP.NET Core 2.1 has already defined that format.

InvalidInput (HTTP 400) – error return

If you create a new ASP.NET Core application the default template turns on 2.1 compatibility, which means that any Web API input is automatically validated, e.g. if a property had the attribute [Range(1,5)] then it would return an error if that property was outside that range. This is a really useful feature, as it saves you from having to add code to validate the input at the start of the Web API action.

The format that ASP.NET Core uses to return the errors is a dictionary of errors returned with Invalid Input (HTTP 400) status code. Here is an example of the format it uses:

{
    "": [
        "Global error message"
    ],    
    
    "MyPropery": [
        "The property is required",
        "Another error on the same property"
    ]
}

Therefore, my Response extension method in the EfCore.GenericServices.AspNetCore library turns any errors found by the EfCore.GenericServices or EfCore.GenericBizRunner libray into the same format. That means there is one common error format the calling code needs to process.

The other responses we need are:

OK (HTTP 200) – successful

This contains a message string, which you can show the user or log. You can set the message yourself, or my libraries will return a message relevant to the. Also, if there is any data to return, then there is a results property. Here is an example with results, which uses the WebApiMessageAndResult<T> response class.

{
  "message": "Success",
  "results": {
    "id": 1,
    "name": "Create ASP.NET Core API project",
    "difficulty": 1
  }
}

This type of response is has a result, as shown in this Get action method

[HttpGet("{id}", Name= "GetSingleTodo")]
public async Task<ActionResult<WebApiMessageAndResult<TodoItem>>> 
    GetAsync(int id, [FromServices]ICrudServicesAsync service)
{
    return service.Response(
        await service.ReadSingleAsync<TodoItem>(id));
}

The other type of action that returns a result is a create. There are two ways to handle this.

1. Create, and return url

The standard Web API way of handling a create is to return a HTTP status of Created (201), with a url to get the newly created item. This library provides that feature, by linking into the ASP.NET Core’s CreatedAtRoute  method. Here is an example of this approach:

[HttpPost]
public ActionResult<CreateTodoDto> Post(CreateTodoDto item, 
    [FromServices]IActionService<ICreateTodoBizLogic> service)
{
    var result = service.RunBizAction<TodoItem>(item);
    return service.Status.Response(this, "GetSingleTodo",
         new { id = result?.Id },  item);
}

NOTE: I have to allow for the result to be null, which can happens if there are errors. Therefore I use result?.Id  (see line 7) to stop me getting a null reference in the error state (thanks to Michael Wilson for spotting that).

On a successful creation this method returns a response header containing the URL you should use to obtain the newly created item. Here is an example of the header, with the url on the third line:

content-type: application/json; charset=utf-8 
date: Thu, 30 Aug 2018 11:07:48 GMT 
location: http://localhost:54074/api/ToDo/7 
... other parts removed 

NOTE: There is one “gotcha” that I need to warn you about. In ASP.NET Core you need to provide the Name of action method that provides the GET method to return the crated item. The Name isn’t the name of the method, but the name provided via the HttpGet attribute, for instance [HttpGet(“{id}”, Name= “GetSingleTodo”)]. If you don’t do that you get the dreaded “No route matches the supplied values” error.

2. Create, and return the created item

The other way to handle a Create is to return the created item. This has the advantage of removing the second HTTP to get the data. The library allows you to do this, and provide the 201 status code if required. Here is an example of a create a new TodoItem, where I return the created item, with its primary key:

[HttpPost]
public ActionResult<WebApiMessageAndResult<TodoItem>> Post(
     CreateTodoDto item,
     [FromServices]IActionService<ICreateTodoBizLogic> service)
{
    var result = service.RunBizAction<TodoItem>(item);
    return service.Status.ResponseWithValidCode(result, 201);
}

NOTE: You will see I use the [FromServices] attribute to allow the dependency injection (DI) provider to inject a service as a parameter in the action. I find this more efficient than the normal constructor injection, as it means each action creates just the service it needs.

NoContent (HTTP 204) – when the requested data lookup returned null

In Web APIs if something like a Get(id) is used, and no data is found (i.e. its null) then then it returns a status code of NoContent (HTTP 204). Therefore, if that happens then you get a NoContent status with a message, but no results. Here is an example of the json that is returned.

{
  "message": "The Todo Item was not found."
}

UPDATE: In version 2.0.0 of the EfCore.GenericServices.AspNetCore library I returned the NotFound status (HTTP 404) because Microsoft used NoFound status for nulls. But following feedback from others, and looking at other documentation, I released version 3.0.0 which returns a NoContent (HTTP 204) return on a null result. Version 3.0.0 also has a more fully defined return types, which helps Swagger to properly describe the API.

If you want to set your own Success and SuccessWithNullResult status you can the ResponseWithValidCode method, e.g. ResponseWithValidCode (result, 201, 404). The second parameter is the status code to return on a successful, non-null result, while the third parameter (which is optional – it defaults to 204) is the status code to return on a successful, but null result.

Telling GenericServices that null is not an error

To make the null handler to work you need to set a configuration parameter when you set up the GenericServices library. This is because in MVC or Razor Pages applications not finding the data asked for is an error, but in a Web API it isn’t. Here is an example of how you would configure GenericServices for WebAPI in the Configuration method in the Startup class – see line 4 for the important configuration option

services.GenericServicesSimpleSetup<ExampleDbContext>(
    new GenericServicesConfig
    {
        NoErrorOnReadSingleNull = true 
    }
    ,Assembly.GetAssembly(typeof(ChangeNameDto)));

UPDATE: Now with unit test support

I’m working on a client’s project which has lots of Web APIs, and we want to check them, because we found errors could creep up if the dtos weren’t properly setup.  I therefore added a new set of extension methods to the EfCore.GenericService.AspNetCore project (look for version 2.0.0 or higher on NuGet) to help test these.

The new extension methods decode the HTTP IActionResult and ActionResult<T> into a form that allows you to access all the parts of the information you sent from your GenericService/GenericBizRunner methods. This makes unit testing or integration testing much easier to do.

Here is an example integration test taken from the IntegrationTestToDoController, which you can find in the the EfCore.GenericService.AspNetCore GitHub project.

[Fact]
public void TestPutNameOk()
{
    //SETUP
    var options = SqliteInMemory.CreateOptions<ExampleDbContext>();
    using (var context = new ExampleDbContext(options))
    {
        context.Database.EnsureCreated();
        context.SeedDatabase();

        var controller = new ToDoController();
        var utData = context.SetupSingleDtoAndEntities
             <ChangeNameDto>(_genericServiceConfig);
        var service = new CrudServices(context, utData.ConfigAndMapper);

        //ATTEMPT
        var dto = new ChangeNameDto()
        {
            Id = 2,
            Name = "Test",
        };
        var response = controller.PutName(dto, service);

        //VERIFY
        response.GetStatusCode().ShouldEqual(200);
        var rStatus = response.CopyToStatus();
        rStatus.IsValid.ShouldBeTrue(rStatus.GetAllErrors());
        rStatus.Message.ShouldEqual("Successfully updated the Todo Item");
    }
}

Looking through the lines of code:

  • Lines 5 to 9: I create a in-memory Sqlite database a seed it with some known data for the test.
  • Line 11: I create an instance of the Web API controller I want to test. Because I inject the services via a parameter on the action method call, then there aren’t any parameters on the Web API controller.
  • Lines 12 to 14: Here I create the service that the Web API action method needs. The code you see is for GenericServices, but you can find an example of setting up GenericBizRunner here.
  • Lines 17 to 22: I call my Web API method, with the required dto class.
  • Line 25: The GetStatusCode extension method gets the HTTP status code, which I then check.
  • Line 26: The CopyToStatus method turns the Web API IActionResult/ActionResult<T> response back into a IStatusGeneric/IStatusGeneric<T> status (it started in from, so its easy to convert it back). This gives you access to any Errors, the Message, and the Result (if it has one).
  • Line 27 and 28 I check the parts of the IStatusGeneric to see what was send

NOTE: I could have added a test of the database to check that the name of the TodoItem with primary key 2 was changed too. It depends on whether you have unit tested the underlying GenericService UpdateAndSave of this entity elsewhere.

The extensions method can be found in UnitTesting.ResponseDecoders class.

Conclusion

I have described the pattern and libraries that allow me to build good, testable Web API code quickly. Previous experience says that this approach makes me a much faster developer – I developed a MVC5 web application around the AdventureWorks 2012 Lite database in 10 days, which is quick (see http://complex.samplemvcwebapp.net/). Real applications that use Web APIs can have hundreds, if not thousands of Web API actions – saving even 25% development time on each Web API action could equate to a saving of many months of your time and the project’s time.

This article and the example Web API in the EfCore.GenericServices.AspNetCore GitHub repo give you a quick way to see the use of EfCore.GenericServices and I threw in an example of calling business logic via my EfCore.GenericBizRunner library (see the Post action which creates a new TodoItem) to build Web APIs.

The examples uses DDD-styled entity classes because I like the way it centralises the create/update code inside the entity class. Using DDD-styled entity classes is optional, as EfCore.GenericServices can work with normal EF Core entity classes as well, but I find it adds an extra level of the robustness of the final solution.

Do clone the the example code and look at the code and run the ExampleWebApi application – maybe using this approach will save you time too.

Thanks to the readers who left comments on the NotFround/NoContent argument! I have updated the EfCore.GenericServices.AspNetCore library to use NoContent, and better response definitions, which helps Swagger to define the API.

Happy coding!

Entity Framework Core – validating data and catching SQL errors

Last Updated: July 17, 2018 | Created: July 15, 2018

This article is about how to catch SQL errors, like unique index violations, and turning them into user-friendly error messages. This capability adds to the general area of validation of data being written to the database, especially around EF Core.

This article comes from needing this capability in my GenericServices and GenericBizRunner libraries while working on a project for a client. I also extended the design to handle concurrency issues in both libraries, as one of the users of my GenericServices library was asking for it.

TL;DR; – summary

When EF Core writes data out to the database it doesn’t validate that data (see Introduction to validation section for more on validation). However, a relational database will apply its own validation, such as checking that a unique index constraint hasn’t been violated, and will throw an exception if any constraint is breached.

The problem with database exception messages is that they aren’t user-friendly, and can reveal something about your database structure (which is a possible security breach), so you can’t show them directly to the user. This article describes how to capture these database exceptions in EF Core and turn them into user-friendly error messages.

Introduction to validation in EF

Note: if you know about data validation and how EF Core does (or doesn’t) validate data on save then you can skip this section.

Data validation in .NET is about checking that the data in a class fits within certain rules. Typical validation rules are things like the attribute [MaxLength(100)] on a string. Other more complex validation rules can be applied via the IValidatableObject Interface. Some of these attributes, like [MaxLength(…)], [Required], etc. are also used by EF to set the size, nullability etc. of columns in the database.

Any validation rules are typically checked at the front end, say in ASP.NET Core, but can also be checked when that data is saved to the database. The default state for EF6.x is that the data written to the database is validated, but in EF Core the data isn’t validated – the reasoning is that the data  its most likely been validated earlier, so leaving out validation makes the save quicker.

Personally, when I create data in a business logic I always validate the data, as no front-end checks have been applied and the business logic might get things wrong. This is why my GenericBizRunner library defaults to validating the data on save, while my GenericServices library, which works with the front-end CRUD (Create/Read/Update/Delete) accesses, defaults to not validating the data.

But as I said at the beginning, whether you validate the data or not the relational database will have some constraints that it will always apply – you can’t bypass them for the good reason that the database wants to keep its state valid at all times. If you breach a database constraint in EF6.x or EF Core you will get an exception.

How to capture a database exception in EF Core

When you call SaveChanges/SaveChangesAsync then a range of exceptions can occur, from EF Core caught issues like concurrency issues (DbUpdateConcurrencyException, to database issues like the example of the unique index violation (DbUpdateException). Here is an example of catching a DbUpdateException taken from my book “Entity Framework Core in Action” (section 10.7.3).

try
{
   _context.SaveChanges();
}
catch (DbUpdateException e)
{
   //This either returns a error string, or null if it can’t handle that error
   var error = CheckHandleError(e);
   if (error != null)
   {
      return error; //return the error string
   }
   throw; //couldn’t handle that error, so rethrow
}

In the case of the DbUpdateException has an inner exception, which is the actual database exception containing the information about the constraint violation. The type and content of this inner exception differs for each database type.

The other thing to note is that the calling method has to return either success, or one or more error messages.

My generalized SaveChangesExceptionHandler

For my libraries I wanted to generalise the approach to cover both a) any type of exception, b) any type of databases. The signature for the SaveChangesExceptionHandler is

Func<Exception, DbContext, IStatusGeneric>

The method takes in the exception to decode and returns either a status, which can have errors or a success message, or null if it doesn’t handle the error type it was given. I also supply the current DbContext, which you would need if you were trying to handle a DbUpdateConcurrencyException.

For my example I am going to show you the code which handles DbUpdateException, where I want to capture the SQL unique index violations and turn them into user-friendly error messages. It starts with the method that detects each type of error you want to handle, and then calling the appropriate method. It’s basically a sophisticated switch pattern.

public const int SqlServerViolationOfUniqueIndex = 2601;
public const int SqlServerViolationOfUniqueConstraint = 2627;

public static IStatusGeneric SaveChangesExceptionHandler
    (Exception e, DbContext context)
{
    var dbUpdateEx = e as DbUpdateException;
    var sqlEx = dbUpdateEx?.InnerException as SqlException;
    if (sqlEx == null)
    {
        //This is a DbUpdateException on a SQL database

       if (sqlEx.Number == SqlServerViolationOfUniqueIndex ||
           sqlEx.Number == SqlServerViolationOfUniqueConstraint)
       {
          //We have an error we can process
          var valError = UniqueErrorFormatter(sqlEx, dbUpdateEx.Entries);
          if (valError != null)
          {
              var status = new StatusGenericHandler();
              status.AddValidationResult(valError);
              return status;
          }
          //else check for other SQL errors
       }   
    }

    //add code to check for other types of exception you can handle
    …

    //otherwise exception wasn't handled, so return null
    return null;
} 

Having found the unique index errors I call a method to specifically handle that sort of error. I should say that I add a constraint/key name following the pattern ‘UniqueError_<EntityName>_<PropertyName> to any unique indexes or primary keys that I want to handle. This both gives me more info to show the user, and stops me trying to handle errors on properties I didn’t expect to have an error. Here is my UniqueErrorFormatter method.

private static readonly Regex UniqueConstraintRegex =
    new Regex("'UniqueError_([a-zA-Z0-9]*)_([a-zA-Z0-9]*)'", RegexOptions.Compiled);

public static ValidationResult UniqueErrorFormatter(SqlException ex, IReadOnlyList<EntityEntry> entitiesNotSaved)
{
    var message = ex.Errors[0].Message;
    var matches = UniqueConstraintRegex.Matches(message);

    if (matches.Count == 0)
        return null;

    //currently the entitiesNotSaved is empty for unique constraints - see https://github.com/aspnet/EntityFrameworkCore/issues/7829
    var entityDisplayName = entitiesNotSaved.Count == 1
        ? entitiesNotSaved.Single().Entity.GetType().GetNameForClass()
        : matches[0].Groups[1].Value;

    var returnError = "Cannot have a duplicate " +
                      matches[0].Groups[2].Value + " in " +
                      entityDisplayName + ".";

    var openingBadValue = message.IndexOf("(");
    if (openingBadValue > 0)
    {
        var dupPart = message.Substring(openingBadValue + 1,
            message.Length - openingBadValue - 3);
        returnError += $" Duplicate value was '{dupPart}'.";
    }

    return new ValidationResult(returnError, new[] {matches[0].Groups[2].Value});
}

There is some weird regex and decoding of the SQL error message, and that is because the error string isn’t that simple to decode. Here is an example error message:

Cannot insert duplicate key row in object ‘dbo.Books’ with unique index ‘UniqueError_Book_ISBN’. The duplicate key value is (9781617294563).

The UniqueErrorFormatter method reformats this into a message

Cannot have a duplicate ISBN in Book. Duplicate value was ‘ 9781617294563’.

Notice on lines 13 to 15 I try to find the entity class so that I can return a user friendly and localized name, but the name in the constraint/key name is pretty good (unless you have a TPH class).

The SaveChangesWithValidation code

The code above is called within my SaveChangedWithValidation method, which I show below. The method returns the IStatusGeneric interface I use in my libiraries. This contains either a success message if there were no errors, or a list of errors if problems were found.

public static IStatusGeneric SaveChangesWithValidation(this DbContext context, IGenericServicesConfig config)
{
    var status = context.ExecuteValidation();
    if (!status.IsValid) return status;

    context.ChangeTracker.AutoDetectChangesEnabled = false;
    try
    {
        context.SaveChanges();
    }
    catch (Exception e)
    {
        var exStatus = config?.SaveChangesExceptionHandler(e, context);
        if (exStatus == null) throw;       //error wasn't handled, so rethrow
        status.CombineStatuses(exStatus);
    }
    finally
    {
        context.ChangeTracker.AutoDetectChangesEnabled = true;
    }

    return status;
}

The ExecuteValidation code looks like this

private static IStatusGeneric ExecuteValidation(this DbContext context)
{
    var status = new StatusGenericHandler();
    foreach (var entry in
        context.ChangeTracker.Entries()
            .Where(e =>
                (e.State == EntityState.Added) ||
                (e.State == EntityState.Modified)))
    {
        var entity = entry.Entity;
        var valProvider = new ValidationDbContextServiceProvider(context);
        var valContext = new ValidationContext(entity, valProvider, null);
        var entityErrors = new List<ValidationResult>();
        if (!Validator.TryValidateObject(
            entity, valContext, entityErrors, true))
        {
            status.AddValidationResults(entityErrors);
        }
    }

    return status;
}

NOTE: You can find all of this code in the class SaveChangesExtensions in the GenericServices library.

Enabling this feature in my GenericServices and GenericBizRunner

You need to configure the libraries to take your error handler. GenericBizRunner is the simplest as it already defaults to validation on writes to the database. Here is the configuration code.

services.RegisterGenericBizRunnerBasic<MyDbContext>(
   new GenericBizRunnerConfig
{
    SaveChangesExceptionHandler = 
        GenericBizRunnerErrorHandler.SaveChangesExceptionHandler
});

The GenericServices is a bit more complex, as by default it doesn’t validate on saving to the database, so your error handler won’t be called. You can turn on validation on a case-by-case via the PerDtoConfig<TDto, TEntity> class, or turn on validation for all writes to the database via the global configuration, as shown below.

services.GenericServicesSimpleSetup<MyDbContext>(
new GenericServicesConfig
    {
        DtoAccessValidateOnSave = true,     //we use  Dto access for Create/Update
        DirectAccessValidateOnSave = true,  //And direct access for Delete
        SaveChangesExceptionHandler = GenericServiceErrorHandler.SaveChangesExceptionHandler
}, Assembly.GetAssembly(typeof(MyDto)));

Other things you could do with this

As I said earlier one of my users of the GenericServices library wanted to know if GenericServices could handle concurrency issues, and I had to say no. But, now with my generalised SaveChangesExceptionHandler both libraries can handle concurrency issues. I haven’t done this yet, but the steps would be:

  • In the ‘switch’ method you would detect that the exception was a DbUpdateConcurrencyException and the type of the entity it happened on. Then you would call a method designed to handle concurrency issues on that entity type.
  • In that method you would (try) to fix the issue. If you have then you need to call SaveChanges (within another try/catch in case the error happened again) to save the corrected update.
  • If successful you return a status of success, which the system would return to the user. But if there were errors it couldn’t handle it can report to the user that the process failed.

NOTE: that handling concurrency issues is quite complex and hard to test, so I’m not saying this is simple. But you can now do this in in my GenericServices and GenericBizRunner libraries.

NOTE: I cover this concurrency issues in detail in section 8.7 in my book – it takes 12 pages!

Conclusion

Catching SQL errors and turning them into user-friendly error messages is hard work, but in real applications its often the only way to ensure the system works. Otherwise the user will be presented with the rather unhelpful “There was an error” page. Certainly not acceptable in real applications.

I spent bit of my own time to generalised the error handler in my GenericServices and GenericBizRunner libraries to cover more possible exceptions and work with any database. Now you have access to this feature in both of my libraries, or you can copy my approach in your own applications.

I hope this helps you.

Happy coding!

ASP.NET Core – fast and automatic dependency injection setup

Last Updated: September 25, 2018 | Created: June 21, 2018

Microsoft’s documentation says “ASP.NET Core is designed from the ground up to support and leverage dependency injection”. It also says that “Dependency injection (DI) is a technique for achieving loose coupling between objects and their collaborators, or dependencies.” (read Martin Fowler’s article for the in-depth coverage of DI).

I have used DI for years and I love it – it allows me to compartmentalize my applications and have DI link everything up at run time. The only chore is making sure all my services are registered properly at startup. This article is about a library I build to automatically register classes in an assembly into Microsoft’s DI provider (known as a DI container).

TL;DR; – summary

This article introduces a small library called NetCore.AutoRegisterDi (NuGet link) that can automatically register classes with interfaces in an assembly into the built-in ASP.NET Core DI container. The reason I built this library is because ASP.NET Core’s DI container turns out to be quick – at least 5 times quicker than AutoFac. I also describe how I use this in applications that have a lot of assemblies.

Setting the scene

The typical approach shown in the Microsoft’s documentation to registering services is to add a line of code to register each service in the ConfigureServices method in ASP.NET Core’s Startup class. For me that is a horrible maintenance issue, because I might forget to register a service I need: and if I do forget then my application will fail when that service is used.

Until now my go-to solution for automatically registering my services in DI is AutoFac’s RegisterAssemblyTypes method. But when I saw David Fowler’s tweet with a link to a set of dependency Injection container benchmarks, I found out that the ASP.NET Core DI container is a lot faster than AutoFac.

I worry about performance, so I thought – how hard would it be to build a similar extension to AutoFac’s RegisterAssemblyTypes method, but for the NET Core DI container,  Microsoft.Extensions.DependencyInjection? Turns out it was easy (2 hours + 2 hours for NuGet+docs). This article is about how you can use the NetCore.AutoRegisterDi in your own applications.

NOTE: Microsoft docs says, “The default services container provided by ASP.NET Core provides a minimal feature set and isn’t intended to replace other containers.” However, I find Microsoft’s DI library has all the things I need for most applications, but some features aren’t well publicised – see Steve Gordon’s articles on three Microsoft DI features that I had to find the hard way:

Introduction to DI – what is a service?

Note: If you already understand DI you can skip this section.

I refer to a service, which is a combination of a class that implements that service, known as the implementation type in the NET Core DI container, which is linked (known as registered) to a service type, which is typically an interface. Here is an example of an interface, followed by the class that implements that interface.

public interface IMyService
{
    string IntToString(int num);
}

public class MyService : IMyService
{
    public string IntToString(int num)
    {
        return num.ToString();
    }
}

There are various ways to use a service in another class. The standard way is to add a parameter to the class’s construction and the DI container will resolve that interface to the implementation class – this is known as constructor injection. Here is an example.

public class UseService
{
    private readonly IMyService _service;

    public UseService(IMyService service)
    {
        _service = service;
    }

    public string CallService()
    {
        int i = 1; //... some calculation
        return _service.IntToString(i);
    }
}

This interface/class pattern “achieving loose coupling between objects and their collaborators” (to quote Microsoft). This has lots of benefits, but here are the two main ones for me:

  • The interface clearly defines the properties, methods, classes etc. that user of the service can access. By using an interface I can’t access things I’m not supposed to use, so interfaces makes me focus on what I should use and ignore the rest.
  • The two classes are “loosely coupled”, i.e. it is easy to replace the original version of the implementing class if it conforms to the interface. This is especially useful when unit testing, as it’s easy to replace the interface with a mock that allows you to test the UseService class on its own.

How to NetCore.AutoRegisterDi works

The NetCore.AutoRegisterDi provides three extension methods – here is an example that uses all three methods.

service.RegisterAssemblyPublicNonGenericClasses(… your assembly ref goes here …)
    .Where(x => x.Name.EndsWith(“Service”))
    .AsPublicImplementedInterfaces();
  1. The first line finds all the public, non-generic classes (see docs for detailed definition of what classes it finds) in the assembly you provide (or assemblies – it takes params Assembly[] assemblies).
  2. The Where method in the second line is optional and it allows you to filter the classes if you need to.
  3. The final line finds all the public, non-nested interfaces (apart from IDisposable) that each class implements and registers each interface against the class.

The AsPublicImplementedInterfaces method takes an optional parameter of the ServiceLifetime you want for the service. It defaults to Transient lifetime where a new service instance is created on every injection, which is a good default for services. The other options are Singleton (same instance used on every injection) or Scoped (a new service instance is created for each HTTP request).

TIP: The NetCore.AutoRegisterDi Where method is useful if you want different ServiceLifetime – I name my classes with an ending that tells me what sort of lifetime I need, and filter on that.

An approach to using AutoRegisterDi in real applications

Now, you could call the RegisterAssemblyPublicNonGenericClasses method directly in the ConfigureServices method in the Startup class, and it would work fine. You just need to define the assemblies you want to scan. Here is how you would do this.

public void ConfigureServices(IServiceCollection services)
{
   //... other configure code removed

   var assemblyToScan = Assembly.GetAssembly(typeof(YourClass)); //..or whatever assembly you need

   service.RegisterAssemblyPublicNonGenericClasses(assemblyToScan)
     .Where(c => c.Name.EndsWith("Service"))
     .AsPublicImplementedInterfaces();

That’s fine, but it’s not what I do. I create an extension method inside each key assembly which registers itself and any other assemblies that it ‘owns’. In a small applciation I would call my ServiceLayer assembly, and it would look after setting up all the other assemblies.

Here is an example taken from a larger application where I have multiple grouped assemblies (see Simon Brown’s package by component approach) .

public static void DevModStartup(this IServiceCollection services, 
    IConfiguration configuration)
{
    //This registers the service layer: I only register the classes who name ends with "Service" (at the moment)
    services.RegisterAssemblyPublicNonGenericClasses(
              Assembly.GetExecutingAssembly())
        .Where(c => c.Name.EndsWith("Service"))
        .AsPublicImplementedInterfaces();

    //Now I register the DevModBizLogic assembly used by DevModService
    services.RegisterAssemblyPublicNonGenericClasses(
             Assembly.GetAssembly(typeof(BuildTestTenant)))
        .AsPublicImplementedInterfaces();

    //Put any code here to initialise values from the configuration parameter
}

NOTE: This shows the two ways of getting an assembly. Assembly.GetExecutingAssembly() gets the assembly that is running and Assembly.GetAssembly(typeof(SomeType)) gets the assembly that the SomeType is defined in.

I then call this DevModStartup extension method in the ASP.NET Core ConfigureServices method like so.

public void ConfigureServices(IServiceCollection services)
{
   //... other configure code removed

   services.DevModStartup(Configuration);
   //... add other assembly startup extension methods here

I think this is cleaner, especially as it keeps the rules about what classes to scan and ServiceLifeTime in the assembly that knows what is going on.

You should also note that I provide the Configuration property to the Startup extension method. This gives the extension method access to the appsetting.json file from which it can read static values, like setting or crypto keys etc. on application startup and store them in internal static variables.

WARNING: I wouldn’t use RegisterAssemblyPublicNonGenericClasses with the ASP.NET Core assembly without a good Where clause. The ASP.NET Core assembly has a LOT of services which it sets up itself, so you don’t want to re-register those.

Conclusion

DI is a great tool and one that I use a lot. The problem is in real applications I can have many hundreds (maybe thousands) of DI services so I want the DI to a) be easy to set up and b) be fast. I fact I think not automatically registering your services is an anti-pattern because it’s so easy to forget to register a service, and your live application will fail when that service is used.

I used to use AutoFac’s RegisterAssemblyTypes method, but that was about the only feature I used. Now that I have the NetCore.AutoRegisterDi library I have an assembly scanning feature that works with ASP.NET Core’s built-in DI container, and it makes my DI faster.

PS. Andrew Lock recommends a DI library called Scrutor – see his article called “Using Scrutor to automatically register your services with the ASP.NET Core DI container“. That might fit your needs better than my library.

GenericServices: A library to provide CRUD front-end services from a EF Core database

Last Updated: September 3, 2018 | Created: April 13, 2018

This article is about a NuGet library designed to make building Create, Read,  Update and Delete (CRUD) web pages quicker to write. GenericServices acts as an adapter and command pattern between EF Core and your web/mobile/desktop application. This article describes why this is useful and how it can save you development time. My examples use ASP.NET Core Razor Pages, but it can work with any type of front-end application.

NOTE: This article is about using the GenericServices library with ASP.NET Core MVC or Razor pages. If you use ASP.NET Core’s Web API you might want to look at the article “How to write good, testable ASP.NET Core Web API code quickly“, which shows how to use GenericServices with Web API.

TL;DR – summary

The GenericServices library bridges the gap between the database and the front-end user: the database is designed around the business domain (say, e-commerce), while the used wants pages that is appropriate to their need (say, listing all the cookery books). The library implements a read/write Adapter pattern, and a Command pattern for writes to the database.

My experience with a previous library for EF6.x was it could save 2 months on a 12 month project.

Each CRUD page takes a little bit of code, but applications have lots and lots of such pages. Thankfully these pages have a set of common patterns, and GenericServices is built to implement these patterns – you just need to provide a ViewModel/DTO (Data Transfer Object) which contains the properties for the CRUD action you want to accomplish. (I use DTO in the rest of this article)

GenericServices feature are:

  • Easy to use: just mark your ViewModel/DTOs to tell GenericServices how to use it.
  • Reduces the amount of CRUD code you need to write – just a ViewModel/DTO and two calls.
  • Can work with standard EF Core entity classes and Domain-Driven Design (DDD) styled entity classes.
  • Contains good checking and security features, especially with DDD-styled entity classes.

NOTE The GenericServices library is open-source (MIT licence) on GitHub and is available on NuGet as EfCore.GenericServices. The code, including an example ASP.NET Core Razor Pages application, is available on GitHub here. There is extensive documentation in the project’s Wiki.

Who should read this article?

If you are building applications with many front-end CRUD displays, then the GenericServices library will help you develop these pages quickly. GenericServices turns each CRUD access into a common set of calls around your display-specific ViewModel/DTO (I will use DTO from now on for this class). That minimises the code you need to write, which saves you time.

You do need to be using EF Core for your database access, as GenericServices is designed around EF Core (the NuGet package is called EfCore.GenericServices). GenericServices is designed to work with EF Core entity classes (i.e. the classes mapped to the database) that use the standard style (properties are changed by setting the property) and Domain-Driven Design (DDD) styled entity classes, where data is updated via methods in the entity class.

To keep this article short, I assume you know C# and have some idea of what the ASP.NET Core framework does. The examples I use in this article use ASP.NET Core’s Razor Pages, but in the article “How to write good, testable ASP.NET Core Web API code quickly” I show how to use GenericServices with Web APIs.

The problem that GenericServices tackles

GenericServices covers CRUD displays especially in web/mobile applications. I’m going to take an “update of an entity” to describe the typical issues that come up. My example application is an e-commerce site selling technical books and I implement a feature where an authorised user can add a sales promotion to a book, by reducing its price. I will show you two forms: a hand-coded version and a version where I use GenericServices.

To set the scene let’s look at what my “add a promotion” feature on a ASP.NET Core web site. Here is the look at the display, which the user has filled in (in red), with comments on the left about the Book properties and how they are involved in the update.

In a web/mobile application a feature like this consists of two stages:

  1. The display of the data to the user where they can put in their changes (see figure above)
  2. Once the user presses the button (“Add” in this case) the information is sent back and the database update is done.

In the “Add promotion” example the first stage needs to read five properties from the database to show the user. In the second stage, triggered by the user pressing the “Add” button”, the primary key (BookId) is used to select the Book to be updated, and then the NewPrice and PromotionalText values are updated in the book.

NOTE: All my examples are synchronous, but GenericServices contains async versions of every command.

1. The hand-coded version

Let’s start by looking at a hand-coded implementation of a service in an ASP.NET Core application. This sets the standard in terms of lines of code, and performance (I compare the performance of the hand-coded version with the GenericServices version near the end of this article).

I start with the DTO, and then the code to read and update the promotion

public class AddPromotionDto
{
    [HiddenInput]
    public int BookId { get; set; }

    public decimal OrgPrice { get; set; }

    public string Title { get; set; }

    public decimal ActualPrice { get; set; }

    public string PromotionalText { get; set; }
}

And now the code to read in the data and then update the Book entity with the new promotion.

public class AddPromotionService : 
    StatusGenericHandler, IAddPromotionService
{
    private readonly EfCoreContext _context;

    public AddPromotionService(EfCoreContext context)
    {
        _context = context;
    }

    public AddPromotionDto GetOriginal(int id)      
    {
        var dto = _context.Books
            .Select(p => new AddPromotionDto
            {
                BookId = p.BookId,
                Title = p.Title,
                OrgPrice = p.OrgPrice,
                ActualPrice = p.ActualPrice,
                PromotionalText = p.PromotionalText
            })
            .SingleOrDefault(k => k.BookId == id);
        if (dto == null)
            AddError("Sorry, I could not find the book you were looking for.");
        return dto;
    }

    public Book AddPromotion(AddPromotionDto dto)
    {
        var book = _context.Find<Book>(dto.BookId);
        if (book == null)
        {
            AddError("Sorry, I could not find the book you were looking for.");
            return null;
        }
        //This is a standard entity class, where you update by setting the properties
        Book.ActualPrice = dto.ActualPrice;   
	 Book.PromotionalText = dto.PromotionalText;
        _context.SaveChanges();                 
        return book;
    }
}

2. The GenericServices version

Now let’s look at how GenericServices would do this. Firstly, we change the DTO slightly, as shown below.

public class AddPromotionDto : ILinkToEntity<Book>
{
    [HiddenInput]
    [ReadOnly(true)]
    public int BookId { get; set; }

    [ReadOnly(true)]
    public decimal OrgPrice { get; set; }

    [ReadOnly(true)]
    public string Title { get; set; }

    public decimal ActualPrice { get; set; }

    public string PromotionalText { get; set; }
}

Two things have changed:

  • GenericServices needs to know what entity class your DTO links to, which you do by adding a ILinkToEntity<TEntity> The interface is empty, i.e. you don’t have to add anything to your DTO – it’s just there for GenericServices to find the DTO and extract the entity class type from the interface.
  • You also need to tell GenericServices which properties are read-only, so it won’t copy them back to the database (that’s a security feature to stop malicious updates). You do this by adding the ReadOnly(true) attributes to the DTO properties that are read-only.

NOTE: Technically you don’t need ReadOnly(true) attributes if you have DDD-styled entity classes, as your entity class method will control access to the data inside the entity class (but I do add them as it allows GenericServices to better match the correct create/update method). But you must add the ReadOnly(true) attributes is you are using standard entity classes as it tells GenericServices what properties should be updated, and which ones should not be updated.

Now the new code for reading and then adding the promotion using GenericServices’ CrudServices.

public class GenericAddPromotionService 
{
    private readonly ICrudServices _service;

    public IStatusGeneric Status => _service;

    public GenericAddPromotionService(ICrudServices service)
    {
        _service = service;
    }

    public AddPromotionDto GetOriginal(int id)
    {
        return _service.ReadSingle<AddPromotionDto>(id);
    }

    public void AddPromotion(AddPromotionDto dto)
    {
        _service.UpdateAndSave(dto);
    }
}

In fact, there is little gain in wrapping the GenericServices’ calls like that and I tend to put the GenericServices directly in the front-end. Here is an example of a Razor page using

public class AddPromotionModel : PageModel
{
    private readonly ICrudServices _service;

    public AddPromotionModel(ICrudServices service)
    {
        _service = service;
    }

    [BindProperty]
    public AddPromotionDto Data { get; set; }

    public void OnGet(int id)
    {
        Data = _service.ReadSingle<AddPromotionDto>(id);
        if (!_service.IsValid)
        {
            _service.CopyErrorsToModelState(ModelState, Data, nameof(Data));
        }
    }

    public IActionResult OnPost()
    {
        if (!ModelState.IsValid)
        {
            return Page();
        }
        _service.UpdateAndSave(Data);
        if (_service.IsValid)
            return RedirectToPage("BookUpdated", new { message = _service.Message });

        //Error state
        _service.CopyErrorsToModelState(ModelState, Data, nameof(Data));
        return Page();
    }
}

What is interesting is the C# code for the hand-coded razor page is the same l length as the GenericServices razor page. This means, if use GenericServices directly in the Razor Page I have saved me from writing 38 lines of code (The AddPromotionService is 41 lines, but I needed to add three [ReadOnly(true)] attributes to the DTO).

But more than that, the Create, Read, Update and Delete patterns are same, apart from the DTO type (line 11). This means I could copy the AddPromotion PageModel code and paste it in a PageModel to say update the publication date of the book, and all I would need to change is the type of the DTO (obviously the razor .cshtml file needs to change – I can’t do that for you (yet)).

GenericServices saves you from having to write repetitive (and boring!) CRUD front-end code. So now you can concentrate on the more complex parts of the application.

For me even a small business application will have more than 50 or more CRUD views, which mean GenericServices can save me from writing many hundreds of lines of CRUD code. I know this because I build a similar library called GenericServices for EF6.x back in 2014 and I have used it on many applications. It has saved me many months of my life from writing CRUD code (and hopefully the same for the other people who downloaded it).

NOTE For anyone who has used my original GenericServices library then the new EfCore.GenericServices provides the same feature, but is built in a very different way. Partly that is because it is designed to handle DDD-styles entity classes, and partly from feedback about making things simpler. I hope you find this new library (more) useful to you, as the original GenericServices library.

How does GenericServices work inside?

GenericServices has commands that cover the Create, Read, Update and Delete database actions. And every command can work with either an entity class or a DTO with a ILinkToEntity<TEntity> interface (apart from Delete, which only works on an entity class). But my experience is that a large proportion of CRUD accesses use DTOs, either to select that is shown to the user or to limit what gets updated – in these situations GenericServices comes into its own.

How it read data from the database

For reading, e.g. ReadSingle<AddPromotionDto>(id), GenericServices uses AutoMapper’s ProjectTo method if it’s a DTO, or simple EF Core Find<TEntity>(id) if the class you provide is an entity class. AutoMapper is a great library with impressive performance (see later) and does a great job on building LINQ Select queries.

NOTE: It’s beyond the scope of this article, but you can alter the AutoMapper read mapping that GenericServices uses by creating a PerDtoConfig<TDto,TEntity> class and overriding the AlterReadMapping property. This means you can changed the default mapping on a per-DTO basis – see the Wiki documentation for more on this.

How it creates a new entity or updates an existing entity

First, let’s look are the create or update of an entity class, which is pretty simple as its already in the correct form.

  1. Create entity: It calls EF Core’s Add method and then SaveChanges
  2. Update entity: If the entity has an EF Core State of Detached then it will call Update, otherwise it assumes you have updated it and will call SaveChanges.

When it comes to create and update the process used depends on whether the entity class is standard style entity or DDD-styled entity. Personally, I really like DDD-styled entity classes but I can see some simple classes, say like entity holding an Address, are quicker to write as a standard style entity.

1. Create or update a standard style entity via a DTO

In this case it uses AutoMapper’s Map method to copy the DTO properties to the entity it has loaded. The default mapping will not copy properties that have the [ReadOnly(true)] attribute on them. Also, it won’t update a property that has a private setter.

NOTE Just like in the read case you have access to the AutoMapper save mapping by creating a PerDtoConfig<TDto,TEntity> class and overriding the AlterSaveMapping property.

2. Create or update a DDD-styled entity via a DTO

GenericServices looks for a constructor/method whose parameters’ name and type match the non-read-only properties. There are lots of features here, but let me give you an example from the AddPromotion example.

The Book entity class has a method called AddPromotion, which looks like this.

public IStatusGeneric AddPromotion(decimal actualPrice, string promotionalText)                  
{
    var status = new StatusGenericHandler();
    if (string.IsNullOrWhiteSpace(promotionalText))
    {
        status.AddError(
            "You must provide some text to go with the promotion.", 
            nameof(PromotionalText));
        return status;
    }

    ActualPrice = actualPrice;  
    PromotionalText = promotionalText;

    status.Message = $"The book's new price is ${actualPrice:F}.";

    return status; 
}

NOTE: In this case the update method returns a type of IStatusGeneric, which allows it to provide a status result (methods that don’t need a status can return void).

GenericServices default process is to find an update method where the method’s parameter’s name/types match the name/type of the non-read-only DTO properties (the name match handles camel/pascal case matching). In this case GenericServices finds the method called AddPromotion and calls that method.

NOTE: There are obviously some complicated situations here, like having two methods that match, or methods that don’t need a property. This documentation for create and update goes into all the matching processes and how to override them.

As you can see from the AddPromotion update method there are several benefits over setting properties in a standard styled entity. In this case I included some checks and returned a success message that could be shown to the user. There are many advantage to using DDD-styled entity with its create and update methods – I recommend you look at my article “Creating Domain-Driven Design entity classes with Entity Framework Core” for more on this.

What about the performance of GenericServices?

I am concerned about performance and I checked this regularly during development. Here are some timings produced by BenchmarkDotNet library (which I found excellent for performance testing). I used the fastest database I could find, which turned out to be SQLite in-memory database, so that the database part was minimised. You can find the performance tests in the project called Benchmarking.

The first example is for the simplest possible update – in this case I update the publication date of a book. I opened up the Book’s PublishedOn property so I could update it via AuthoMapper as well as via a DDD-styled update method.

Method Mean (us) Error (us) StdDev (us)
HandCoded, PropetyUpdate 530.9 4.003 3.343
GenericService, AutoMapper 551.4 5.624 5.261
4%
HandCoded, MethodCall 529.1 5.583 5.223
GenericServices, MethodCall 555.7 5.828 5.451
5%

 

This is about the fastest database update I could find, and the difference between hand-coded and GenericServices’ versions were only 20 to 25 us. (us = 1/1,000,000 of a second), which equates to 5% slower. Note also that calling a method is just as fast as setting the property (GenericServices builds and caches LINQ expressions to call the methods, which are very fast).

The second example is a more complex update, where it adds a new Review entity to the Book. This takes a little longer inside EF Core and the database provider.

Method Mean (us) Error (us) StdDev (us)
HandCoded, MethodCall 773.8 8.882 7.874
GenericServices, MethodCall 800.7 11.262 7.874
3%

 

This shows that the difference is still about 25 us, but because the database accesses take longer it now only equates to a 3% difference.

Finally, I do a complex Projection of 100 Book’s to provide the display of the user. This is using AutoMapper via GenericServices (the hand-coded version is a LINQ Select method with hand-coded mappings).

Method Mean (ms) Error (ms) StdDev (ms)
HandCoded 11.35 0.2433 0.4970
GenericServices 11.23 0.0838 0.0784
-1%

 

This is a bit misleading, as the GenericServices isn’t faster than the hand-coded – sometimes it comes out faster and sometimes slower. Basically, the performance cost of GenericServices is smaller than the error margin so you can’t really measure it.

How hard is it to set up GenericServices?

I have tried to make it simple to add GenericServices to an existing or new application.  There is full documentation on installing GenericServices, but here is an overview, assuming an ASP.NET Core application.

  1. You install the NuGet library EfCore.GenericServices in your ASP.NET Core project. If you hold your ViewModels/DTOs in another project, then you need to install EfCore.GenericServices in that project too.
    1. You might want to include the companion NuGet library called GenericServices.AspNetCore, which includes a method to copy errors from GenericServices into ASP.NET’s ModelState.
  2. You mark all your DTOs with the ILinkToEntity<TEntity> interface so that GenericServices can find them and work out what entity class they are linked to.
  3. You add a call to GenericServicesSimpleSetup<TContext> in the ConfigureService method in the Startup class of a ASP.NET Core application. This registers GenericServices with the DI – here is a link to the code in my Startup class showing what I did.
  4. You use the ICrudServices interface to inject an instance of the CrudServices into your controller or razor PageModel.

NOTE: There is other versions for registering GenericServices. Another simple one that takes a GenericServiceConfig and a more complex version to handle the case where you use multiple DbContexts to cover your database. There is also a non-DI version for registering one or two DTOs useful for unit testing or serverless applications (See this documentation).

Conclusion

I have described what the GenericServices library is designed to do – to save you time by building robust CRUD web pages. My experience with my previous library with the same name (but totally different design) for EF6.x is that it can really reduce the time it takes to build applications which have a lot of CRUD pages (see this example, which I build in 10 days using the old library).

The new EfCore.GenericServices library is designed to be easier to use and has several new features. The biggest new feature is its ability to work with standard entity classes, where you update via the properties, and DDD-styled entity classes, where you update via methods in the entity class. Personally, I have found DDD-styled entity classes are good for hiding some of the more complex rules for creating and updating entities and their aggregates (see my article that explains that).

Please do have a look at the library and see what you think. The GenericServices GitHub repo contains an example ASP.NET Core Razor Pages application – its uses an in-memory database so that it can run anywhere, and always starts at a known setting. I also have a similar example of using GenericServices  with Web APIs in this GitHub repo, which also uses a in-memory database so that its easy to run.

Happy coding.

Six things I learnt about using ASP.NET Core’s Razor Pages

Last Updated: April 13, 2018 | Created: April 9, 2018

ASP.NET Core 2.0 introduced a new way to build a web site, called Razor Pages. I was interested in the new Razor Pages approach, as I hoped Razor Pages would allow me to code better by following the SOLID principals – and they do. But first I had to work out how to use Razor Pages. This article my journey in learning how to use Razor Pages.

TL;DR – summary

ASP.NET Core Razor Pages make building web applications simpler, but they work differently to the existing controller->actions->views approach. This article about the things that have changed and what is the best way to use the new approach provided by Razor Pages.

My view of Razor Pages are they are a very good improvement! They produce better structured code, with a pattern that follows the Single Responsibility Principle (SRP), and you get better Separation of Concerns (SoC).

The aims of this article

This article will be useful to people who want to use ASP.NET Core razor pages. It looks at areas that I found that were different from the existing controller->actions->views approach that has been around for many years. The six areas I cover are:

  1. Replacing the action: How should I construct my Razor Page code part?
  2. Routing: Using folders and filenames, with a different handling to invalid URLs
  3. It’s getting messy – how do I segregate the “Home” razor views?
  4. Model Binding: What is the best way to get data in/out to a razor action method?
  5. Changed: The validation name in the View is a bit more complex
  6. How can I implement AJAX requests in razor pages?

This article assumes you know C# and have some idea of what the ASP.NET Core framework does. I do talk about the existing controller->actions->views approach, as a contrast, so its useful (but not essential) you know that. The code is focused on ASP.NET Core 2.0 or above, as that is the point where Razor Pages was introduced.

NOTE: I should have read the ASP.NET Core documentation page “Introduction to Razor Pages in ASP.NET Core” before I started. It has a great overview of what Razor Pages are and how to use them. Do have a look if you have never come across Razor Pages before.

Setting the scene – how razor pages changes the design of your web application

I have used ASP.NET Core and ASP.NET MVC for many years, using its controller->actions->views approach. I found that the controller class could get very messy inside, with a whole host of action methods jumbled together. I learnt to put the absolute minimum of code in each action method, both to keep the controller as simple and to allow the code to be tested outside the controller. Even so my controllers contains a mash-up of different actions handling different URLs – definitely not a nice structure (in my opinion).

Razor pages changes all this. The controller has gone, and the action methods move into their own, per-URL class that deals with all the HTTP requests (GET, PUT, POST etc) to one specific URL. The code is associated to the View file of the same name: the code provides the dynamic data and he view generates the HTML to show to the user.

That means Razor Pages applications are much cleaner and simpler to work with. The only problem I found was working out how to do all the things I used to do with controller and actions, but now with Razor pages. That took a bit of head scratching and experimenting, which is why I wrote this article.

Replacing the action: How should I construct my Razor Page code part?

If you create a Razor Pages application in Visual Studio or via the command line (dotnet new razor) then you get an ASP.NET Core application with three views, Index.cshtml, About.cshtml and Contact.cshtml. Each view has a file with the same name and extension, but with .cs on the end, e.g. Index.cshtml.cs, which contains the code. If we look at the About.cshtml.cs file its content looks like this.

public class AboutModel : PageModel
{
    public string Message { get; set; }

    public void OnGet()
    {
        Message = "Your application description page.";
    }
}

This threw me to start with, as action methods inside a controller must return a result – the same About action in a ASP.NET Core controller application would return a IActionResult result. Let’s understand what is happening here, and then we can see how this is helpful.

What Razor Pages does is return the class in the …cshtml.cs file, known as the PageModel. This contains various public properties and methods. In the About page case it returns the AboutModel – this class inherits from the PageModel class, and adds one public property called Message and an OnGet method. When a HTTP GET is sent to the URL /About then

  1. The OnGet method is called, which sets up the property Message.
  2. Then the associated About.cshtml razor view is executed to create the HTML. This has access to the public property Message in the AboutModel class via the @Model.Message.

The About case is very simple, but many requests can be handled with no return from the method. But there are other options. The first is you don’t have to have a PageModel file at all if the page contains static data, or its simple enough to handle using razor (enhanced) features. But I quite like the PageModel for the typical code you would have put in the controller’s action method.

The other case is where the OnGet (or other On… methods) needs to return a result. One case is if you want to redirect, or return a HTTP response such as 404 NotFound. Here is an example of a redirect.

public IActionResult OnGet(int id)
{
    Data = _context.Find<Book>(id);
    if (Data == null)
    {
        return RedirectToPage("MyError", 
              new { message = “I could not find that book” });
    }
}

The other case is if you are responding to a AJAX request and need to return data. I cover this in my AJAX article – use this link to go directly to the code that returns a JsonResult.

Routing: Using folders and filenames, with a different handling to invalid URLs

The first thing to say is Razor Pages route HTTP requests in a different way to the controller->actions->views approach. It uses the folder that the razor page is in as the first part of the URL, and the name of the file for the second part. So, a razor page called “PlaceOrder” in the folder “Pages/Checkout” would give a URL of Checkout/PlaceOrder.

There are a couple of rules that are similar to the controller->actions approach.

  • Any razor page in the top level of the “Pages” folder are accesses with no URL prefix. Bit like the HomeController.
  • Any razor page called “Index” is access without needing a URL prefix, same as with the controller->actions->views approach.

NOTE: Microsoft’s full documentation on routing and URL construction can be found here.

Now we need to talk about the different HTTP requests, like GET, POST, etc, and Async handling of requests. These are all done by the name of the methods in the PageModel file: methods starting with “On” then have the HTTP request type added, e.g. OnGet, OnPost, OnPut etc. Then, if you want async you need to add “Async” to the end of the name, e.g. OnGetAsync, and make the method async. These methods are known as handler methods.

NOTE: There is a way to have named handler methods – see Microsoft’s documentation on this. I also cover this in my article on AJAX and Razor Pages.

Warning: If you leave out the On… method, or misname it then the HTTP request won’t fail, but will just show the same view again. I mistyped the OnPost method as OnPut (I’m dyslexic) and chased around for an hour before I spotted it.

It’s getting messy – how do I segregate the “Home” razor views?

As I built my application I found I had more and more razor pages at the top level of the “Pages” directory (the “Home” directory in the controller->actions->views approach). The problem was that my “Home” pages were mixed in with the “Shared” pages like _Layout.cshtml, _ViewStart.cshtml etc., which made finding things more complex.

Thanks to the article “Setting a Custom Default Page in ASP.NET Core Razor Pages” by Matthew Jones I could move my razor pages into a subdirectory, which I called “Home” and then added this code you the Startup.cs file in my ASP.NET Core application

services.AddMvc().AddRazorPagesOptions(options =>
{
    options.Conventions.AddPageRoute("/Home/Index", "");
});

That means that on startup the top URL will go to my Home/Index page. Now my “Home” razor pages are easy to find in my application.

Model Binding: What is the best way to get data in/out to a razor action method?

Model binding is what happens when you receive a HTTP request with information in the url, header, form etc. Model binding is a big topic and I’m not going to cover it all, but with Razor Pages applications you get an interesting alternative to binding in the handler method parameters.

I’m going to show you the default binding approach to use with forms, which adds the [BindProperty] attribute to the properties in the PageModel. But first let me explain the model binding that we are all used to from the controller->actions->views approach. Here is very simple example of method parameter binding taking in the a ViewModel which has the data from the form

public MyFormViewModel Data { get; set; }

public IActionResult OnPost(MyFormViewModel formInput)
{
    if (!ModelState.IsValid)
    {
        return Page();
    } 
        //… do something with the formInput
}

That would work with razor pages, but what happens if the ModelState isn’t valid – it redisplays the form, but the data provided by the user is lost! That is because the “Data” property is empty. So, the correct way to do it is like this

[BindProperty]
public MyFormViewModel Data { get; set; }

public IActionResult OnPost()
{
    if (!ModelState.IsValid)
    {
        return Page();
    } 
        //… do something with the Data
}

The [BindProperty] attribute (line 1 in the code above) says that when a POST HTTP request comes in then it will try and bind the data in the property(s) that have the [BindProperty] attribute. So, if the form data doesn’t pass the validation, i.e. ModelState.IsValid is false, then you can return the page and the data will still be there. That is very nice!

For a more in-depth look at form input to razor pages I recommend this page on model binding on the excellent www.learnrazorpages.com site.

Changed: The validation name in the View is a bit more complex

I haven’t shown you a .cshtml page yet, so let’s look at one with a form to update the title of a book.

@page
@using Home
@model Home.ChangeTitleModel
@{
    ViewData["Title"] = "ChangeTitle";
}


<h2>Change book title</h2>


<div class="form-horizontal">
    
<form method="post">
        @Html.ValidationSummary(false, null, new { @class = "text-danger h4" })
        <input name="Data.BookId" type="hidden" value="@Model.Data?.BookId" />
      
<div class="form-group">           
<div class="col-sm-2">
                <label class="control-label">Book Title</label>
            </div>
          
<div class="col-sm-10">
                <input asp-for="Data.Title" class="form-control col-sm-10" />
                <span asp-validation-for="Data.Title" class="text-danger"></span>
            </div>
        </div>
       
<div class="form-group">
            
<div class="col-md-offset-2 col-md-10">
                <button type="submit">Update</button>
            </div>

        </div>
    </form>
</div>

Now, if you look at the that says asp-validation-for, then you will see that its set to “Data.Title”. When using ViewModels (which I do quite often) the name has a prefix of whatever you call your property – in my case “Data”.

The problem comes if you do your own validation tests somewhere else in your application, as you need to add the name of your property as a prefix. I do validation in my business logic and I therefore must prefix the property name with “Data.” (or the name of your property) when transferring the error to the ASP.NET ModelState. If you don’t you get a “hidden error”, where the page is redisplayed, but no error message is shown. This is very annoying to users!

Here is a very simple (naïve) approach for copying ValidationResult errors to the ModelState and adding the prefix.

var prefix = “Data.”;
foreach (var error in service.Errors)
{
    var properties = error.MemberNames.ToList();
    ModelState.AddModelError(properties.Any() 
        ? prefix + properties.First() 
        : "", 
        error.ErrorMessage);
}

NOTE: This naïve approach works, but you need to careful that the property names that your business logic correspond to actual properties in the ViewModel you are using, otherwise your error will still not be seen. I have a more sophisticated version here that checks that the MemberNames match a property in my ViewModel, otherwise I provide no name, which means the error is shown in the ValidationSummary part.

How can I implement AJAX requests in razor pages?

My application uses AJAX to fill a user-selectable filter, and I wasn’t sure how to do that in a Razor Pages application. There were some suggestions out there, but none of them were quite as optimal as I wanted.

I wrote a long article called “ASP.NET Core Razor Pages: How to implement AJAX requests” where I describe the whole process, but in summary I added another razor page where the handler method returns a JsonResult. It works quite well, and continues the use of a SOLID software approach. If you need AJAX then have a look at the article.

Further room for thought

In the controller->actions->views design I made the action methods in my controller as simple I possibly could, and moved all the real code into a project I call a “Service Layer” (see the point 2 in my “Six ways to build better Entity Framework (Core and EF6) applications” article to see what I mean by the Service Layer).

With Razor Pages, which isolate the code of each view, I find myself want to put more code in the PageModel class. The only problem then is – can I unit test it? Unit testing of ASP.NET Core is supposed to be easier, but the current approach isn’t simple. But Scott Hanselman has highlighted a new package coming out with ASP.NET Core 2.1 release called Microsoft.AspNetCore.Mvc.Testing, which looks very promising.

If that turns out to make unit testing of razor pages easier, then I think I will migrate much of the code that lives in my service layer into the razor pages handler methods. That would mean I could put more complex code in my razor pages and still be able to unit test the code. I look forward to trying that out.

NOTE: I am also working on a library to make building ASP.NET Core pages that access the database quicker to implement. Follow me on twitter @thereformedprog, or follow this technical blog site to be informed when something is available.

Conclusion

I am delighted by the cleaner approach that Razor Pages provides over the other approach, i.e. controller->actions->views. For human-focused applications I think Razor Pages is a good solution, but ASP.NET Core with swagger is most likely a better solution for a full-on WebAPI application.

Sure, there is bit of learning to do to find the right ways to use Razor Pages, but the effort is rewarded with a cleaner and more refactorable web/mobile application. I hope these points help you in understanding and adopting Razor Pages.

Happy coding.

ASP.NET Core Razor Pages: How to implement AJAX requests

Last Updated: May 1, 2018 | Created: April 2, 2018

ASP.NET Core 2.0 introduced a new way to build a web site, called Razor Pages. I really like Razor Pages, but I needed to work out how to do a few things. In this article I describe how to handle AJAX requests. It turns out there are two ways to implement the C# code that handles HTTP requests that return json or other data.

TL;DR – summary

ASP.NET Core’s Razor Pages normally deliver HTML pages, but there is still the need to deliver data for AJAX requests. This can be done in two ways: via what is known as a named page handler, or by using a normal razor page.

I describe two approaches, but I think the second is better. I also cover how to handle HTTP POST requests via AJAX, as there is need to add the anti-forgery token to the request.

Who should read this

This article is focused on how to handle AJAX requests in Razor Pages, so if that is your thing then this is useful to you.

To keep this article short, I assume you know C# and have some idea of what the ASP.NET Core framework does. I do give a very brief introduction to Razor Pages, but there are much better overviews elsewhere.

The code in this article is focused on ASP.NET Core 2.0 or above, as that is the point where Razor Pages was introduced.

NOTE ASP.NET Core documentation page “Introduction to Razor Pages in ASP.NET Core” is a great overview of what Razor Pages. The site https://www.learnrazorpages.com/ is also a good resource to look at.

A very brief introduction to Razor Pages

The classic way to deliver web pages in ASP.NET Core and ASP.NET MVC is via the controller->actions->views approach. The controllers contained many methods (known as actions) which dealt with specific parts of the URL. For instance, the URL /Orders/Index would run the method called Index in the controller called OrdersController.

Razor pages provides a different, simpler (better!) approach. The controller has gone, and the action methods move into their own, per-URL class in a directory. For instance, with Razor Pages, the URL /Orders/Index would look for a Razor Pages view named Index in the directory called Orders.

The Razor Pages view for the /Order/Index consists of two files:

  1. Order/Index.cshtml – the Razor Pages’ view that converts to HTML (must be there)
  2. Order/Index.cshtml.cs – the Razor Pages’ PageModel, which contains the code (optional)

The PageModel file is optional, but if you have any amount of C# code then this file is the place to put it. The only reason for not having a PageModel file is if your page static (i.e. no dynamic data), or the code is so simple you can write it using razor syntax. (Razor syntax is quite powerful now, with functions etc, but I still think the PageModel file is the right place for code to put the was typically written in the controller’s action methods before Razor Pages came alone).

What I really like about Razor Pages is they follow the Single Responsibility Principle (SRP) in that the two files deal with a specific URL and nothing else – they are very focused and meet the S.O.L.I.D. principles  of software design.

Here are two example files that implement the About page in the default Razor Pages application. This shows how they focus on one thing – the About page.

About.cshtml

 
@page 
@model AboutModel 
@{ 
    ViewData["Title"] = "About"; 
}

<h2>@ViewData["Title"]</h2>


<h3>@Model.Message</h3>


Use this area to provide additional information.

About.cshtml.cs

using Microsoft.AspNetCore.Mvc.RazorPages;

namespace RazorPageApp.Pages
{
    public class AboutModel : PageModel
    {
        public string Message { get; set; }

        public void OnGet()
        {
            Message = "Your application description page.";
        }
    }
}

The problem I am trying to solve – AJAX requests

In any web application you are bound to come across places where you want to fill in or change some data on the page without doing a whole refresh (see this intro to AJAX). AJAX really helps in making a fast, user-friendly web interface.

In my case I had an application with a filter where the user can choose how they wanted to filter a list of books (see the live http://efcoreinaction.com/ example site). Once the user has selected the filter type I use an AJAX request to get the values to put in the “Filter By” dropdownlist (greyed out in the figure below because the user hasn’t yet selected the filter type).

I know how to do this in the classic controller->action->view approach (you add another action to your controller), but how do you do it with Razor Pages?

It turns out there are two ways, and I will describe both (hint: but I like the second way more!). Then I will look at dealing with AJAX requests that use a HTTP POST, as there are extra security issues to handle.

NOTE that my examples work for AJAX but they would also work for implementing WebAPI for say for an external application. But ASP.NET Core with swagger is most likely a better solution for full-on WebAPIs.

Solution 1: Using named page handlers

In a typical PageModel file you have named methods called OnGet, OnPut, OnGetAsync etc. These deal with the different types of HTTP request to the URL defined by the folder the Razor Page is in, and the name of the Razor Page. But there is a way to define a call to a method outside of this naming convention, called named page handlers. The example given by Microsoft is providing different options for a particular page, say two different ways to register a user. But you can use it for AJAX calls too.

The default format for handlers is to add “?handler=YourMethodName” to the URL. So my first solution was to alter my JQuery ajax call as shown below:

$.ajax({
   url: '/?handler=Filter',
   data: {
       FilterBy: filterByValue
   }
})
.done(function(result) {
   //… rest of code left out

Then I changed my Index.cshtml.cs file to look like this

public class IndexModel : PageModel
{
    private readonly IListBooksService _listService;
    private readonly IBookFilterDropdownService _filterService;

    public IndexModel(IListBooksService listService, 
        IBookFilterDropdownService filterService)
    {
        _listService = listService;
        _filterService = filterService;
    }

    public SortFilterPageOptions SortFilterPageData { get; private set; }
    public IEnumerable<BookListDto> BooksList { get; private set; }

    public void OnGet(SortFilterPageOptions options)
    {
        BooksList = _listService
            .SortFilterPage(options)
            .ToList();

        SortFilterPageData = options;
    }

    public JsonResult OnGetFilter(BooksFilterBy filterBy)
    {
        return new JsonResult(_filterService.GetFilterDropDownValues(filterBy));
    }
}

The last four lines (the OnGetFilter method) are the ones to focus on. This is the named page handler with the name Filter, and I am picking up a HTTP GET request (I talk about HTTP POST AJAX request later).

This works fine, but it has made my Index PageModel not follow the Single Responsibility Principle, in that its dealing with two types of request. Hence, I looked for a more SOLID design, which I cover next

Solution 2: A dedicated Razor Page just for the AJAX request

While a razor page normally displays HTML there is no reason why its handles methods can’t return json, or any other data type. I therefore made myself a dedicated Razor Page for my filter. Here is the code:

Filter.cshtml

This is the razor page

@page
@model RazorPageApp.Pages.FilterModel
@{
    ViewData["Title"] = "Filter";
}
<h2>Filter</h2>

You must have a .cshtml file, as this sets up the routing. By trial and error, I found you must have the first two lines, but you don’t need the rest.

 Filter.cshtml.cs

The code now just contains the filter code, which is much clearer.

public class FilterModel : PageModel
{
    private readonly IBookFilterDropdownService _filterService;

    public FilterModel(IBookFilterDropdownService filterService)
    {
        _filterService = filterService;
    }

    public JsonResult OnGet(BooksFilterBy filterBy)
    {
        return new JsonResult(_filterService.GetFilterDropDownValues(filterBy));
    }
}

And my ajax call now has a url of  ‘/Filter’, as seen below

$.ajax({
   url: '/Filter',
   data: {
       FilterBy: filterByValue
   }
})
.done(function(result) {
   //… rest of code left out

I think this is much more SOLID. The filter has its own PageModel which just deals with the filter and my Index PageModel isn’t ‘polluted’ with the AJAX filter code. I do need a Filter.cshtml to define the routing, but its pretty simple (and the ASP.NET Core default .cshtml is fine). This is my chosen solution because its much simpler and easy to change without any side-effects.

But I’m not finished yet, because we need to handle POST requests with the XSRF/CSRF security feature.

How to handle POST requests with the XSRF/CSRF security feature

Razor Pages provide an anti-forgery token/verification to stop possible security issues in HTTP POST page requests, such as forms (hurrah! – safe by default). The anti-forgery token (you may know it as the [ValidateAntiForgeryToken] attribute) stops cross-site request forgery (known as XSRF or CSRF).

However, for any AJAX POST we must provide the anti-forgery token ourselves. There are two parts to this:

1. Make sure the anti-forgery token is in the page where your AXJAX request comes from

Razor will create right token, if you ask it to, but there is no default for AJAX. Therefore, you need to do something to get razor to add the anti-forgery token. The easiest is to use the Html helper @Html.AntiForgeryToken(), which will add the token. But if you already have a form tag which contains method=”post” then razor will have already added the anti-forgery token.

In my case I had a form, but it used the default HTTP GET, so I had to add the  @Html.AntiForgeryToken() code to my razor page.

NOTE: Microsoft documentation on anti-forgery token in JavaScript shows another way, by adding a function, but I am used to getting the value using JQuery. Either works, but I show the JQuery way.

2. Add the anti-forgery token to your request data

You need to add the anti-forgery token to your JQuery AJAX request. The default name that ASP.NET Core is looking for is RequestVerificationToken in the request header, and the value should be the anti-forgery token. Here is my modified JavaScript to add this.

$.ajax({
   url: '/Filter',
   type: 'POST',
   data: {
       FilterBy: filterByValue
   },
   headers: {
       RequestVerificationToken: 
           $('input:hidden[name="__RequestVerificationToken"]').val()
})
.done(function(result) {
   //… rest of code left out

My approach uses JQuery to find the hidden anti-token input tag and extract the value to send back to ASP.NET Core. Having done that your AJAX POST will work – if you got either of those wrong you will get a status 400 on your POST AJAX request.

Handling json data in POST

One of my readers,

NOTE: See Andrew Lock’s detailed article “Model binding JSON POSTs in ASP.NET Core“, which explains why this is needed, and some more details on how to use [FromBody]. Its worth reading.

Typically json returns are used to return more complex objects. In this example I use a Person class, taken from Andrew Lock’s article.  The following JavaScript code shows the ajax call that return a json content type – note that the contentType is set to json and the data has to be JSON.stringified to work.

var person = {"FirstName":"Andrew","LastName":"Lock","Age":"31"};
$.ajax({
   url: '/People',
   type = 'POST',
   contentType = 'application/json; charset=utf-8',
   headers = {
      RequestVerificationToken: 
          $('input:hidden[name="__RequestVerificationToken"]').val()
   },
   data: JSON.stringify(person)
})
.done(function(result) {
   //… rest of code left out

The json data will be returned into a class, in this case a Person class, as shown below.

public class Person  
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public int Age { get; set; }
}

And here is a simple Razor PageModel that takes the json data and binds it to a new instance of that Person class.

public class PeopleModel : PageModel
{
    public void OnPost([FromBody]Person person)
    {
        //do something with the person class
    }
}

NOTE: The Microsoft Binding documentation points out that you can only have one property or parameter with the [FromBody] attribute, as the binding reads the input stream. This means you can’t have both a property and a method parameter with the [FromBody] – if you do the parameter will be null.

Conclusion

It took me a little while to work out the best way to handle AJAX requests in Razor Pages – I even learn a few more things writing this article! But now I have an approach that works, and it follows the SOLID software principles.

I really like Razor Pages, mainly because that separate each page into its own, singled-focused Razor Page (the structure of the class controllers, with all actions mixed together, was messy and hard to work with). I may be writing more about Razor Pages in the future, so keep your eye out for more.

Happy coding.