Is the Repository pattern useful with Entity Framework? – part 2

This is a series:

1. Part 1: Analysing whether Repository pattern useful with Entity Framework
2. Part 2: Four months on – my solution to replacing the Repository pattern (this article).
3. UPDATE (2018): Big re-write to take into account Entity Framework Core, and further learning.

Four months ago I was reviewing the design of Spatial Modeller™ , version one and I come across people who said that the Repository/Unit of Work pattern was not useful with Microsoft’s main data access technology, Entity Framework (shortened to EF from now on).

See items marked with ‘UPDATED 2015‘ for new comments added eight month on from first article.

And ‘UPDATED 2017′ for new comments added from my book Entity Framework Core in Action, which I am writing for Manning Publishing.

At that time I wrote a the blog post called ‘Is the Repository pattern useful with Entity Framework? – part 1‘ where I looked critically about what people said. At the time I was fairly convinced, but I needed to write some real code to see how I would do things without a repository.

Well, four months on the answer is – I’m convinced. The Repository/Unit or Work pattern does NOT add anything to EF. In fact it hides some useful features that a repository has trouble using. (UPDATE 2015: eight months on I am even more convinced as I have now built some more representative applications).

However the issue is – what replaces the good bits of the repository, like handling Domain-Driven Design (DDD) which tries to treat the data persistence library as a simple in-memory collection. This means we want to hide some of EFs code, but still be able to use all of its features. It has taken me a little while to refine my approach, but here are my thoughts so far.

One size does not fit all

Design 1: CRUD data access for User Interface layer and WebApi

The front-end of any application needs to do CRUD operations: that is Create, Read, Update and Delete of entries. Create and Update can be particularly difficult as it might need other data to guide the user. For instance on my example site the create/update of a Post needs the user to select the right Tags and, in my example, the Author (try http://samplemvcwebapp.net/Posts and then edit an entry). That needs extra data, which is where Data Transfer Objects (DTOs) come in.

I am a strong proponent of DTOs I have used them heavily and I find they really help decouple the data and business layer from the UI or WebApi. DTOs are also good at dealing with the issue of Aggregates, that is a class that has subclasses under it. For instance you might have a class called Order, which has collection of LineItems under it, and most likely a link to a Customer. DTOs can extract all the various parts and either ‘flatten’ them into one class, or for collections extract only the parts of the data that the user wants/is allowed to see.

The important point is to design the transfer for reads from the database so that they turn into efficient SQL requests. This needs a bit of thought, but isn’t that hard to do. By using LINQ and IQueryable<T> requests then this allows paging and filtering to be added at the last minute in the UI/WebApi end. Also adding the .Select function allows only the columns/properties you need to be read in. The benefit of all this is that the SQL request only asks for those rows and properties that the request needs, which makes the request as efficient as possible.

So an approach that uses DTO is good. The only down side is that writing the data access code for DTOs is repetitive and boring. I therefore looked at how to automate this. As I said in part 1 of this post I thought I might use T4 code generation. However in the end I pursued a route using .NET Generics. The result is an Open Source library called GenericServices which provides List, Detail, Create, Update and Delete functions on either the EF data classes themselves or more interestingly via DTOs linked to EF data classes.

Design 1: Implementation (UPDATED 2015)

Rather than describe how this all works here I have build two example web sites.

1. SampleMvcWebApp.net, which is simple application to show GenericServices in action.

2. (UPDATE 2015) Complex.SampleMvcWebApp, which is a much more realistic example of a real application. The interesting thing for me is that it only took me two weeks to write the whole application (well, plus a few days for adding supporting comments and a few tweaks to improve it). That is way faster than my previous applications and proved to me the power of GenericServices.

I have written two articles that describe the design of the Complex.SampleMvcWebApp site. They are:

• Part 1: Using Entity Framework with an Existing Database: Data Access describes handling existing SQL databases using Entity Framework.
• Part 2: Using Entity Framework with an Existing Database: User Interface describes the overall architecture of the system.

(UPDATE 2015) The GenericServices Wiki now contains lots of information  and GenericService is now available on Nuget – see GenericServices on NuGet. This means you can try GenericServices yourself.

Design 2: The Business Layer (UPDATED 2015)

UPDATE: I now have a improved approach to handling business logic which you can find in the article Architecture of Business Layer working with Entity Framework (Core and v6) – revisited and in Chapter 4 of the book I am writing for Manning Publishing, Entity Framework Core in Action.

For the business layer I have taken a different approach. From a design/architecture point of view I do three things.

1. I allow the business layer to directly access the data via Entity Framework (EF). No repository, not unit of work, just good old EF. This means that I have access to all of EFs commands and nothing is hidden from me.
2. Problem is that EF on its own can be a bit of a handful so I tame it with ‘Query Objects’. Mainly I want to make repetitive, or complicated EF code appear as simple methods with intention revealing names. See implementation section for an example of this.
3. In the business layer I try to treat the database as an in-memory collection which some search procedures, i.e. Query Objects. I try to not use the data access methods other than for loading or inserting new entries. Any other commands are hidden in Query Objects or other methods.

Let me explain how this works by talking about the implementation.

Design 2: Implementation

So the implementation of using EF in the business layers consists of four parts:

1. Building ‘Query Objects’ to hide some of the complex filtering/loading

UPDATE 2017: In chapter 2 of the book Entity Framework Core in Action I describe building a book listing with sorting, filtering and paging using a series query objects. You can see a live example site showing you those query objects in action.

In the first post I mentioned that people were taking about Query Objects, which is an encapsulation of particular database queries or updates. In C# Extension methods are a great tool for building Query Objects.  If you have seen LINQ fluent commands then that is how extension methods work. Below is an example from Spatial Modeller™ where it loads all the schemes who’s key is in the array parameter, with the schemes layers, members, attributes and services.

public static IQueryable<SmScheme> GetSchemesByIdIncludingAllChildClasses
    ( this SpatialModellerDb db, ICollection<int> schemeIdsToLoad)
{
    return db.SmSchemes
        .Include(x => x.Grades)
        .Include(
            x => x.Layers.Select(y => y.Members
                              .Select(z => z.Attributes)))
        .Include(
            x =>
                x.Layers.Select(y => y.Members
                               .Select(z => z.ServicesHandled)))
        .Where(
            x => schemeIdsToLoad.Contains(x.SmSchemeId));
}

This code hides the EF code needed to perform the database action inside a nicely named method. This code would be in the data/infrastructure layer, which means the EF code is more centralised in one place. I would recommend Edward Charbeneau‘s article ‘Giving Clarity to LINQ Queries by Extending Expressions‘ which has some excellent ideas and source code for building ‘query objects’.

2. Using Domain-Driven Design (DDD) principals

I really like the thinking behind DDD. It has made me a better programmer. While DDD accepts that the way data is stored/persisted will have an affect on the design (see Eric Evans book, page 159) the aim is to allow Business Layer code treat the database as an in-memory collection which some search procedures, with as little interaction as possible with the data access code.

This isn’t done just to hide the database, but mainly to ensure that the developers working in the business layer are thinking about the business problem first, not how the database works. DDD recognises that there might need to be some implementation compromises to work with the database, but the overall aim is to keep the focus on the business problems.

The other issue that effects the approach used is that the actual data saved in the database will be designed for the business layer. Therefore the business layer will use the data classes directly, as they are a good fit to the business problem.

Update after a lot more business layer work

I still use a DDD approach, but I have found that EF doesn’t really support a pure DDD design for a number of reasons. Firstly the EF database classes need to be at the data layer level, so these classes aren’t really the right place to have business logic (I have a separate layer/assembly for that). Also EF sets certain restrictions on the way collections are handled, which means you can’t use a IReadonlyCollection, which can make building a DDD root/aggregates approach where the sub classes of a DDD root can only be changed via the root hard to enforce. Because of this I still follow a DDD design approach, but some of the more stringent access rules have to be implemented simply by setting standards, not by programatically enforces rules.

Even so my business layer is able use the DDD principal of treating the database as just classes, which make the business logic much simpler. To do this a typical business pattern has three phases:

1. Read in any data it needs for the business logic.
2. Some pure business logic working on the data classes – the bulk of the code.
3. An optional add/remove phase if any new data needs to be added to the database (note: updates happen automatically if you just change the database classes read in in phase 1).

3. The Business Layer never calls EF’s .SaveChanges() to persist data

The business layer should not know about persisting/updating data. I therefore design the business layer call .add() or .remove() on the EF collections, but the final update is triggered by the Service Layer after the business process returns.

This sounds subtle, but it has proved to be really useful. I go into this in more detail in my new article  Architecture of Business Layer working with Entity Framework but here are three good reasons why it helps.

• The single big commit meant that the modelled data was either there or not – no cleaning up if the business layer had done part of the job and committed it but a later part failed.
• The Business Layer does not have to handle database validation errors, which can occur when ‘SaveChanges’ is called. The Service Layer does that, which is better placed to deal with any errors.
• The business layer was easier to test as it added data to the EF collections, but didn’t change the database.