This post is an appendix to an article I am writing for the Simple-Talk blog site. It contains the detailed results of side-by-side comparisons of sync and async versions of the Entity Framework (EF) 6 data accesses. This shows that async EF commands are not that much slower than normal sync commands.
I am writing a article called ‘The .NET 4.5 async/await Commands in Promise and Practice‘ for the Simple-Talk blog. In this I look at the whole area of the async/await commands for tasking.
In putting this article together I ran extensive side-by-side comparisons of normal, synchronous Entity Framework (EF) commands and the new async versions of the same commands. I found some surprising results which I think others might be interested in.
The Simple-Talk article wasn’t the right place to put all this technical detail so I have written this blog as an appendix for those of you that would like to drill-down into the detail. I think you will find the results interesting.
Overview of the test database
I am building a .NET library called GenericServices (see previous blog post on this) and as the test database I have a simple blog site with three classes in it: Blog, Post and Tag. The diagram below shows the data classes and their relationships.
Most of the tests are done on the Post class, which is the more complex to update and deleting it does not cause cascade deletes. Also, for update I assume a disconnected update, i.e. as would happen on a web site, where the original Post is read and then specific properties are updated before they are written back to the database.
Test 1. Raw EF performance
The first test results I will share with you is the raw speed of the sync and async version of the EF commands. This was done inside an NUnit test running inside Visual Studio 2013 on my development PC which has an Intel i7 processor clocked at 3.31GHz. The database is localDb running SQL server 2012. I ran 1000 tests on a database filled with 1000 entres and averaged out the time for each command.
You can see the unit test code here (I used method Perf21NCompareEfSyncAndAsync1000Ok around line 61) and the actual Ef commands here (note: GenericServices is an open-source project).
Let me list the results first and then draw some conclusions from this.
| Sync (ms) | Async (ms) | Diff | Notes | ||
|---|---|---|---|---|---|
| List, Direct | 2.80 | 7.80 | 279% | Just reads Post class | |
| List, DTO | 16.80 | 21.00 | 125% | Reads Post, Blog and Tags | |
| Create | 10.40 | 8.80 | 85% | Reads Tags and write Post | |
| Update | 15.70 | 9.70 | 62% | Reads Post, Tags and write Post | |
| Delete | 0.90 | 1.10 | 122% | Simple state update |
Analysis of raw EF performance
- Async does not have much of an overhead
I was surprised that the async versions were so fast. Lots of blog posts warn about async being slow, but on the simplest method, which was listing the Post class it was only 5ms slower. That, I think is impressive. However in the unit tests the context it was saving was small (see my simple-talk article to learn more about context) and it also caches the context so one off commands might take longer. I look at single reads later in the real-world section below. - Some of the async commands are faster!?
You can see that the async version of create and the update are faster (see blue percentages). Why is that? I am not totally sure, but I think it is because there are multiple database accesses (see the notes column) and I think it manages to overlap these. If someone has a better explanation then please let me know.
Test 2. Real-world performance on ASP.NET MVC5 site
Raw figures like that are interesting, but its real-world performance that matters. I have a companion MVC5 web site called SampleMvcWebApp, also open-source, which I try things out on. This has the same Blog, Post, Tag format described at the start, but with only a few entries (default 17). I have this MVC web app on different hosting environments, plus internally:
The SampleMvcWebApp MVC5 web site is live and available for you to access and play with. Its address is http://samplemvcwebapp.net/
- A low-cost shared site running Windows Server 2012 R2 through a UK company called WebWiz. I would recommend WebWiz (no, they are not paying me) as they seem to support the latest Microsoft environments quickly, (one of the first to support SQL 2012 in the UK) and they are relatively cheap.
- A Windows Azure account where I can change the level of hosting performance.
- Locally on my development server.
My tests are done using ab, the Apache HTTP server benchmarking tool. This allows me to load the site and also get averaged performance figures. I used the benchmark tool to read the posts list (17 entries) in two modes: a) one user trying 100 times, b) 50 users trying twice, all at the same time. The results are:
| Host+ action | Sync (ms) | SD (ms) | Async (ms) | SD (ms) |
|---|---|---|---|---|
| WebWiz | ||||
| – List Posts ave(100), one user | 90 | 4 | 95 | 9 |
| – List Posts ave(100) , 50 users | 1065 | 450 | 1050 | 450 |
| Azure, standard | ||||
| – List Posts ave(100), one user | 110 | 11 | 120 | 50 |
| – List Posts ave(100) , 50 users | 1200 | 500 | 1200 | 500 |
Notes:
– The Sync (ms) and Async (ms) columns are the average total time to get one list result from the web site using a synchronous and async EF method respectively.
– The SD (ms) column holds the standard deviation and shows the amount of variability of the results.
Analysis of MVC web site performance
- Async listing is slightly slower for one user
There is a small difference, less than 10%, between the sync and async versions. However this must all be due to the use of async as the data is the same. It amounts to 5 to 10 milliseconds, which is about the same as we saw in the raw EF figures earlier. - Using Async seems to have had no effect on the scalability of the web site.
You will see that the figures for 50 simultaneous users is almost the same for both sync and async commands. That makes sense as the database is small and the query fairly simple and there is almost no waiting for the database (raw numbers for the database access part on small databases is 2 to 3 milliseconds).Clearly with much more complex data accesses then async would start to pull away in terms of scalability of the web site because the thread would be freed up for longer while the database is doing its work. - Most of the time is spent in the MVC/Http part of the process.
If you look at the timings for one request captured using Google Chrome’s developer tools you can see the parts of the timeline below:
- This clearly shows what I said in the Simple-Talk article – that optimising/caching the other files, like css and JavsScript files would have far more effect than worrying about whether to use a sync or async data request.
I will end this section by saying it is quite difficult to test scalability on live web sites because they are designed to take a lot of load. You might like to look at Steven Sanderson’s video about threads and scalability which has an excellent demo http://channel9.msdn.com/Events/TechDays/Techdays-2012-the-Netherlands/2287.
Overall Conclusions
EF async data accesses have only a small overhead over the standard, sync commands. That is big news, and contrary to some of the older documentation that is out there. So feel free to use async EF commands in your next application.
Happy coding.