A .NET distributed cache with a ~25 nanosecond read time!

This article described a .NET distributed cache library called Net.DistributedFileStoreCache (referred to as FileStore cache in this article). The FileStore distributed cache library is very quick to read a cache value (known as Get), for instance, FileStore cache take ~25 ns. to Get a cache value, but a SQL Server distributed cache would take at least 0.1 ms – that means FileStore cache is 4,000 faster on Gets! Typically, you don’t need that speed, but I have a situation where I needed to read lots of cache value in every HTTP request, so I built this FileStore cache.

The other positive of the FileStore cache is its very easy to setup and use. The FileStore cache design is based on ASP.NET Core’s appsetting.json files so it stores the cache entries in a json file which is shared by all the instances of the application. This also means you don’t have to set up, or pay, for a database for your caching 😊.

The downside of the FileStore cache is it’s slower than database caches when adding a new cache value (known as Set). For instance, to Set a cache value to an existing 100 cache values takes ~1.5 ms., while a database would normally execute a Set in less than one millisecond. And it gets worse as the cache gets bigger, e.g. to Set a cache value to an existing 10,000 cache values takes ~9 ms. So, the FileStore cache library is useful when your application needs fast cache reads and only a few cache writes. Also, it’s NOT a good fit for large cached values, like a image, as the bigger the cache gets the slower the update will be.

The other (smaller) downside is the FileStore cache doesn’t implement the IDistributedCache’s SlidingExpiration feature, because that would make the read performance slow. But the FileStore cache does support the two AbsoluteExpiration versions.

Read on to understand how the FileStore cache works so you can decide if it would be useful to your application.

TL;DR; – Summary of this article

• The FileStore cache library provides a .NET distributed cache that has a very fast – it only takes ~25 ns. to Get of one entry in a cache containing 10,000 entries, but it is slow when you use Set / Remove (100 entries = ~1.3 ms., 1,000 entries = ~1.7 ms., 10,000 = ~7.9 ms.)
• The FileStore distributed cache relies on a json file that all the the instances of the application can accesse, just like your ASP.NET Core appsettings.json files (see Azure’s Scale-Out approach). This make it’s easy to setup because it just relies on a json file in your applications directory so you don’t have to set up, or pay, for a database for your SQL Server cache, Redis cache, or NCache cache.
• The main FileStore cache implementation has a value of type string, mainly because the values are stored in a json file. This Version is known as String.
• There are three other versions which have different cache value types / interfaces convert their cache value to a string and then call the String version:
  • Version=Class: This inherits the String version and adds three method types that serializes the class to a json string. Useful when you want to store a complex data I the cache.
  • Version=Bytes: This has a value of type byte[] with extra features over the IDistributedCache interface.
  • Version=IDistributedCache: The Bytes version is accessed via the IDistributedCache interface. Useful if you have an existing caching using the IDistributedCache interface.
• Net.DistributedFileStoreCache is an open-source library under the MIT license and a NuGet package. The documentation can be found in the GitHub wiki.

Useful links to general info on caching

• Distributed caching in ASP.NET Core – different distributed caching libraries with ASP.NET
• Caching guidance from Microsoft – covers all sorts of caching arrangements
• System Design Basics: Getting started with Caching – explains different caching approaches
• Caching in .NET Framework Applications – other types of caching, e.g WCF REST caching

Setting the scene – why and how I built the FileStore cache library.

The articles / documentation about a software cache talk about improving performance and scalability. For instance, in my book “Entity Framework Core in Action” I use two different cache approaches to improve performance of displaying and searching first 100,000 books and then ½ million books (see this article for the details and link to a video).

But I want to talk about to another use of caches, especially distributed cache, where they are used to manage application data that changes and all the instances of the application need to access. I have used a distributed cache in this way to:

• How to take an ASP.NET MVC web site “Down for maintenance” (see this old (MVC5) article)
• Updating a logged-in user’s claims when something changes (see this section / article)

But recently I used the ASP.NET Core’s appsetting.json file as a simple distributed cache when adding sharding to the multi-tenant part of the AuthPermissions.AspNetCore library. In this case I configured a separate json file containing the list of database information used ASP.NET Core’s IOptionsSnapshot service to read it. This means I can update the json file and then get the latest data when I use IOptionsSnapshot – and IOptionsSnapshot is very quick, which is good as that data is accessed every HTTP request.

IOptionsSnapshot is great, but it turns out there are some issues to fix if its going to properly implement a distributed cache. For instance, if two instances update the json file at the same time, then there would be a problem. At best result would be that one instance throws an IOException, but the worse result is one of changes is lost and you aren’t aware of it. This breaks the “coherent (consistent) across requests to multiple servers” rule of a distributed cache.

I tried to fix these problems using the IOptionsSnapshot but I found a number of situations that I couldn’t fix. But the idea is good, so I went back to the basic .NET features needed to make a true distributed cache. The two .NET features that I used to make a true distributed cache with fast read are:

1. File Locking: locking a file during an update means no other processes can read or write the file until the updated file has been saved. If another process tries to access a locked file, then it throws an exception which the library catches and retries the access after a delay. This makes sure that the FileStore cache implements the “coherent (consistent) across requests to multiple servers” rule.
2. FileSystemWatcher: the FileSystemWatcher class will alert each the instance of the application that the cache file has been changed, which means the (hidden) local static cache is out of date. On the next time the application code accesses the FileStore cache will read the file (within a lock) and update the local static cache before executing the application’s request.

The diagram below shows the four steps that happens when the FileStore cache goes through an update (e.g, Set or Remove).

NOTE: Even if you have only one application instance to the four steps are still run as a single application can be running multiple processed at the same time.

DESIGN NOTES

• The FileSystemWatcher class is known to create two events when a file is changed and this can happen in the FileStore cache. For that reason, the library invalidates the local cache and then waits for another access to the FileStore cache: that way it’s less likely that the read of the FileStore cache file will runs twice.
• For performance reasons a cache entry’s AbsoluteExpiration has expired it will return null when accessed, but the cache entry is still in the FileStore cache file. Once any cache update (i.e. set, remove, or reset) is executed, then the expired cache entries still present entries are removed. This keeps the read performance of the library high.
• As stated earlier the FileStore cache doesn’t implement the IDistributedCache’s SlidingExpiration to keeps the performance of the library high.  I could have used a similar approach to the AbsoluteExpiration in the last design note, but it would only be based on only local accesses.

The end result is that the Net.DistributedFileStoreCache library is an cache that meets the requirements of a distributed cache. It also contains some extra useful features over the default IDistributedCache interface (described later).

The four different FileStore cache interfaces

The primary .NET interface for distributed caches is IDistributedCache which has only four methods types: Set, Get, Remove and Refresh, plus async versions of each, with the key being a string and the value is a byte[]. For instance, the sync Get method has the following signature:

public byte[] Get (string key);

But because I am storing the data in a json file its much better to have the value of type string, called DistributedFileStoreCacheString, which contains the primary code. But to make the library useful to users already using the IDistributedCache I added other versions where the value is byte[]. Also, I added class that added a few extra features to the primary …String version. The figure below gives you a summary of the four interfaces the FileStore cache library provides.

The table below shows the which interface is registered is defined by the WhichVersion parameter in the FileStore’s options – they are:

Registering the distributed FileStore Cache

the AddDistributedFileStoreCache extension within FileStore cache library allows to register the FileStore cache version you want to use as a service. It also sets up / find the FileStore cache file name and location using your environment information, which is simpler (and cheaper) compared to using a distributed cache that uses a database. The code below shows how you would this in an ASP.NET Core’s Program startup.

builder.Services.AddDistributedFileStoreCache (options =>
    {
        options.WhichInterface = String;
    }, builder.Environment);

The registration code above would register a service with the interface IDistributedFileStoreCacheString. It also uses the ASP.NET Core environment class to create an different cache filename based on your EnvironmentName, for instance the file name would be “FileStoreCacheFile.Development.json” when in a development environment, and file is stored directory defined by the environment’s ContentRootPath FilePath. This makes sure that your cache file is in the right place and your development c and doesn’t interfere with your production cache file.

NOTE: Go to the Register the FileStore Cache documentation for more information on the setup and the various options you might need.

The performance of the distributed FileStore Cache

I measure the performance of the FileStore cache String version by the excellent BenchmarkDotNet library. My performance tests both reads and writes of the cache on a cache that already has 100, 1,000 and 10,000 cached values in it. The performance tests where run on an Intel Core i9-9940X CPU 3.30GHz.

The full performance figures are available in the repo’s README file but here is a summary.

Read performance

• Reads a single cache value took ~25 ns at the three levels of cache size.
• Getting a Dictionary of ALL the cache key/values took ~80 ns at the three levels of cache size.

Write performance

The time taken to add a cache value to cache goes up as the size of the cache is – see table below. This makes sense as unlike a database you are reading and then writing ALL the cache values into a file. The async versions are slower than the sync versions, but it does release a thread while reading and writing.

NOTE: If you want to make sure your use of the FileStore cache is the best it can be, then I recommend you read the Tips on making your cache fast document.

Using the distributed FileStore Cache

If you are familiar with any of .NET’s distributed cache libraries like Distributed SQL Server cache, Distributed Redis cache, Distributed NCache cache etc. then it will the same, but without of the hassle / cost of setting up a database.

The biggest difference if you use the String version is that the value is a string instead of byte[]. But each FileStore version has some other differences, as detailed below:

• All the versions apart from the IDistributedCache version has two extra method types:
  • GetAllKeyValues/Async methods, which returns all the key/values as a directory. I find this useful if I want to load multiple cache values as it only takes ~85 ns.
  • ClearAll, which drops all the key/values. Useful for testing and stackoverflow has may questions on how to clear out all the key/values in a cache.
• To keep the read performance FileStore cache doesn’t support SlidingExpiration. This has three effects:
  • All four versions will throw an exception if you try to set the SlidingExpiration option via the DistributedCacheEntryOptions parameter in the Set/SetAsync.
  • The two FileStore cache versions that use the byte[] type for the value has a Refresh / RefreshAsync method, but if you call them it will throw an exception. This makes these two versions compatible to the IDistributedCache interface.
  • In the FileStore cache versions that use a string type for the value do not contain the the Refresh / RefreshAsync methods.

The code below uses the String version and shows the use of the GetAllKeyValues (see line 12) and the Set method (line 30).

public class HomeController : Controller
{
    private readonly IDistributedOptionsCacheString _fsCache;

    public HomeController(IDistributedOptionsCacheString optionsCache)
    {
        _fsCache = optionsCache;
    }

    public IActionResult Index()
    {
        var allKeyValues = _fsCache.GetAllKeyValues();
        foreach (var key in allKeyValues.Keys)
        {
            logs.Add($"Key = {key}, Value = {allKeyValues[key]}");
        }

        return View(logs);
    }

    public IActionResult AddCache()
    {
        return View();
    }

    [HttpPost]
    [ValidateAntiForgeryToken]
    public IActionResult AddCache(string key)
    {
        _fsCache.Set(key, DateTime.Now.ToString("s"), null);
        return RedirectToAction("Index");
    }
}

NOTE: The 1.1.0 version of FileStore cache adds the following extra commands:

• SetMany / SetManyAsync which adds multiple entries in one go, which is quicker than individual Set / SetAsync calls.
• The ClearAll method has optional parameters to add multiple entries into the cache after the cache has been cleared.

Looking at the IDistributedFileStoreCacheClass version

The reason for building the FileStore cache library was to improve my AuthPermissions.AspNetCore library and its handling of multi-tenant applications. In this library I have some data in the appsettings.json files which would be better in a distributed cache, but they have multiple parameters.

I saw this article which suggested some extension methods to turn a class into a json string before saving that json string in a cache, and I though that would be really useful for me. Rather than creating extension methods I built a small class called DistributedFileStoreCacheClass that inherits the string FileStore cache version that adds the following method types (the first threehave an async version):

NOTE: The <T> part has the following where cause: where T : class, new()

• SetClass<T>(string key, T yourClass, …) – this serializes the “yourClass” into a json string and saved to the string to the FileStore cache.
• SetManyClass<T>(List<KeyValuePair<string,T> manyEntries, ...) which adds multiple entries in one go, which is quicker than individual SetClass calls.
• T? GetClass<T>(string key) – This reads in the string from the string FileStore cache and deserializes the string back to a class.
• T? GetClassFromString<T>(string? jsonString) – This is useful if you want to obtain a class from a cache entry obtained by the GetAllKeyValues / Async method.

NOTE: Be aware that the cache does not hold the class type in the cache. It is up to you to use the same <T> on the GetClass as you used in the SetClass method.

Here is an example from my unit tests to show how it works:

[Fact]
public void DistributedFileStoreCacheSetClass_JsonClass_Example()
{
    //SETUP
    _distributedCache.ClearAll();

    //ATTEMPT
    _distributedCache.SetClass("test", new JsonClass2 { MyInt = 3, 
        MyClass1 = new JsonClass1 { MyInt = 1, MyString = "Hello" } }, null);

    //VERIFY
    var jsonClass2 = _distributedCache.GetClass<JsonClass2>("test");
    jsonClass2.ShouldBeType<JsonClass2>();
    jsonClass2.ShouldNotBeNull();
    //… other tests left out
}

The string below shows how your class is turned into a string to save in the cache.

"{\"MyClass1\":{\"MyInt\":1,\"MyString\":\"Hello\"},\"MyInt\":3}"

The byte[] value versions of the distributed FileStore cache

Because many existing usages of a distributed cache uses byte[] for the value. I created two versions that supports byte[] for the value, which then convert the byte[] value into a string and the calls  String version to access the FileStore cache.

There is one DistributedFileStoreCacheBytes class, but it can be registered either the IDistributedCache interface, or the IDistributedFileStoreCacheBytes interface which contains the extra methods, GetAllKeyValues/Async and ClearAll.

Conclusions

I needed a distributed cache that has a very fast read because wanted to read a lots of cache values on every HTTP request. If I used a database-based distributed cache could have taken at least a millisecond and more, but I remembered how fast the ASP.NET Core IOptionsSnapshot<T> was when it returns data in the appsettings.json file.

After quite bit of work, I found I couldn’t build a distributed cache using ASP.NET Core’s IOptionsSnapshot<T>. But it did guide me to a design that that a) implements a true distributed cache, b) which is blistering fast on reads, and c) is very easy to setup. From my point of view the slower update of the cache (e.g. Set, Remove) is a pity, but in my situation adding a new cache value happens very infrequently.   

In the end I put in extra features, like the byte[] values version, because there must be users already using the IDistributedCache interface. To my mind the FileStore cache’s simple setup and no extra costs (for Redis, SQL Server, etc.) may be more attractive than its super-fast read! Good luck and do let me know how you get on with this library if you use it.

Happy coding.