Improving the support of multiple languages in .NET applications

One of users of my AuthPermissions.AspNetCore library (shortened to AuthP in this article) asked for support for multiple languages – known in .NET as Localization. I looked at the .NET localization solution and it lacked some features I needed for my situation. I took some time to work out to use the .NET localization code and in the end, I built a small library called Net.LocalizeMessagesAndErrors which wraps the .NET version with a interface that adds a some extra features.

This article explains how this new library it easier to add / manage multiple languages in your applications, with articles showing how to use this new localization library. The full list of the articles in this series are:

1. Improving the support of multiple languages in .NET applications (this article)
2. Applying an improved multiple languages library to .NET applications.
3. A library / pattern for methods that return a status, including localization (coming soon)

TL;DR; – Summary of this article

• This article provides a super-quick introduction to .NET localization feature, as some of the concepts weren’t obvious to me at the start.
• My problem was if I added the .NET localization to my AuthP library, then no one could use the AuthP library unless they had set up the .NET localization with resource files, which is lot of work.
• My solution was to build a service that wraps around the .NET localization service and provides extra features. Specifically, you can build applications without .NET localization, and it will still work. This feature will also help developer who need to add localization to an existing application, as you can apply localization in stages.
• Once I started looking at localization, I found several ways to either make the code easier to understand, or easier to use. The result is the Net.LocalizeMessagesAndErrors library.
• After the quick explanation how the Net.LocalizeMessagesAndErrors’s IDefaultLocalizer service works I detail six localization challenges and how I got around them, plus a tip on how to setup localization resource files.  

A brief view of how .NET localization works

At first, I was confused by how to support multiple languages because I has no idea what the terms means and how they work together. Therefore, here is a short introduction from me with links at the end to other articles that I found very useful:

• Localization means providing different languages, e.g. English, German, Mandarin, in your application – (I like the name Multilingual support as its more obvious, but I use localization because that what Microsoft calls it).
• Globalization is about showing dates and numbers in the correct format, with some compare / order string methods.
• With .NET localization you store the different messages in resource files in your application. Each resource file has a name based on:
  • A name, usually taken from a class’s Fullname, e.g. Controllers.HomeController
  • A name representing the language it contains, eg. “en-GB”
  • And has extension of .resx.
• A resource file has entries with a Name (which I call the localize key) and Value which holds you’re message in the correct language. The Name / Value entries in the resource file works like a dictionary, with the Name being the key.
• You add a resource file for each language (known as culture) and in each resource you would add an entry for each message (Value) you want to show, with a unique localize key (Name) to use as the lookup.
• You also need to setup the localization services – see this for how to setup an in ASP.NET Core application and the other links below.
• You would get a localize service, like IStringLocalizer<TResource>, to obtain the localized message. There are three parts to get the localised message:
  • The start of the resource filename is defined by the TResource’s FullName.It then adds the current culture Name from the user, cookie, or other source (depends on what you setup).Your .NET service which takes your localize key, e.g. _localizer[“YourKey”], which return a string containing the entry found in the selected resource file.
  • You can also have formatted messages, such as $”Date is {0}”, which would need extra data, e.g. _localizer[“YourKey”, DateTime.Now].

Once I understood the names / concepts the Microsoft’s documentation of .NET localization made much more sense to me. Here are links to articles about .NET localization that I found useful:

• Microsoft’s ASP.NET Core globalization and localization document – good, but steps are confusing.
• Building Multilingual Applications in ASP.NET Core – excellent step-by-step article
• Localization in ASP.NET Core Web API – another excellent step-by-step article (.NET 6)
• ASP.NET Core Localization Deep Dive – shows all the different localization parts

Super-quick explanation on the IDefaultLocalizer works

This article is about why I implemented the Net.LocalizeMessagesAndErrors library and what new features that it contains, but here is an overview of the IDefaultLocalizer service to help you understand the extra features this service provides.

In the nutshell the IDefaultLocalizer service lets you to put strings like “Hello!” or FormattableString like $"The date is {DateTime.Now}" in your code (I use the term message for these two types), which makes your code easier to understand. Have look at the diagram below and read the blue italic words which explains how the IDefaultLocalizer service works.

This article doesn’t tell you how to use the Net.LocalizeMessagesAndErrors library, but it highlights the main change – there is a message in each call. If you want more information on how use the library then see the “How to add multiple languages to a ASP.NET Core using the Net.LocalizeMessagesAndErrors library” (coming soon) or the Net.LocalizeMessagesAndErrors documentation.

The localize challenges I found and how I fixed them in my library

I spent a lot of time trying to come up with ways to use the .NET localization to work with my AuthP library, but it just didn’t work for me. Some of the problems were around adding localization to a NuGet package, but the biggest issue was the massive changes I would have to make to the AuthP library if I changed over to .NET localization.

The list below gives the localize challenges I found and how I overcame them. The list is in order with the biggest challenges first. They are:

1. I didn’t want to turn all the messages / errors into just a localize key.
2. I wanted a NuGet that works without having to setup the .NET localization.
3. I didn’t like .NET’s localization’s handling of missing resource files / entries.
4. I wanted a better format for the localise key to help creating unique strings.
5. I wanted to unit test without setting up localization.
6. I wanted to check the localized messages with their localize key.
7. Tip: Use Excel (or other app) to setup the data for the resource files.

1. I didn’t want to turn all the messages / errors into just a localize key

As of version 4.0.0 of the AuthP library has 100+ messages over five projects. Most of these messages are error messages while the rest are success messages. Here are an example of success and error message:

• "Successfully added the new user."
• $"There is already a Role with the name of {0}"

If I used .NET localization these messages would be turned into a localize key, which from my view has the following downsides:

• The messages make great comments and turning into just a localise key messages would make the code harder to understand.
• It’s a lot of work to move these messages to a resource file, and the messages are much harder to update.

My solution was to leave the current success / error messages where they are and define them as generic English (culture “en”) – I call these messages as default messages. I already have a common pattern for my methods / services which handles success and error message, so it was easy to update the code to pass the default messages to my localization wrapper called DefaultLocalizer. The process the DefaultLocalizer follows are:

1. On registration of the DefaultLocalizer service, I define the culture of the default messages, in this case “en”.
2. If the user’s / app’s culture started with default culture, then the default message is returned without having to use the .NET localization.
3. If the user’s / app’s culture doesn’t start with default culture, the DefaultLocalizer service uses the .NET localization to obtain the message from the resource files.

Here is an example of my improved common method to handle localization showing a success message and an error message.

public class ExamplesOfStatusGenericsLoc<TResource>
{
    private readonly IDefaultLocalizer<TResource> 
        _defaultLocalizer;

    public ExamplesOfStatusGenericsLoc(
        IDefaultLocalizer<TResource> defaultLocalizer)
    {
        _defaultLocalizer = defaultLocalizer;
    }

    public IStatusGeneric CheckNull(string? month)
    {
        var status = new StatusGenericLocalizer(_defaultLocalizer);
        status.SetMessageString(
            "Success".ClassMethodLocalizeKey(this, false), 
            "Successful completion.");

        if (month == null)
            return status.AddErrorString(
                "NullParam".JustThisLocalizeKey(this), 
                "The input must not be null.");

        return status;
    }
//...rest of class left out

NOTE: If any errors are added to the status, then the Message is changed to “Failed with {n} errors”. That’s just in case the success Message is incorrectly shown.

2. I wanted a NuGet that works without having to setup the .NET localization

If I just applied the .NET localization to the AuthP NuGet library, it would mean everyone that used this library they would have to set up .NET localization with resources etc. The library is already complex and with the extra needed to understand / setup .NET localization would put off developers from using this library.

The solution I added is into the DefaultLocalizer service is to return the default message if the .NET localization hasn’t been setup. This means when the localization version of the AuthP library is released:

• The AuthP library doesn’t get any more complex unless the developer what’s to use this new localization feature.
• Developers that are already using the AuthP library can upgrade to the localization version without needing to change their code.

I hate to think what new and existing users would think if they had to set up .NET localization to use the AuthP library!

NOTE: You might not be creating a NuGet like I has, but if you are adding localization to an existing application, then this approach allows you to add localization in stages. That might be pretty useful.

3. I didn’t like .NET’s localization’s handling of missing resource files / entries

The .NET localization will return the localize key if no entry is found in the resource files. This typically doesn’t provide a good experience for the user. The DefaultLocalizer service can provide the default message which isn’t in the correct language, but easy for the user to translate.

The other issue of missing resource files / entries is reporting. The NET localization does provide a ResourceNotFound parameter, which will be true if the localized message isn’t found, but if you want a log / event then you need to add that to each localization call. On the other hand, the DefaultLocalizer service provides a very detailed log – a example is shown below.

The message with the localizeKey name of 
'MissingResourceEntry_MissingEntry' 
and culture of 'fr' was not found in the 
LocalizedWebApp.Resources.Controllers.HomeController' 
resource. 
The message came from 
DefaultLocalizerController.MissingResourceEntry, line 38.

This provides everything you need to correct this problem, including the class, method, and line number of where the localization came from.

4. I wanted a better format for the localise key to help creating unique strings

The .NET localization service allows you to use any string as the localize key, and its up to you to make sure it is unique. You can use string constants, e.g. “HelloMessage” for the localise key, but when I build (and used!) the DefaultLocalizer service I found string constants were hard work and error prone.

My view is that string constants are fine for small applications, but for larger applications the localize key needs a standard format and methods to help the developer to create unique localize keys quickly. My solution has a format of “{className}_{methodName}_{localKey}”, with the className and methodName being optional. The table below shows are three main versions that are used, with the first one used on 90% cases.

To implement this localize key format I have created a set of extension methods that can automatically fills in the “{className}” and “{methodName}” for you. This has two advantages:

• Easier for the developer to create a unique localize key.
• The developer can work out where the localize key was created.
• You can cut / paste your localize code and the localise key will automatically change to the new class & method parts of the localize key.

Here are two examples taken from the ExamplesOfStatusGenericsLoc method shown earlier in this article:

• "Success".ClassMethodLocalizeKey(this, false)
• "NullParam".JustThisLocalizeKey(this),

You can get a full set of the extension methods in the “Creating localize keys” document also cover some of the problems and solutions of the balance between readable localize key and being unique in this section.

5. I wanted to unit test without setting up localization.

As I said the AuthP library has five project containing code and I have nearly 400 unit tests, of which a third check errors or success messages. If I used .NET’s localization on its own, then I could easily stub out (see this Microsoft document about stubbing) the .NET’s localize methods but would still have to change many of the unit tests to use the localize key instead of the actual error / success messages. It’s more work and makes the unit tests less easy to understand as the actual error / success strings are gone.

Because the DefaultLocalizer can return the default messages it’s easy to create a DefaultLocalizer  stub can return the actual error / success strings. The Net.LocalizeMessagesAndErrors repro contains several stubs, but in this case, you need the StubDefaultLocalizer class.

The StubDefaultLocalizer class has the same methods as the DefaultLocalizer class, but it a) returns the default message, and b) holds the localize key data of the last localize. This allows the unit test to continue in the same way, but if I want to you can check on the localizer key. See the code below which shows how the StubDefaultLocalizer class works.

[Fact]
public void TestStubDefaultLocalizer()
{
    //SETUP
    var defaultLoc = new StubDefaultLocalizer();

    //ATTEMPT
    var message = defaultLoc.LocalizeStringMessage(
        "MyLocalizeKey".MethodLocalizeKey(this),
        "My message");

    //VERIFY
    message.ShouldEqual("My message");
    defaultLoc.LastKeyData.LocalizeKey.ShouldEqual(
        "TestStubDefaultLocalizer_MyLocalizeKey");
}

This works fine, but I found another type of stub that solved another issue I came across, as described in the next section.

6. I wanted to check the localized messages with their localize key

Once I starting to localize my AuthP library, which has ~110 localization I soon found I really needed an overview of all the localizations to check on localise key uniqueness, format, duplicates etc. Stepping though the code to fine each message is hard work and its easy to miss one.

So, I thought – can I write the localize information to a database when running my unit tests. At that point I created StubDefaultLocalizerWithLogging class, which returns the default message, but (optionally) logs the full information of the localization data to a database. This provides a quick way to look at the localized messages, and it can find certain problems.

For each use of a DefaultLocalizer usage it logs the localize key, culture, the message and where the localised entry was created (the full list of what is in the log can be found in the LocalizedLog class, which has 9 parameters).

The screenshot below is a section of the logged localization data. Note the PossibleErrors column which has found an existing entry in the database with the same localize key, but the message format is different. NOTE Click the screenshot to get a bigger version.

I have found quite a few of localization issues by looking through the sorted data. I also I found the logged list very useful when building resource files for other languages because it gives me the Name (localize key) and the Value (string / format) that needs translating. My unit tests only find 75 localized messages when in fact that there are ~110 localized messages. For the 35 localize message ones that aren’t logged I had to go three extra steps to set up the entry in the resource file(s):

1. Find the code that created the localized message.
2. Work out what the localize key is.
3. Copy the message format.

These three manual steps are tedious and error prone. It enough to make me improve my unit test coverage 😊.

The only downside of logging to a database is the unit tests are slower – in the Net.LocalizeMessagesAndErrors library that has ~100 the unit tests which take ~1.5 seconds without logging to the database, but ~2 seconds with logging to the database. In the AuthP library, which has nearly 400 tests the difference between log to database being on / off is a smaller percentage.

Thankfully you can turn the database logging on or off by setting the `SaveLocalizesToDb` to true or false respectively – see the documentation for the StubDefaultLocalizerWithLogging here.

7. Tip: Use Excel (or other app) to setup the data for the resource files.

This isn’t anything to do using DefaultLocalizer, but I found that adding entries to a resource files isn’t a nice process in Visual Studio (VS Code, with the ResX Editor extension, is better). In the end used Excel to entry the resource Name / Value and then turn in into a .csv file. The code below (taken from my AuthP repo) converts CVS to a resource file.

public void CreateResxFileFromCSV()
{
    var csvFilePath = "filepath to csv file";
    var resxFilePath = "filepath to EMPTY resource file";
    
    //see https://joshclose.github.io/CsvHelper/getting-started/#reading-a-csv-file
    using (var reader = new StreamReader(csvFilePath))
    using (var csv = new CsvReader(reader, 
          CultureInfo.InvariantCulture))
    {
        var records = csv.GetRecords<CsvInputOfResx>();
        //see https://learn.microsoft.com/en-us/dotnet/core/extensions/work-with-resx-files-programmatically#create-a-resx-fil

        using (ResXResourceWriter writer = 
             new ResXResourceWriter(@resxFilePath))
        {
            foreach (var entry in records)
            {
                writer.AddResource(entry.Name, entry.Value);
            }
        }
    }
}

I find this is especially useful if you need to change / add to your resource file, as its much easier to search / change in Excel.

Conclusions

The Net.LocalizeMessagesAndErrors is relatively small (the DefaultLocalizer is only has ~100 lines of code), but it took me more than five weeks of work! That’s because when I started to use the library I found a load of improvement – I got to local version 1.0.0-preview034 before I had finished. The result is that the library is much easier to use when updating an existing application to support multiple languages, and hopefully nicer to work with.

The changes I added came from applying the library to a) my AuthP library, b) adding a demo ASP.NET Core app within the library repo (see LocalizedWebApp), c) localizing the Example1 ASP.NET Core in my AuthP library and d) writing the Net.LocalizeMessagesAndErrors documentation (writing the docs always shows me any bad interfaces).

In the following articles I will show how to use the Net.LocalizeMessagesAndErrors library to build localized .NET applications. There also good documentation for this library now that contains all the details if you want to try it out now.

Happy coding.