Converting your ASP.NET MVC5 application to use Bower, Grunt and Gulp

I changed over to Visual Studio 2015 (VS2015) a while ago, and one of the main reasons for moving was to use some of the new features for handling web packages. I also wanted to look at the preview of ASP.NET Core 1 to see how they used these features.

VS2015 contains as standard Bower, which is a web package manager used across the whole of the web development community. In addition VS2015 supports a web build automation system using Grunt and/or Glup. These are great tools for a number of reasons that I won’t cover now, but at the end of this article I list some of the advantages I found when I swapped over – and I think they are really worth having!

Note: While this article talks about VS2015 it is also applicable to VS2013 as the same Bower, Grunt and Gulp features are available for VS2013 via add-on extensions. If you have VS2013 then read this article by Scott Hanselman on what packages you need.

I personally needed the power of these tools on an ASP.NET MVC5 e-commerce project which is nearing completion and we are doing fairly drastic things to the design and JavaScript front-end. I didn’t want to upgrade to ASP.NET Core 1 this late in the project, especially as ASP.NET Core 1 is very different to MVC5 and not released yet. I therefore set about harnessing these tools in an existing project, with the aim of following the ASP.NET CORE 1 style to make upgrading later as simple as possible.

NOTE: There is a sample application on GitHub to go with this article. In this application I have converted an existing ASP.NET MVC5 project away from using the normal NuGet/BundleConfig to using Bower and Grunt for handling web packages. It is open-source so please have a look and see if it is useful.

This sample application also contains a library called BundlerForBower, B4B for short, which takes over the role of MVC’s BundleConfig. I mention the use of B4B towards the end of this article, but I have also written a more detailed article about B4B called Introduction to BundlerForBower for ASP.NET MVC5.

UPDATE: BundlerForBower is now available on NuGet

BundlerForBower is available on NuGet – see https://www.nuget.org/packages/Bundler4Bower/ 

Swapping from NuGet to Bower

If you create a brand new ASP.NET CORE 1/MVC solution in VS2015 then the MVC project looks very different to the current ASP.NET MVC5 structure. This is because the whole structure has been reconfigured to be more applicable to modern web development (see the ASP.NET CORE 1 overview article for more information).

We are interested in how it handles the web packages because if we are going to adopt Bower etc. we want to use the same approach as the new ASP.NET structure so that any future upgrades will be simpler. There are many changes in ASP.NET CORE 1 but the key web package changes are:

  1. It automatically uses Bower, not NuGet, to load web packages like JQuery, Bootstrap etc.
  2. It includes a ‘Dependencies’ virtual folder that contains:
    1. A Bower folder that containing our web libraries, e.g. JQuery
    2. An npm folder that contains our build automation tools, e.g. Gulp and plugins.
  3. It doesn’t have an App_Start/BundleConfig.cs as Gulp does the bundling/minification.

So let us now change our existing MVC5 project to match this approach.

1. Use Bower, not NuGet for web package management

If you have an existing MVC5 application you won’t see the Manage Bower Packages option in any of your menus. It needs a file to unlock the feature. Therefore the very first step is to do is create a bower.json file to your MVC5 project, by right clicking the MVC project folder in the solution explorer and select: Add>New Item and then find the “Bower Configuration file” in the list of templates, e.g.

CreateBowerFile

This will create the file bower.json and activate bower. The bower.json is the file that Bower writes to when you load a packages. This is useful for package restore, i.e. restoring the bower files if haven’t already got them loaded. As you don’t normally include the web packages in your source control then you need this if loading the application into a different development environment.

You don’t have to, but I also set up a .bowerrc file. This is the Bower configuration file and contains things like what directory where the packages will be written to. I added the following in the .bowerrc file:

{
  "directory": "lib"
}

which set the top-level directory ‘lib’ as the place that bower will place the libraries it loads. If your don’t do this then I think (I haven’t checked and the docs don’t say) the default is the top-level directory ‘components’, based on this stackoverflow question.

Once you have these files you when you right click the MVC project folder in the solution explorer you should see a new option: Manage Bower Packages (Note: I have found that this command doesn’t immediately appear once you have added the files. Restarting the Visual Studio and reloading the solution fixes this).

Selecting the Manage Bower Packages function brings up a screen (shown below) which is very like the new V3 NuGet screen. However in this case it is accessing the bower.io API to allow you to browse the packages available via Bower.

ManageBowerPackages

BE WARNED: I have found a few of issues in the current Bower interface:

  1. I used the interface to try and find a package called ‘fontspy’ but it could not find it, although Bower’s online search said it was there. However if I went into the bower.json file and typed “fontspy” intellisence knew about it and gave me the version. I then right-clicked the bower.json file and selected Restore Packages and fontspy loaded ok. The interface never showed that fontspy was loaded.
  2. I found Underscore using the interface, but it only gave version 1.5.2, which is an older version. If I went into bower.json and typed “underscore” it found version 1.8.3. I used that version by using Restore Packages to load it (see point 1).
  3. The interface says ‘No dependencies’ for packages that do actually have dependences, which is a bit confusing. Clicking the ‘project URL’ and looking at a file called ‘bower.json’ will show you the actual dependencies. However, even if the screen doesn’t show the dependencies bower will load any dependant packages if required.

If you select and install a package two things happen:

  1. The bower.json file is updated with the loaded package(s).
  2. The package and any of its dependant packages are loaded to the directory pointed to by .bowerrc.

Here is the content of bower.json after JQuery, BootStrap and Microsoft.jQuery.Unobtrusive.Validation has been loaded.

{
  "name": "ASP.NET",
  "private": true,
  "dependencies": {
    "jquery": "2.1.4",
    "bootstrap": "3.3.5",
    "Microsoft.jQuery.Unobtrusive.Validation": "~3.2.3"
  }
}

By default the loaded packages are not included in the project, which is good as we don’t want them to be sent to the web site, or store in source control. However this means to see them we need to turn on the ‘Show all Files’ option in VS’s Solution Explorer, and after installing a new package you will need to ‘Refresh’ the Solution Explorer view. Here is a view of the loaded packages directories with bootstrap opened up.

Note: You won’t see the lib directory in the sample application as they are not included in source control. If you download the application you need to right-click on the ‘bower.js’ file and click Restore Packages. That will download the packages and populate the lib directory.

BowerLibShowingBootStrap

Note: you will see that package jquery.validation is loaded even though it does not appear in the bower.json file. This is because Microsoft.jQuery.Unobtrusive.Validation’s own bower.json files says it dependents on jquery.validation so Bower automatically loads that if it is not already there.

2. Remove web packages loaded by NuGet

Tip: It took me a bit of searching to find the equivalent web packages in Bower. JQuery was easy but some of the packages have difference versions with very similar names. It’s worth cross-referencing the files that NuGet loaded with the bower equivalent before you uninstall the NuGet versions.

Previously you used NuGet to load your web packages, but when changing to Bower I really recommend your remove those packages from NuGet. This ensures that you won’t confused between the two packages, and more importantly if Bower updates its packages there isn’t a chance that you are using an older package loaded by NuGet. I also find it nice to not have a long list of JavaScript files under the ‘Script’ directory – now I can put my JavaScript files in there and it’s not confusing.

To Uninstall NuGet loaded packages then simply open NuGet and select the ‘Installed’ tab, then ‘Uninstall’ the packages that Bower is now handling. Note that your current views and layouts will stop working, which leads me on to the next section.

Swapping from BundleConfig to Grunt/Gulp

In ASP.NET MVC5 CSS and JavaScript files were normally delivered by App_Start/BundleConfig.cs which also looked after bundling and minification in a released application. Also, if you required compiling of Less, Sass, TypeScript, CoffeeScript, templates, image spriting etc. you may have used Web Essentials.

The new way is to use build automation tools like Grunt or Glup to achieve the same thing. These tools have a large base of what are called ‘Plugins’ which include all the features of Web Essential and much more.

You have a choice of using either Grunt or Gulp, or both. They both do the same thing but in slightly different ways. I ended up using Grunt for no more reason than I found some useful commands in Grunt format. Gulp is the standard automation tool if you create a new ASP.NET CORE 1 project, but as I said either will do the job.

Note: If you want to use Gulp instead of Grunt then I recommend this ASP.NET CORE 1 documentation on Gulp. If you want an example Gulp file then create an ASP.NET CORE 1/MVC project and look at the gulpfile.js that it produces.

1. Installing Grunt

The build automation tools need to be loaded, and for that we use NPM. NPM is a library system that makes it easy to share code. Grunt (and Gulp), plus there extensions, called plugins, can be loaded by NPM into your application.

First you need to add a NPM configuration file to the project in a similar way you did for the Bower configuration file, i.e. right clicking the MVC project folder in the solution explorer and select: Add>New Item and then find the “NPM Configuration file” in the list of templates.

This adds a file called package.json and opens you in that file. Now you manually add the Plugins you need. Thankfully you get great intellisence help with package names and their versions, so it’s not too hard. What you put in depends on what you want to do, but here is my file as a start:

{
  "version": "1.0.0",
  "name": "ASP.NET",
  "private": true,
  "devDependencies": {
    "grunt": "0.4.5",
    "grunt-contrib-clean": "0.6.0",
    "grunt-contrib-cssmin": "0.14.0",
    "grunt-contrib-jshint": "0.11.0",
    "grunt-contrib-concat": "0.5.1",
    "grunt-contrib-uglify": "0.8.0",
    "grunt-contrib-watch": "0.6.1",
    "grunt-contrib-copy": "0.8.2"
  }
}

Here is link to the reference article that I got this from. The only extra command I added was grunt-contrib-copy, which I needed for copying the bootstrap font files.

Once you have filled or changed the package.json file you should right-click the file and click Restore Packages. This will cause npm to look for those packages and load them. If you do a Refresh in the VS Solution Explorer you should find the package/plugins in the ‘node_modules’ directory.

NOTE: Be warned. If you have made a mistake in the file you won’t get any kind of error feedback when you run Restore Packages – it still says ‘Installing packages complete’. However if the files don’t seem to update look at the Output Window>Bower/NPM.

2. Define your Grunt tasks

You define the tasks you want to call in the Grunt configuration file, using the normal right-click the project and select Add > New Item. Select the Grunt Configuration file option, leave the default name, gruntfile.js, and click the Add button.

There are lots of ways of doing this and not really the space to explain all the possibilities, but I do give some links later. To give you a general introduction I have listed a very cut-down gruntfile.js which only concatenates and minifies the CSS files.

module.exports = function (grunt) {

    // Project configuration.
    grunt.initConfig({
        concat: {
            css: {
                src: ['lib/bootstrap/dist/css/bootstrap.css', 'Content/Site.css'],
                dest: 'css/styles.css'
            },
        },

        cssmin: {
            css: {
                src: 'css/styles.css',
                dest: 'css/styles.min.css'
            }
        }
    });

    // Load the plugin that provides the tasks we need
    grunt.loadNpmTasks('grunt-contrib-concat');
    grunt.loadNpmTasks('grunt-contrib-cssmin');

    // Default task(s).
    grunt.registerTask('default', []);

    // Build task(s).
    grunt.registerTask('build:css', ['concat:css', 'cssmin:css']);
};

In the example above I listed the files I wanted to concatenate from the packages that Bower loaded earlier. The simplest way to do this is compare the names of the files you loaded via your existing BundleConfig class and look for the same name in the lib directory. The files you want are often in a directory called ‘dist’, but some packages like Microsoft.jQuery.Unobtrusive.Validation don’t follow that rule.

With the gruntfile.js set to the code shown above up then if you can open the Task Runner Explorer window (try View > Other Windows > Task Runner Explorer) then you should see something like this:

TaskRunnerWindowYou can see the concat:css and cssmin:css tasks, plus the build:css task which executes concat:css followed by cssmin:css. The result of running concat:css is you will find a file called ‘styles.css’ which is a combination (bundle) of the bootstrap.css file followed by the Site.css file. If you run cssmin:css then a second file called ‘styles.min.css’ will appear which is the minified version of ‘styles.css’.

You run commands by right-clicking on a command and selecting Run. Feedback is pretty good, with the output of the task shown to the right of the command part of the window.

There is plenty more you can do but the example above gives you the basic idea. I would refer you to these useful articles for more information.

Note: different gruntfile in the sample application

You will find that in the sample application I use a different approach to specifying the files that are in each bundle, because I want another feature, BundlerForBower (explained later), to also have access to the list of bundles and their files. Therefore I place the array of files in a json file which I store in the App_Data directory. You can see the complete version of my gruntfile.js here and the json file it reads here.

3. Binding actions to Visual Studio Events

TaskRunnerWindowYou can bind any of the actions in the GruntFile to a Visual Studio events: Before Build, After Build, Clean and Project Open. You do this by right-clicking on a command you want to link and selecting Bindings. Below is an example taken from the ASP.NET grunt documentation, where you bind the watch feature to Project Open.

These bindings are stored as a specially formatted comment at the top of the gruntfile.json.

4. Delivering CSS/JavaScript to the Brower

The new ASP.NET CORE 1/MVC6 project does not use BundleConfig.cs to combine and minify CSS and JavaScript. It uses some new razor <environment names=”Development”> feature to choose between individual files or the concatenated & minified files produced by the Grunt/Glup task. It seems that the approach is to include all the individual files in in Development mode. In Production/Release it includes the concatenated and minified file with a cache buster suffix added using the new asp-append-version tag inside a HTML <script>. The asp-append-version tag generates unique hash based on the file content (see this helpful stackoverflow answer explanation).

In an existing MVC5 project we don’t have the new <environment> or the asp-append-version tag so we need to find an alternative. There are two ways to do this:

a. Use BundleConfig.cs to deliver the concatenated files

One of the reasons I chose the Grunt approach is it created both a concatenated, non-minified file and the minified file for both CSS and JavaScript. This means I could create a bundle inside BundleConfig.cs that looked like this:

bundles.Add(new StyleBundle("~/Content/css").Include("~/css/mainCss.css"));

In development mode it would deliver one large CSS file, styles.css, which isn’t minified. In release mode it would use the styles.min.css file with a cache buster suffix, i.e. a string that changes when the file changes which makes sure an older, browser-cached version isn’t used.

b. Using the library BundlerForBower (B4B)

The above solution works but throws up loads of issues when debugging of JavaScript. As the whole point of using Bower etc. is because you have a lot of front-end code to debug the first solution is less than idea.

I therefore built a specialised version of the MVC5 BundleConfig, called BundlerForBower or B4B for short, which is specifically written to work closely with Bower, Grunt and Gulp. It also follows some of the design styles used in the current ASP.NET Core 1 applications.

B4B consists of some code that performs a similar role at MVC’s BundleConfig class plus two extension methods that are very similar Html helper methods to MVC5’s `Styles` and `Scripts` classes to deliver bundles in a view. It is therefore quite easy to convert an existing MVC5 views over to B4B by making the following replacements:

@Styles.Render(“~/Content/mainCss”) is replaced by @Html.HtmlCssCached(“mainCss”) and @Scripts.Render(“~/bundles/javaScriptBundle”) is replaced by @Html.HtmlScriptsCached(“javaScriptBundle”)

As well as bundles that deliver groups of files from within your application ASP.NET Core 1 has a set of tags for delivering files from a Content Delivery Network (CDN), with fallback capabilities. I have reproduced this feature in B4B as it is so useful. Public CDNs are available across the world for many of the standard libraries like JQuery and Bootstrap, and can be faster than local delivery especially if the user has already loaded the standard library when using another web site.

NOTE: I have written an article called ‘Introduction to BundlerForBower for ASP.NET MVC5‘ which describes B4B in detail. B4B is an open-source (MIT) project and is included in the sample application.

My reflections on using Bower and Grunt

I think that Bower and Grunt/Gulp are a really good step up from the old NuGet, BundleConfig and Web Essentials. I spotted a few issues that I have noted in this article, but they were easy to get round and are likely to be fixed soon. The difference was immediately apparent as I was now much more in control on my web packages. It solved three problems that I had with the NuGet, BundleConfig and Web Essential’s approach which are worth listing, as they point out some of the gains this new approach brings.

1. Better coverage of packages

I use a JavaScript package called JQuery-FontSpy, which isn’t in NuGet. Previously I had to copy the files I needed by hand from GitHub. However JQuery-FontSpy is, of course, available in Bower. This is typical of more specialised packages and one of the reasons why switching to Bower is so useful.

2. More up to date packages

I am using handlebars.js templates in my web application to dynamically create complex HTML at runtime via JavaScript. While a handlebars package is available via Nuget it is at version 3.0.0 which isn’t the latest and misses a feature I would have liked to use. However the Bower version is at version 4.0.3, which is the latest, and because Bower is the main way of releasing packages like this then it will always have the latest.

This reminds us that web packages only get into NuGet because someone takes the trouble to transfer it to NuGet, and they may not feel the need to keep it updated. Whereas Bower is the primary tool for deploying web packages so the latest and greatest of a package will be available.

3. Bower libraries can contain more information

The Nuget version of libraries like BootStrap and JQuery contain just the final .js and .min.js versions of the file. However if you load these libraries via Bower you find they have a directories that contain the various parts of the library.

This is very helpful for me as I change my e-commerce site from using BootStrap to the final CSS design scheme. While I don’t want much of BootStrap’s CSS I would like to use some of its modules, like Modal, to save me reinventing that code. If I was using NuGet I would have had to go the bootstrap site and make a custom download, which I would have to redo if any new version came out. However with Bower I simply installed the bootstrap package and add the individual modal JavaScript and CSS, via less, files I need from the bootstrap\js and bootstrap\less directories to my gruntfile.js build script. If I want to add another module, like tooltip, I just include those files in my build script too. This is a much nicer approach than a custom download.

Even more web tools…

I should mention an alternative/adjunct to Bower etc. in the form of ‘Node.js tools for Visual Studio’ extension for VS. This has some extra features over the plain Bower, Grunt/Glup such as JavaScript debugging and profiling. It is definitely worth a look.

I used Node.js as a separate application some years ago when developing a Single Page Application and Node.js was great, if a bit daunting to learn coming from a VS background. However it seems like ASP.NET CORE 1 is going to use Bower, Grunt/Gulp as standard so I decided to start with that for now. You can use both the build in Bower and the Node.js extension together, so maybe I might progress to the Node.js extension in the future.

Conclusion

I have really appreciated the new Bower, Grunt/Gulp tools. They have made changing my e-commerce web site over to a proper design much easier. I also find the build tools are more comprehensive, yet allow me to just run one commend, ‘build’, to make sure everything is up to date.

I did have to create BundlerForBower (B4B) for helping with the process (see separate article on this). B4B means I have one file, BowerBundles.json, to update and everything ripples through the system. I also like the Unit Test checking that B4B contains as it stops me forgetting something before I release to production.

Please feel free to take a copy of the sample application and play with it. Hopefully this article and the sample will help you convert to Bower and be ready for ASP.NET Core 1.

Happy coding!

Introduction to BundlerForBower for ASP.NET MVC5

I have recently changed over my ASP.NET MVC5 application to use Bower and Grunt for handling all my web packages, e.g. JavaScript and CSS libraries. You can read about this in another of my articles called Converting your ASP.NET MVC5 application to use Bower, Grunt and Gulp. In changing over that I tried to follow the approach that ASP.NET Core 1 uses with web packages. Most things, like its use of Grunt/Gulp for web build tasks was fine, but when it came to the the inclusion of JavaScript and CSS files in a HTML web page, then I had to find a new way.

This article is about my solution called BundlerForBower, or B4B for short, which is available in an open-source project. My B4B solution provides similar features to MVC5’s BundleConfig approach, but is designed specifically to work with Bower and the new Grunt/Gulp build process.

UPDATE: BundlerForBower now available on NuGet

BundlerForBower is available on NuGet – see https://www.nuget.org/packages/Bundler4Bower/ 

ASP.NET Core 1 approach

When swapping over to Bower and Grunt/Gulp I wanted to follow as closely as possible the approach that ASP.NET Core 1 uses, as I expect to change over to using ASP.NET Core 1 some time soon. As you will see from my other article it wasn’t hard to follow the same approach for most things, but for delivering JavaScript and CSS files in a HTML web page I had two problems.

The first was a technical problem, in that ASP.NET Core 1 uses new <link> and <script> tags which MVC5 does not have access to. The second was a design problem in that ASP.NET Core 1 approach was to have the user list the files to be delivered in TWO places: one in the Gulp file for the build process and the second in the HTML views that needed them. That isn’t DRY (Don’t Repeat Yourself)!

These two things spurred me on to create a better solution. In fact I have made B4B configurable to work in both an ASP.NET MVC5 application AND in the new ASP.NET Core 1 application as I think the design of B4B, which is DRY, is useful in ASP.NET Core 1 too.

The parts of BundlerForBower

I am going to describe the various part of B4B, but as a start let me introduce the three main parts.

  1. A extension class called BowerBundlerHelper which needs to be placed in you MVC application so that it has access to various MVC features.
  2. A BowerBundles.json file that contains the list of bundles and their files. This is used both by Grunt/Gulp to prepare the files and by B4B to deliver the correct files at run time. Note: This file should be placed in the App_Data directory of a MVC5 project (not sure where to put it in ASP.NET Core 1).
  3. A class library called B4BCore which the BowerBundlerHelper class uses to handling bundling. This class library also contains a useful class called CheckBundles that is useful for checking all your bundles are up to date before your release anything to production.

As you can see from point 2 this approach is DRY, as the same file that is used for building the concatenated and minified production files is also used by B4B to delivery the individual files in Debug mode and the production files in non-Debug mode.

Using BowerBundlerHelper to deliver bundles

The BowerBundlerHelper extension class has two Html helper methods that are very similar to MVC5’s Styles and Scripts classes, but applied as extension methods on the HtmlHelper class. They are:

  1. @Html.HtmlCssCached("bundleName"), which is equivalent to @Styles.Render("~/Content/bundleName")
  2. @Html.HtmlScriptsCached("bundleName"), which is equivalent to @Scripts.Render("~/bundles/bundleName")

B4B makes a decision to delivers individual files or the single minfied file that Grunt produced is defined by whether the code was compiled in DEBUG mode or not. This feature can be overridden on each method by the optional ‘forceState’ parameter.

Using BowerBundleHelper to deliver static files with cachebuster added

When delivering static files, e.g. images, you need to think about what happens if you change the file content. The problem is if you change the file content but not its name then if caching is turned on the user’s browser will use the old file content, not the new file content.

The BowerBundlerHelper has a command to turn a normal file reference into one containing a cache  busting value. For instance for an image you would use something like this in your razor view:

<img src='@Html.AddCacheBusterCached("~/images/my-cat-image.jpg")' />

There is a more detailed section on this later in this article.

BowerBundles.json file format

The BowerBundles.json file holds the data on what files are in reach bundle. This is equivalent MVC5′ s BundleConfig.cs in that you define your bundles here. I happen to think its a bit simpler than BundleConfig.cs, mainly because Grunt/Gulp is doing much of the hard work.

The key thing is that this file is the single source of what files are in what bundles. This file is used both by Grunt/Gulp to concatenate and minify the files and by B4B to deliver the right files at run time. It is also used by B4B’s CheckBundles (see later) class to check that your bundles are all correct.

The file is, by default, must be called BowerBundles.json and should be placed in MVC5’s App_Data directory. The file format is a json object which can contain a mixture of two formats: one for files delivered from your web application and a second format for using a Content Delivery Network (CDN) to delivery standard libraries

1. Delivery of files from your application

For delivering bundles of files from your application then each bundle is property that contains an array of strings holding the relative file references to each file you want in a bundle. Here is a simple example:

{
  "mainCss": [
    "lib/bootstrap/dist/css/bootstrap.css",
    "Content/site.css"
  ],
  "standardLibsJs": [
    "lib/jquery/dist/jquery.js",
    "lib/bootstrap/dist/js/bootstrap.js"
  ],
  "appLibsJs": [
    "Scripts/MyScripts/*.js"
  ]
}

The name of the property, e.g. mainCss is the name of the bundle and the array is the list of files in order that should be included. So to include the mainCss bundle you would include the command @Html.HtmlCssCached("mainCss") in your _Layout.cshtml file, or whatever View that needed it.

As you can see you can specify an exact file, or add a search string like "Scripts/MyScripts/*.js", but the order is then dependant on the name and some (many) files need to be loaded in a specific order. Directory searches can include file searches as well, e.g "Scripts/*/*.js", but at the moment I have not implemented the Grunt/Gulp’s /**/ search all directories and subdirectories feature.

Please see the section in the ReadMe file which gives you the steps to add a new file bundle.

2. Delivery of files from Content Delivery Network (CDN), with fallback

B4B can also handle the delivery of JavaScript via a Content Delivery Network (CDN). You can define a CDN url, with fallback in the BowerBundles.json file using the following syntax:

  "standardLibsCndJs": [
    {
      "development": "lib/jquery/dist/jquery.js",
      "production": "jquery.min.js",
      "cdnUrl": "https://ajax.aspnetcdn.com/ajax/jquery/jquery-2.1.4.min.js",
      "cdnSuccessTest": "window.jQuery"
    },
    {
      "development": "lib/bootstrap/dist/js/bootstrap.js",
      "production": "bootstrap.min.js",
      "cdnUrl": "https://ajax.aspnetcdn.com/ajax/bootstrap/3.3.5/bootstrap.min.js",
      "cdnSuccessTest": "window.jQuery && window.jQuery.fn && window.jQuery.fn.modal"
    }
  ]

The individual properties are explained in the B4B ReadMe CDN section but you can see from the data above we have both a ‘development’ file, which is delivered in debug mode, and a ‘production’ file that is delivered if the ‘cdnSuccessTest’ fails.

Using the above CDN bundle, standardLibsCdnJs, in your application will insert two <script> loads, each with a JavaScript section which used the {cdnSuccessTest} code to check that the CDN had loaded properly. If the test was false then the CDN worked and nothing else happens. However if it fails then the JavaScript inserts a extra <script> load to pull in the file given by the {production} property. The code output by the first CDN definition would look like this:

<script src='https://ajax.aspnetcdn.com/ajax/jquery/jquery-2.1.4.min.js'></script>;
<script>
   (window.jQuery||document.write(
   "\x3Cscript 'src=~/js/jquery.min.js?v=SnW8SeyCxQMkwmWggnI6zdSJoIVYPkVYHyM4jpW3jaQ\x3C/script>'));
</script>

Please see the section in the ReadMe file which gives you the steps to add a new CND bundle.

NOTE: At the moment I have not implemented CSS CDN support. The testing code is quite complex and I left it out for now. If someone wants to implement that then please let me know.

Adding a cachebuster to other static files

As well as bundles B4B can help with individual static files, e.g. images. These is a command called @Html.AddCacheBusterCached(“~/images/my-cat-image.jpg”). In this case a checksum of the file will be calculated based on its content and added as a cachebuster value.

In the case where you can pre-calculate the cachebsuter value then there is a second version which looks like this @Html.AddCacheBusterCached(“~/js/jquery.js”, “2.1.4”).

The way that the cachebuster is applied is set by the `StaticFileCaching` property in the B4B config. This means you can use different ways of applying caching busting by adding your own `BundlerForBower.json` file with a different cache busting scheme (see next section).

By default B4B uses the standard ASP.NET approach of adding a suffix, e.g. the command @Html.AddCacheBusterCached(“~/images/my-cat-image.jpg”) would produce the following html.

http://localhost:61427/images/my-cat-image.jpg?v=xKyBfWHW-GTt8h8i8iy9p5h4Gx9EszkidtaUrkwVwvY

Note: I use the SHA256 Hash which produces a hash which is related to the content. However this does take some time on large files, so I cache the SHA256 Hash to improve later access times.

B4B’s options: BundlerForBower.json

I have tried to make B4B flexible so I have put some of the key setting is a json file so that you can override them if you want to change things. I is also useful for me as ASP.NET Core 1 will need different settings. Below is the default setting for B4B, held in the file defaultConfig.json.

{
 "BundlesFileName": "BowerBundles.json",
 "StaticFileCaching": "{fileUrl}?v={cachebuster}",
 "JsDirectory": "js/",
 "JsDebugHtmlFormatString": "<script src='{fileUrl}'></script>",
 "JsNonDebugHtmlFormatString": "<script src='{fileUrl}?v={cachebuster}'></script>",
 //see http://stackoverflow.com/a/236106/1434764 about why we need to escape the document.write()
 "JsCdnHtmlFormatString": "<script src='{cdnUrl}'></script><script>({cdnSuccessTest}||document.write(\"\\x3Cscript src='{fileUrl}?v={cachebuster}'\\x3C/script>\"));</script>",
 "CssDirectory": "css/",
 "CssDebugHtmlFormatString": "<link href='{fileUrl}' rel='stylesheet'/>",
 "CssNonDebugHtmlFormatString": "<link href='{fileUrl}?v={cachebuster}' rel='stylesheet'/>",
 "CssCdnHtmlFormatString": "" //I have not currently implemented CDN for CSS files. Doable, but complicated.
}

I have used meaningful names to make the setting more comprehensible. These cover things like the names/locations of the directories where the minified files are found and the <link> and <script> code they output.

Can I point out that you can see a parameter in the setup called {cachebuster}. B4B adds a suffix to production files, just like MVC’s BundleConfig does, so that if the file changes the the new file will be used rather than the previous version in the browers local cache.

I actually use a SHA256 Hash as the cachebuster suffix rather that say the time the file was last written. This allows me to just rebuild everything and the caching suffix won’t change on files where the content is the same.

Changing the B4B options

I have used a flat object structure as that allows you to override just the item(s) you want while leaving the other properties at their default state. For instance to override just the directory where B4B looked for the JavaScript minified files then you would place the following json in a file called BundlerForBower.json in the MVC App_Data directory.

{
  "JsDirectory": "differentTopDir/bundles/"
}

See the following examples from the sample application:

  1. Override just the name of the bundle file see this file
  2. Override all the properties – see ASPNET Core 1 Config/bundlerForBower.json

Note: This last example shows how you would change the setting to match what ASP.NET Core 1 would need.

Unit Testing your bundles

Early on I was using a prototype of B4B in an e-commerce site I am working on. I deployed some code to my test site and it didn’t quite do what I had thought it should. I realised I had changed some JavaScript code and had not rebuilt the minified file.

I am pretty paranoid about problems that could hit a production site so I build a fairly comprehensive set of tests to check for any problems in the JavaScript and CSS bundles. The class is called CheckBundles.

To Unit Test your bundles then you need to create the CheckBundles in such a way that it knows where you MVC project is. If you are using the standard setup then the ctor can work this out by giving a type that is in your MVC application, e.g.

var checker = new CheckBundles(typeof(BowerBundlerHelper));

Notes:

  1. I use typeof(BowerBundlerHelper) rather than something like typeof(HomeController) as I wanted a type that did not need me to add the System.Web.Mvc assembly to my Unit Tests.
  2. There are other version of the CheckBundles ctor if you have an unusual setup. Please consult the CheckBundles code.

You most likely should run two tests:

  1. checker.CheckAllBundlesAreValid(). This checks all the bundles found in the BowerBundles.json file and returns a list of error messages. If there are no errors it returns an empty collection. (see sample project CheckBundles Unit Test example). The rules its checks against are:
    – Does the bundle contain any file references?
    – Do any of those file references use a search string that B4B does not support, e.g /**/
    – Do all of those files and their directories exist?
    – Does the concat file exist? (can be turned off via ctor param if not using concat files).
    – Was the concat file updated more recently than all the files referenced in the bundle?
    – Does the minified file exist?
    – Was the minified file updated more recently than the concat file (or all the files referenced if no concat)?
    For CDN bundles it checks:
    – Does the configuration support CDN for this file type?
    – Does your CDN bundle contain all the properties that the CDN format string needs?
    – Does any of the file definitions contain a search pattern? That is not allowed.
    – Does the ‘Development’ file and the ‘Production’ file exist?
  2. checker.CheckBundleFileIsNotNewerThanMinifiedFiles(). This does what it says. It checks that you haven’t changed the BundleFile and not run the Grunt/Gulp build process to ensure the minified files are up to date.

There is a really good example of using these methods to check your MVC bundles in the sample application. Have a look at this Unit Test class which uses CheckBundles in an NUnit based Unit Test.  I find this very helpful.

Conclusion

Hopefully this article, plus the sample application with its own ReadMe files and Unit Tests will give you good idea on whether B4B could help you. I do recommend you look at the other article called “Converting your ASP.NET MVC5 application to use Bower, Grunt and Gulp” for an overview of how to use Bower etc and how B4B fits into this.

I would appreciate your feedback on B4B. I have used it and been very happy with it, but I haven’t created a NuGet package yet. Anyone got a good link on how to produce multiple versions for each .NET version?

Happy coding!

Reflections on Unit Testing in C# and JavaScript

It has been a while since I wrote my first blog post called ‘Why I’m not scared of JavaScript any more‘ and since then I have written a fair amount of JavaScript. One of the things that I have found interesting is how I use different approaches to Unit Testing in each language. I’m not saying I have the ‘best’ way of Unit Testing each language, in fact I think some areas are a bit weak, but maybe you will find my experiences and reflections useful.

If you want to get to the meat of the post skim the next three paragraphs that set the scene and jump to ‘The differences in how I Unit Test between the two languages’ section.


First, respect to Gilles Ruppert…

Before I start I should say that I was heavily helped on the JavaScript side by an excellent programmer called Gilles Ruppert. As I was new to JavaScript I sort help and my son recommended Gilles. Gilles worked for me for about 10 weeks and set up the architecture of the single page application that I needed. He also set up an excellent JavaScript development environment using Karma test runner, Mocha test framework, Expect assertions and finally Sinon stub/spy/mocks. I am building on the shoulders of a giant.

Setting the scene

have just finished the first release of Spatial Modeller™ , a medium sized ASP.NET MVC web application using ASP.NET MVC4 written in C# and a large single page application written in JavaScript using backbone with marionette.

When Gilles finished we had just under 300 JavaScript Unit Tests, a handful written by me. Now I have added a lot more code and written an additional 400+ Unit Tests, taking the JavaScript tests to over 700.

On the C# side I use NUnit unit testing framework with a small amount of Moq for mocking etc. I use Resharper for running the testing inside Visual Studio. I have used this combination for some years.

Numerically I have less Unit Tests than JavaScript, currently C# has just under 500. However each test is often much more complicated and test many items per test. The reasons for this difference in approach is one of the reasons I wrote this blog, so read on to find out why.

My style of Unit Testing

I really believe in Unit Testing and I don’t think I have a project that doesn’t used Unit Testing. However I am not a fan of Test Driven Development (TDD) as I have found that way I don’t come up with coherent design. Writing Unit Tests is not an excuse for not doing some good designing of the system first.


The differences in how I Unit Test between the two languages

After maybe 10 months of JavaScript Unit Testing (and 5+ years of C# Unit Testing) I find it really interesting that I use different styles between the two. Here are the raw  differences and I’ll make some conclusions at the end.

1. In C# my first Unit Test in the compiler

Because C# is typed the compiler gives me lots of feedback. Therefore I my refactor code before I have even run it if I see opportunities to improve the code. This doesn’t happen with JavaScript, partly because JavaScript is not typed in the same way so tools like JSHint cannot give such comprehensive feedback.

(hint: In Visual Studio you can install Web Essentials which will then run JSHint and JSCS when a JavaScript file is closed. It may not be as good and the C# compiler at spotting things, but it can help spot silly mistakes.)

This means I am willing to write more C# code before I Unit Test than I would with JavaScript.

2. Large JavaScript is much easier to partition than C#

The ‘up’ side of not so tight checking in JavaScript is much easier to build in smaller chunks. The ‘interface’ between these parts is often an object which, by the nature of JavaScript, is not directly linked between modules. As well as making Mocking really easy it seems to help me think about each part separately.

In C# I use interfaces and layer to separate chunks of code, but for something complex  with five+ significant classes that I tend to think of them as a whole. The typed nature makes them more ‘linked’ than the JavaScript equivalent and mocking is harder.

3. The unit testing frameworks make a BIG difference

I have found that test frameworks like Mocha and Jasmine have some featured that encourage small tests. This is because these frameworks support ‘nested setups’, which other frameworks like NUnit don’t have. Let me explain what ‘nested setups’ are and why they encourage smaller, one item tests.

For most tests we need to setup some instances or data before we run the test. In NUnit we can run a setup method a) once at the start of the test class and/or b) once before each test is run. This is fine, but I find that I often need to run something specific before each test, which in NUnit you to at the beginning of the specific test. See example below where I run a ‘once at start’ method called ‘SetUpFixture’ (lines 1 to 6) and then an additional setup phase inside the specific test ‘CheckValidateTagError’ (line 14).

[TestFixtureSetUp]
public void SetUpFixture()
{
    using (var db = new SampleWebAppDb())
        DataLayerInitialise.ResetDatabaseToTestData(db, TestDataSelection.Simple);
}

[Test]
public void CheckValidateTagError()
{
    using (var db = new SampleWebAppDb())
    {
        //SETUP
        var existingTag = db.Tags.First();

        //ATTEMPT
        var dupTag = new Tag {Name = "duplicate slug", Slug = existingTag.Slug};
        db.Tags.Add(dupTag);
        var status = db.SaveChangesWithValidation();

        //VERIFY
        status.IsValid.ShouldEqual(false);
        status.Errors.Count.ShouldEqual(1);
        status.Errors[0].ErrorMessage.ShouldEqual("The Slug on tag 'duplicate slug' must be unique.");
    }
}

This fine, but I find that the individual setups can become quite long. I then have three choices if I have multiple things to check: a) duplicate the individual setup code in each test (Not good for DRY), b) write a small helper method which encapsulates the setup code (takes time and hides the setup), or c) test multiple items in the one test, which is what I have done above.

Both the Mocha and Jasmine JavaScript testing frameworks allow nesting of the setups, so in my last example I could have an outer group with the ‘SetUpFixture’ in it and then a small nested group, with a setup just for CheckValidateTagError, and then three separate tests for the three parts.

Here is an example from my actual code with some nested test groups, which Mocha does with the ‘describe’ command:

... various require code

describe('views/InfoPanel', function () {
    afterEach(function () {
        delete this.model;
        delete this.view;
    });

    describe('single item to display, no grades', function () {
        beforeEach(function () {
            this.layerInfo = helpers.loadFixture('/api/schemes/1').layerInfos[0];
            this.members = helpers.loadFixture('/api/layers/1/members');
            this.model = new InfoPanelModel({ selectedMembers: this.Members });
            this.view = new InfoPanelView({ model: this.model });
        });
        afterEach(function () {
            delete this.layerInfo;
            delete this.layer;
            delete this.members;
        });

        it('should be a Backbone.View', function () {
            expect(this.view).to.be.a(Backbone.View);
        });

        describe('after showing view', function () {
            beforeEach(function () {
                this.mainRegion.show(this.view);
            });

            it('should have something on view', function () {
                expect(this.view.$el).to.have.length(1);
            });

            it('title in window header should start with layer name', function () {
                ... etc.
            });

You can see the first setup, called ‘beforeEach’ on lines 10 to 15 and the the nested ‘beforeEach’ on lines 27 to 29. This works really well and is the reason why my JavaScript Unit Tests test almost always check one item per test. I really like Mocha’s approach and miss it in NUnit.

4. Mocking databases is hard!

One thing that definitely affects me is that the database side is hard to mock. I have used a number of ORMs, and direct T-SQL, and there isn’t a nice way to replace the real code with something for testing that will catch the same errors. Foreign keys, relational fixed etc. etc. is realy hard to mock (just have a look at my post Updating a many to many relationship in entity framework to see the complications that arise).

This means some of by Unit Tests in C# are more integration tests as I use the real database layer, but with a test database loaded with test data. There is a cost to that, but my experience is anything else lets errors through.

JavaScript using backbone.js does access data, but in a more controlled way and Gilles set up the classic ‘fixtures’ and filled it with test data created by the C# application. That makes JavaScript testing much easier, but only because the server is dealing with the complex parts.

5. PS: You must be able to debug inside Unit Tests

It goes without saying that Unit Tests are for finding bugs before you use the code in the real application. Therefore you MUST be able to debug code, with breakpoints and variable inspection etc., when you are running a unit test.

I mention this only because Resharper, which I find really useful, is very bad in this area when unit testing JavaScript. Resharper makes running unit tests really easy, especially on JavaScript using QUnit or Jasmine. However I have found it almost impossible to use it to debug Jasmine tests, and I assume QUnit tests, when running in the Resharper harness.

However the Mocha JavaScript test environment that Gilles set up is excellent on that score. It allows running with connected browser, use of debugger; statement to cause pauses when the debugger is open and the .only() statement to allow only certain tests to be run.

How this affects my Unit Testing

  1. My JavaScript Unit Tests are shorter and just test one item per test.
  2. I write new Unit Test groups more often in JavaScript than in C#.
  3. My C# Unit Tests are often more complex and often test multiple items per test. This is because:
    1. The setup code is complicated or time consuming.
    2. NUnit does not have nested setups like Mocha does
      (see The unit testing frameworks make a BIG difference above).
  4. I often write more C# code before writing Unit Tests as the the compiler catches a lot of ‘silly’ mistakes in C# which can get through in JavaScript.
  5. C# sees more ‘linked together’ than JavaScript, which affects mocking and designing.

I know that testing multiple items in one test, point 3 above, is often seen as ‘wrong’ by some, but cost of both code to set it up and time to run the test are all factors I have weighted up in coming to my current style.

I have more C# code than JavaScript code by maybe 3 to 1. My 500 C# Unit Tests take nearly 2 minutes to run while my 700 JavaScript Unit Tests take 30 seconds.

Conclusion

I like both C# and JavaScript. They each have their strengths and weaknesses. I also could not work a project without the feedback that Unit Tests give me on the quality of the software.

With all of this in mind it is worth thinking about how you Unit Test in each language. The language, the need, the project and the environment will change the way Unit Tests can and should be approached. Happy programming.

Why I’m not scared of JavaScript any more

My recent programming experience has been in C# and .NET, mainly on windows apps to support my business. However I needed to move to a web-based application, especially for our work in Africa.

The problem was the only bits of JavaScript I had seen up to this point was a few lines to say handle a button click. It looked very unstructured and, for a person used to OOP and C#,  rather nasty. So, what do I do in these cases… I bought a book, or in this case I bought two books. The effect was magic.

javascript design patterns bookBook 1: Learning JavaScript Design Patterns

The first book I read was Learning JavaScript Design Patterns by Addy Osmani. I bought this because a) it was on design patterns and b) it was very recent. The last is important because JavaScript is developing at such a fast pace.

This book was just what I needed. I opened my eyes to how to Construct JavaScript cleanly and provide good structure to my programming. It made me much more confident of how to write well structured JavaScript.

I really recommend this book to anyone who is a programmer but doesn’t really know about the new ways of  building JavaScript programs.

effective javascript bookBook2: Effective JavaScript

However I knew that a book on design patterns wouldn’t tell me the nitty gritty of the language. I already had the book JavaScript: The Good Parts by Douglas Crockford, but that hadn’t helped me to get inside the thinking of JavaScript.

I found a recent book called Effective JavaScript by David Herman. I have two Effective C# books and they really get into the depths of C#, so I got the JavaScript version. I was not disappointed.

Where the Design Pattern book gave me the overview the Effective JavaScript book took me deep into the inner workings of JavaScript. Also, from skim reading it to get a good idea of what to do and, more importantly, what NOT to do. This is now my bible of good coding standards in JavaScript.

Did the books help?

So, did reading these books make me a great JavaScript programmer? Of course not. But I was confident enough to design a fairly complex geospatial data visualisation system in JavaScript to work with OpenLayers.

I then when on to write the first basic module and some constructors to do the initial display of data on a map. They have a nice separation of concerns and when I wanted to refactor something it was nice and easy. It also came together quite easily with few bugs, although I really miss not having Unit Tests (that is another blog post to come!) Its likely I will get a JavaScript Guru to build the proper system, but at least I feel confident that I know what is going on now.

Maybe in a future post I will write why I actually LIKE JavaScript now because there are some bits that are really nice. But more importantly JavaScript is now the primary (only?) way ahead for responsive web design.