OpenCover, an excellent alternative to Visual Studio’s code coverage tool
The project I'm currently working on places a heavy emphasis on having a high percentage of the code covered by unit tests. As you may already know, Visual Studio provides a decent code coverage tool integrated into the IDE.
The problem...
Visual Studio's tool comes with a number of issues:
The lack of customizing what code gets checked for coverage besides the limited ExcludeFromCodeCoverageAttribute.
The lack of basic user interaction features like having the ability to filter the list of results produced by the tool.
The faulty computation of code coverage for code that makes use of the async and await keywords that has already been reported numerous times (StackOverflow, Microsoft Connect #1, Microsoft Connect #2)
The fact that this feature is only available for the commercial versions of Visual Studio.
With this in mind I decided to try OpenCover, an open-source project that seeks to provide precise code coverage information for .NET code licensed under the MIT Licence.
Integrating OpenCover in our existing project took around a couple of hours and produced excellent results and feedback from my team mates. These are the steps I followed in order to get everything running:
Right-click your solution in Solution Explorer and choose Manage NuGet Packages - Install the OpenCover and the ReportGenerator NuGet packages at solution level (no need to actually add these packages to any project, we need them here for ease of use)
Create a Windows batch file (or a Powershell script if that's your thing) that will contain the following lines:
Create a "coverage\MSTest" directory in the solution's root folder if it doesn't already exist: if not exist "%CD%\coverage\" mkdir "%CD%\coverage\" if not exist "%CD%\coverage\MSTest" mkdir "%CD%\coverage\MSTest
Delete any file in this directory if it exists: del "%CD%\coverage\MSTest\*.*" /F /Q
Rebuild your solution in order to have the latest .PDB file in line with the code (this is required for OpenCover to work correctly): "C:\Program Files (x86)\MSBuild\12.0\Bin\MSBuild.exe" FooBar.sln /p:Configuration=Debug /v:minimal /t:rebuild
Run the OpenCover console using the MSTest test runner on the assemblies containing the unit tests: "./packages/OpenCover.4.5.3723/OpenCover.Console.exe" -register:user -target:"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe" -targetargs:"/noisolation /testcontainer:"%CD%/ProjectOne.Tests/bin/Debug/ProjectOne.Tests.dll" "%CD%/ProjectTwo.Tests/bin/Debug/ProjectTwo.Tests.dll" /resultsfile:%CD%\coverage\MSTest\FooBar.trx" -excludebyattribute:*.ExcludeFromCodeCoverage* -filter:"+[*]FooBar.* -[*.Tests]*" -mergebyhash -skipautoprops -output:%CD%\coverage\MSTest\FooBarReport.xml Use the -filter argument to specify what code needs to be checked for coverage and what code is excluded. (See the official documentation for more information)
Run the ReportGenerator tool to get a nice looking report: "./packages/ReportGenerator.2.1.4.0/ReportGenerator.exe" -reports:"%CD%\coverage\MSTest\FoorBarReport.xml" -targetdir:".\coverage
And finally, open the report page with your favorite browser: start file://%CD%\coverage\index.htm
I also created a small Visual Studio plug-in that executes this batch file for ease of use.
Bear in mind all of the file paths for executables such as MSTest and MSBuild work for Visual Studio 2013. Other versions might have different locations for these executables.
In conclusion OpenCover proved to be superior to Visual Studio's implementation by being a lot more flexible in deciding what code gets checked for code coverage due to the filter argument, by giving accurate metrics results, even for the async and await keywords and by providing a better UI that allows the user to search in the generated report
