Thursday, August 25, 2011

Code Coverage Tools

Code coverage allows us to see what lines of code have been executed. There are multiple ways of getting the code to be executed, but the most common way is unit tests. Your unit tests pass parameters into the methods, and as the lines are hit, they’re marked as covered. Clearly, this is not a good indicator of how solid your unit tests are, but there aren’t many other ways better that I can think of. You can also run the application to see what code is being executed, but this is less common.

I looked at NCover, TestDriven.Net, and DotCover. NCover was (may still be) the industry standard a few years ago and now went commercial. TestDriven.Net uses an open source version of NCover, and is viewed more as an add on into Visual Studio. DotCover behaves like an addition for ReSharper. All three work with most continuous integration servers.

NCover is really neat in that it offers a lot of reports and analysis on your code. It is neat to see the percentage of coverage change over time (Hopefully always rising). What I don’t like it is a standalone application, requiring a separate project. When am I ever going to look at code coverage when I’m not developing?

TestDriven.Net is the one I’m most familiar with. To me, if you take away the reporting from NCover, and add in some Visual Studio add-ins, you get TD.Net Additionally, it allows you to debug your unit tests, and easily select what fixtures/tests you want to run.

DotCover is relatively new, and I really like it. It is outstanding for TDD as it allows you to quickly execute tests repeatedly. The code higlighting is right in Visual Studio which I love, and it lists exactly what tests are hitting your lines. There have been issues with getting an accurate measurement from DotCover, but I didn’t experience them. I’ve had more issues with my unit Test Session, which is solved by closing the session and starting it again (which is relatively painless, and was an issue with Resharper).

I like having the tool built into Visual Studio, so I can run tests quickly after changes. Additionally, nCover is more expensive, and robust than what I need. TestDriven.Net is solid, but it just doesn’t have the features that DotCover does. Plus it is built in to ReSharper’s unit testing tools which I love, so it makes it even better for me. There are a few things to consider that I haven't gone into detail here, but everything points to DotCover being the way to go for me.