Stop doing TDD and start doing BDD with SharePoint!

I’m certainly one of those guys that like to do TDD and If you ask people what differentiates me from other developers you will almost certainly get the answer that I’m really serious about quality.  

But after my last project I’ve been become “wiser” or more careful with what kind of tests to use when and where. So why the unusually provoking title?

Last week I got a tweet from my friend Johan about tips on how to do TDD in SharePoint with assemblies that reside in the GAC.Stop doing TDD and start doing BDD with SharePoint

For a brief second I thought that I would just give him a list with the different methods/hacks/workarounds I’ve seen and then I thought:

“you’re only fighting the symptoms, we deserve better”

SharePoint lacks good mockable Interfaces and serves you with lots of sealed classes and static classes. That’s not a great foundation for TDD.

Furthermore we have the whole GAC thingy, also not a great foundation for TDD. If only I would have received a euro for every time someone in my last project asked me why the tests were not passing and whereby I asked “do you have an older assembly in the GAC?”

I can’t see how trying to do TDD in SharePoint will lead to anything but workarounds or hacks to get it working.

Another aspect of truly doing TDD is that you really need a vast knowledge of the domain and IMHO very few developers doing SharePoint development have the necessary knowledge.

In fact I did a test with two groups in my last SharePoint project. Both groups were told to TDD a small Document Set feature. Both groups were given 1½ hour. I coached the second team and 5 min into the exercise I told my group that let’s skip TDD and just do the feature.

The result was spectacular as you might have guessed. The first group almost didn’t make any progress at all and the second group didn’t come very far either. Why? Not enough knowledge of the domain and no mockable interface makes it almost impossible to do TDD in my opinion.

Still not convinced? Well it’s a free world go ahead, try it, fail and become wiser yourself.

I’ll even give you some methods I’ve used/seen/hacked/worked around the last couple of years (without any specific order of importance):

  1. Set Assembly version to 1.0.0.*, this way you won’t have any issues with the GAC thingy. But you’ll have lots of other side effects that you need to mitigate/automate.
  2. Retract dll’s in test setup/teardown.
  3. Write you class under test in the test project and then move it. Johan wrote about a great spin-off on this method here.
  4. Wrap SharePoint artifacts with your own Interfaces.
  5. Separate your domain from the infrastructure, and TDD the domain object only leave the infrastructure alone!
  6. Use a mocking technology like Fakes/Shims/Moles/TypeMock. This demands a really deep knowledge and understanding of SharePoint so if you think this is something you’d like to try, I say beware you’re treading in deep dark waters.

What you should do instead is to treat SharePoint as the framework it is much like the way you would treat the .Net Framework.

BDD is a nice way to do this, you invest more time in designing behaviors or capabilities of your application using SharePoint as one of those capabilities.

But that’s just my very personal 2  cents,

Hugo

Fakes, Stubs, Shims, Moles and how to verify them?

Background

If you have been following my posts here and here before you probably know that when I have to I use Moles from Microsoft Research. So the other day a friend of mine mentioned this new cool thing called Fakes, Stubs and Shims in Visual Studio 11 so I decided to look into that.

The first thing I actually did was to ask the following question on Twitter because I was curious to see if Moles and Fakes were similar:

Ok so Fakes in VS 11 are derived from Moles. My next question to Peter was one I’ve been thinking about alot lately:

So Microsoft are looking into adding verify support to Fakes, great!!! this means that you don’t have to use this hack going forward 🙂 Thanks Peter for that fast feedback!

For all you using Moles this hack will still apply. And for all you SharePoint developers out there; YES this is for you guys because you need to be doing more TDD!

My next step was to create a bootable vhd with Windows 8 Consumer Preview and Visual Studio 11 using my preferred method here and start hacking away.

Step 1

Get up to speed with Fakes, Stubs, Shims by reading “Using shims to isolate calls to non-virtual functions in unit test methods” found here; go on I’ll wait until you’re finished.

You’ll probably end up with a test method looking like the one shown below:

    [TestClass]
    public class Y2KCheckerTests_Step1
    {
        [TestMethod]
        [ExpectedException(typeof(ApplicationException))]
        public void Check_WhenCalledWithTheDate_2000_1_1_ThenApplicationExceptionIsThrowned()
        {
            using(ShimsContext.Create())
            {
                ShimDateTime.NowGet = () => { return new DateTime(2000, 1, 1); };

                Y2KChecker.Check();
            }
        }
    }

Well this is all fine, but what if you have a thousand tests that want to detour DateTime.Now? If you read my previous post here I introduced a Testable object that encapsulates behaviour.

Step 2

So the next step is to encapsulate this into a Testable object that I call Y2KShimDateTime like the code below shows:

[TestClass]
public class Y2KCheckerTests_Step2
{
    [TestMethod]
    [ExpectedException(typeof(ApplicationException))]
    public void Check_WhenCalledWithY2KShimDateTime_ThenApplicationExceptionIsThrowned()
    {
        using (ShimsContext.Create())
        {
            Y2KShimDateTime shim = new Y2KShimDateTime();

            Y2KChecker.Check();
        }
    }
}

public class Y2KShimDateTime
{
    public Y2KShimDateTime()
    {
        ShimDateTime.NowGet = () => { return new DateTime(2000, 1, 1); };
    }
}

Wow now we’re getting somewhere. We can reuse our Y2KShimDateTime in all our test methods. But hey what if we really just want to verify that a specific method has been called? You could just add some counter to the Y2KShimDateTime and call a property like the recommendation here.

Step 3

I use Moq to Mock my code because it’s simple and I love the way I can control return values with setups but mostly I love the way I can verify that a certain method was called with a specific value! If you haven’t looked at Moq yet I strongly encourage you to do so!

To accomplish this with Fakes, Shims or Moles you will need to trick Moq a bit (a hack I know). First of all let’s take a look at my verifiable Y2KShimDateTime class called VY2KShimDateTime.

public class VY2KShimDateTime
{
    public void InitFake()
    {
        ShimDateTime.NowGet = () =>
        {
            return VNowGet();
        };
    }

    public virtual DateTime VNowGet()
    {
        return new DateTime();
    }
}

The first part of the trick is to move the initialization of the detour from the constructor to a public method. This enables us to initialize the detours from Moq as you will see soon. The next thing is to create a public virtual method (VNowGet in this case) that will be called by Moq. There you have it! Easy enough?

Lets see how we use the VY2KShimDateTime from Moq.

using (ShimsContext.Create())
{
    Mock mock = new Mock();
    mock.Object.InitFake();
    mock.Setup(v => v.VNowGet()).Returns(new DateTime(2000, 1, 1));

    Y2KChecker.Check();
}

As you can see from the highlighted row above we’re initializing our detours after we create our mock this way we can be sure our detours will work. And now we can use the usual setup from Moq to Control what our detour will return. Brilliant don’t you say?

Let’s throw a verify into this equation and look how that looks.

using (ShimsContext.Create())
{
    Mock mock = new Mock();
    mock.Object.InitFake();
    mock.Setup(v => v.VNowGet()).Returns(new DateTime(1, 1, 1));

    Y2KChecker.Check();

    //Uncomment line below to see verify that mock.Verify throws correct message
    //Y2KChecker.Check();

    mock.Verify(v => v.VNowGet(), Times.Once());
}

In this case we’ll have to supply a date other than Y2K because otherwise we’ll get an exception but as you can see in the verify method we verify that DateTime.Now is called only once. If I would to uncomment the second Y2KChecker.Clean(); call the verify on our mock would throw an exception.

Summary

I’ve showed how to create verifiable Fakes, Stubs and Shims (works for Moles as well) using a small hack. I use this approach daily when I work with Moles because I really like the combination of Moq verify and setup and Fakes/Moles isolation and detouring.

Hopefully Microsoft will incorporate this in a near future in Fakes but until then give this a try.

All code can be downloaded from GitHub here.

Cheers,

Hugo

How Do I Do Moles?

Background

I’ve just recently started my first Open Source project (there I said it now I definitely have to deliver it). I can go so far to say that it has something to do with TFS 2010 and a Visual Studio package. I got you curious now right! Anyway there are some challenges to be solved when integrating your own code with existing APIs like the TFS Client API, it’s hard to unit test right? Well some would argue that you shouldn’t care to test your interaction with other APIs but for me that’s not an option. So I’ve chosen 2 basic strategies:

  1. Wrap the TFS Client API with my own classes and Interfaces.
  2. Test my own classes with the help from Moles. Moles is a Mocking framework from Microsoft Research.

I’ve written about my experiences with Moles here and you should really read that post if you’ve never used Moles before and want to continue reading this post.

A simple wrapper class

Lets start off with a very simple wrapper class that enables you to connect to a TFS 2010 instance.

using System;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Framework.Common;

namespace HugoHaggmark.Blog.Moles.HowDoIDoMoles
{
    public class TfsConnectionWrapper
    {
        private TfsTeamProjectCollection tfsTpc;
        public TfsConnectionWrapper()
        {
        }
        public void Connect(string url)
        {
            tfsTpc = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri(url));
            tfsTpc.Connect(ConnectOptions.None);
            tfsTpc.EnsureAuthenticated();
        }
    }
}

As you can see this wrapper class uses some TFS Client API like the call to TfsTeamProjectCollectionFactory.GetTeamProjectCollection for instance. Well if we take a close look at TfsTeamProjectCollectionFactory we’ll find the following declaration:


using System;
namespace Microsoft.TeamFoundation.Client 
{ 
    public static class TfsTeamProjectCollectionFactory 
    { }
}

As you can clearly see TfsTeamProjectCollectionFactory is a static class with now chance of sub classing or extending to enables some sort of mocking. So this is really where a mocking framework like Moles comes to play.

Before we start looking at any unit test code I must confess that I’ve started to categorize all my unit tests that use Moles with a test category called MolesTest. I started using the

[TestCategory(“MolesTest”)]

notation at first but then I just created my own TestCategoryAttribute like this:

using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace HugoHaggmark.Blog.Moles.HowDoIDoMoles.Tests
{
    public sealed class MolesTestAttribute : TestCategoryBaseAttribute
    {
        public override IList<string> TestCategories
        {
            get { return new[] { "Moles" }; }
        }
    }
}

Doing this enables me to just add MolesTest next to all my TestMethod attributes like so which makes it easy to filter Moles tests if necessary.

[TestMethod, MolesTest]

 

How I used to do Moles

The following example illustrates how I used to use Moles as part of my unit testing. (by the way looking at this code now makes my body shiver with disgust)

using System;
using System.Net;
using Microsoft.TeamFoundation.Client.Moles;
using Xunit;
namespace HugoHaggmark.Blog.Moles.HowDoIDoMoles.Tests
{
    [Microsoft.VisualStudio.TestTools.UnitTesting.TestClass]
    public class TfsConnectionWrapperTests
    {
        private const string validUrl = "http://valid";
        [Microsoft.VisualStudio.TestTools.UnitTesting.TestMethod, MolesTest]
        [Microsoft.VisualStudio.TestTools.UnitTesting.HostType("Moles")]
        public void using_an_valid_url_string_should_not_throw_the_old_way_I_used_to_do_moles()
        {
            Assert.DoesNotThrow(() =>
            {
                MTfsTeamProjectCollection mole = new MTfsTeamProjectCollection();
                MTfsConnection.AllInstances.ConnectConnectOptions = (connection, options) => { };
                MTfsConnection.AllInstances.EnsureAuthenticated = (connection) => { };
                MTfsTeamProjectCollectionFactory.GetTeamProjectCollectionUri = (uri) =>
                {
                    if (uri.Equals(new Uri(validUrl))) return mole;
                    throw new WebException();
                };

                TfsConnectionWrapper wrapper = new TfsConnectionWrapper(); wrapper.Connect(validUrl);
            });
        }
    }
}

Let’s go through a couple of things before I completely throw this in the recycle bin for ever:

  1. Yes I’m using xUnit.net as my preferred Assert framework, because I think it’s more descriptive then the default in Visual Studio and it makes the transition to xUnit.net in the future simpler. And because of that I need to prefix all my attributes with Microsoft.VisualStudio.TestTools.UnitTesting. So why not use xUnit all the way? Well that’s a whole other future blog post to be written. Let’s stick with fact that the combination xUnit, Moles and TFS Build and getting stuff published back to TFS caused me some serious headache in the past and that deserves a place in an other post.
  2. I’m not going to bother right now verifying the input to the Moles delegates except for the MTfsTeamProjectCollectionFactory.GetTeamProjectCollectionUri delegate where I want to make sure that any invalid uri throws.
  3. This would be my starting point then I would probably write another test method to make sure that passing invalid url would indeed throw an exception. In creating that second method I would of course refactor all my moles setups into separate methods. The end result would look something like so:
using System;
using System.Net;
using Microsoft.TeamFoundation.Client.Moles;
using Xunit;
namespace HugoHaggmark.Blog.Moles.HowDoIDoMoles.Tests
{
    [Microsoft.VisualStudio.TestTools.UnitTesting.TestClass]
    public class TfsConnectionWrapperTests
    {
        private const string validUrl = "http://valid";
        [Microsoft.VisualStudio.TestTools.UnitTesting.TestMethod, MolesTest]
        [Microsoft.VisualStudio.TestTools.UnitTesting.HostType("Moles")]
        public void using_an_valid_url_string_should_not_throw_the_old_way_I_used_to_do_moles()
        {
            Assert.DoesNotThrow(() =>
            {
                SetupAllMoles();
                TfsConnectionWrapper wrapper = new TfsConnectionWrapper();
                wrapper.Connect(validUrl);
            });
        }

        [Microsoft.VisualStudio.TestTools.UnitTesting.TestMethod, MolesTest]
        [Microsoft.VisualStudio.TestTools.UnitTesting.HostType("Moles")]
        public void using_an_invalid_url_string_should_throw_the_old_way_I_used_to_do_moles()
        {
            Assert.Throws<WebException>(() =>
            {
                SetupAllMoles();
                TfsConnectionWrapper wrapper = new TfsConnectionWrapper();
                wrapper.Connect("http://invalidurl");
            });
        }

        private static void SetupAllMoles()
        {
            SetupConnectMole();
            SetupEnsureAuthenticatedMole();
            SetupGetTeamProjectCollectionUriMole();
        }

        private static void SetupConnectMole()
        {
            MTfsConnection.AllInstances.ConnectConnectOptions = (connection, options) => { };
        }

        private static void SetupEnsureAuthenticatedMole()
        {
            MTfsConnection.AllInstances.EnsureAuthenticated = (connection) => { };
        }

        private static void SetupGetTeamProjectCollectionUriMole()
        {
            MTfsTeamProjectCollection mole = CreateTeamProjectCollectionMole();
            MTfsTeamProjectCollectionFactory.GetTeamProjectCollectionUri = (uri) =>
            {
                if (uri.Equals(new Uri(validUrl)))
                    return mole;
                throw new WebException();
            };
        }

        private static MTfsTeamProjectCollection CreateTeamProjectCollectionMole()
        {
            return new MTfsTeamProjectCollection();
        }
    }
}

So in the past I would feel reasonably happy with this and then move on with the next test case. But then I would come to a point that other test classes needed the same Moles setup and then I would create a class that all test classes derived from. Phuu…there had to be a better approach and as I never settle and always seek ways to improve my skills and code I’ve now reached an approach that for the moment feels comfortable.

Introducing Testable Moles classes

Nowadays I just create base classes that derive from (if possible) a Moles class. This way I can define my default behavior for my Testable Moles class and reuse it in any other test classes. And by marking the methods in the class used in the Moles delegates as virtual I can create different Testable Mole classes. Clever right? Let’s take a look at my implementation of a base class for a TestableTfsConnectionBase:

using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Client.Moles;
using Microsoft.TeamFoundation.Framework.Common;
namespace HugoHaggmark.Blog.Moles.HowDoIDoMoles.Tests
{
    internal abstract class TestableTfsConnectionBase : MTfsConnection
    {
        protected TestableTfsConnectionBase()
            : base(new MTfsTeamProjectCollection())
        {
            InitConnectMole();
            InitEnsureAuthenticatedMole();
        }

        private void InitEnsureAuthenticatedMole()
        {
            AllInstances.EnsureAuthenticated = (connection) =>
            {
                SetupEnsureAuthenticatedMole(connection);
            };
        }

        private void InitConnectMole()
        {
            AllInstances.ConnectConnectOptions = (connection, options) =>
            {
                SetupConnectMole(connection, options);
            };
        }

        protected virtual void SetupConnectMole(TfsConnection connection, ConnectOptions options)
        {
        }

        protected virtual void SetupEnsureAuthenticatedMole(TfsConnection connection)
        {
        }
    }
}

After this I would just go ahead and create a “default” test class

namespace HugoHaggmark.Blog.Moles.HowDoIDoMoles.Tests 
{ 
    internal class TestableTfsConnection : TestableTfsConnectionBase 
    { 
    } 
}

and if I need I can create another class that implements another behavior like the following class that always throws when trying to connect:

using System; 
using Microsoft.TeamFoundation.Client; 
using Microsoft.TeamFoundation.Framework.Common; 
namespace HugoHaggmark.Blog.Moles.HowDoIDoMoles.Tests 
{ 
    internal class TfsConnectionThatThrowsOnConnect : TestableTfsConnectionBase 
    { 
        protected override void SetupConnectMole(TfsConnection connection, ConnectOptions options) 
        { 
            throw new DivideByZeroException(); 
        } 
    } 
}

I hope you get the picture on how the testable classes works. Now let’s return to our unit test class and see how this would look in practice:

[Microsoft.VisualStudio.TestTools.UnitTesting.TestMethod, MolesTest] 
[Microsoft.VisualStudio.TestTools.UnitTesting.HostType("Moles")] 
public void using_a_connection_that_always_throws_should_throw_the_new_way_I_use_moles() 
{ 
    Assert.Throws<DivideByZeroException>(() => 
    { 
        MolesFactory.SetupMole<TfsConnectionThatThrowsOnConnect>(); 
        MolesFactory.SetupMole<TestableTfsTeamProjectCollection>(); 
        MolesFactory.SetupMole<TfsTeamProjectCollectionFactoryThatNeverThrows>(); 
        TfsConnectionWrapper wrapper = new TfsConnectionWrapper(); 
        wrapper.Connect("http://invalidurl"); 
    }); 

As you can see any new test method would just simply initialize any testable mole class needed and then of you go. I made a small class called MolesFactory that simply creates an instance of the supplied type in the SetupMole just because I believe it makes my code more descriptive and readable. The complete solution with code and binaries can be downloaded from here.

This is the first time I reveal my inner thoughts and code approaches so I really hope you got a clear understanding and some inspiration. Really looking forward to get feedback from the Moles community regarding the approaches described in this post.

Until next time,

Hugo