How I used Moles in a real life BizTalk scenario

Background

When you’re building BizTalk applications with TFS Build and then doing some deployment you pretty much end up with the situation where you DO NOT want to create BizTalk applications manually or automatically from some text-file. There is actually a place where this information is stored and it’s called a Bindings File.

A Binding File contains information specific for a BizTalk application and it’s in XML. If you really are into details I encourage you to read this article on MSDN. In short you should use Binding Files and only Binding Files as a part of your automatic deployment process. Thanks to Johan Hedberg and Mikael Håkansson for pointing this out.

Hey this has nothing to do with Moles! Be patient and start by installing Moles here are the free download links from the Visual Studio Code Gallery:

Moles

Moles is a lightweight framework for test stubs and detours in .NET that is based on delegates. Moles may be used to detour any .NET method, including non-virtual/static methods in sealed types. Moles is freely available on Visual Studio Gallery or bundled with Pex.

That’s what Microsoft Research introduces Moles as so now back to my real life scenario.

Writing extensions to the BindingInfo class with Moles

So there is actually a class in the assembly Microsoft.Biztalk.Deployment.dll that wraps the structure of a Binding File into a .Net class. So my thought was to write some helpful extensions methods that could help me in retrieving the name of the BizTalk application configured in the Binding File.

Are you still with me?

If we take a closer look at the BindingInfo class:

image

The BindingInfo has a default constructor that takes no parameters and then there is a LoadXml method that takes a string to the path of the Binding File. So now I have several options:

  • I could create my own implementation of BindingInfo and create a subclass that I could mock
  • I could use any other Mocking framework, like moq
  • I could use strictly integration tests (tests that use external resources) to test my extension methods

In this case I decided to use Moles because I like to try out new stuff. (Mind you this is not the most common scenario on when to use Moles but just bare with me)

Have you installed it yet? Good lets move on.

Adding a reference to a Moled assembly

Right click any referenced assembly and choose Add Moles Assembly like so:

image

Rebuild your test project and you’ll find two new items added to your project.

image

The first new item is a reference to an assembly that is the Moles representation of (in this case) Microsoft.BizTalk.Deployment.Moles.

And the second new item is a .moles file is a simple XML file telling Moles which Moles representation should be created at build time. This is how my Microsoft.BizTalk.Deployment.moles looks like:

image

This can have implications on build performance that I will try to write another post about soon.

Now you have everything in place to start writing your tests right? Let’s goooo! First of all you’ll need to decorate all your test methods where you’re going to use Moled objects with the HostType attribute. I use my own code snippet for this (based on the TestMethod code snippet already in Visual Studio).

image

And then start writing test cases. But how do you find your Moled objects? Well I usually find my Molable object using the complete namespace to the stuff you want to Mole + Moles + M + intellisense. Look at the example below:

image

If you use complete namespace to the stuff you want to Mole + Moles + S + intellisense then you’ll get the stubbable instances instead like so:

image

In my scenario I wanted to make sure that when the LoadXml method is called on any BindingInfo object in my tests it doesn’t read stuff from my disk but it just returns or initializes any properties I’m interested in. This is how I would write this:

image

A quick recap:

  • MBidingInfo was found using the complete namespace to the stuff you want to Mole + Moles + M + intellisense method mentioned above.
  • AllInstances means that for all instances I want to replace some method with my own delegate.
  • LoadXmlString is the method I want to Mole. After a while using Moles you’ll see that methods have there original name followed by the Type of the input parameters if any.
  • Then I’ve written my delegate declaration and body that takes in two parameters called instance and path.

To make this even more interesting you can add Asserts in the delegate (I say you can, I’m not sure this is a good practice) like the following picture:

image

Now I can continue writing my test as if my BindingInfo objects had loaded the xml from file. But how can I verify that some Moled method has been called? Well Moles doesn’t intend to provide you with Mocking framework abilities that you might be used to. Instead you’ll have to do this yourself. Here is a simple example how to test if a method is called.

image

So this was a real short introduction to Moles that I hope helps you come up to speed quickly. The final point that I would like to make is that you can use this on any .Net object even static or sealed objects like DirectoryInfo for instance (try using the code below without Moles):

image

Big thanks to my colleague Marcus Hammarberg for challenging me to write this post!

Enjoy,

Hugo

Build recipe for running unit tests using file masks in TFS Build 2008

Background

The background for this build script recipe can be found here and here. Instructions on how to install and configure a build server to run these build scripts can be found here and here.

The Challenge

The build recipe has to conform to these constraints/requirements:

  • No test lists
  • No maintenance of testsettings/testrunconfig files
  • Using file mask .Tests to identify which assemblies that contain unit tests
  • TFS 2008
  • Visual Studio 2010
  • BizTalk 2010
  • MSTest

The Dev.CI.testsettings file

As you might already know the testsettings file (formely know as testrunconfig) defines important configuration that tells MSTest various information; where to look for deployment items for instance.

I create one testsettings file per build that I’ve safely in source control and in that file I add this:

<Deployment>
  <DeploymentItem filename=\\bldserv\Bld\1\TestResults\DeplymentItems\/>
</Deployment>

These lines tells MSTest to look for deployment items in the specified directory/share. This works for now as we only have one build server that executes the CI-builds but when we add build servers later on we’ll have to come up with a better solution using environment variables for instance.

It would be great if TFS Build could automatically pick up the testsettings file for each solution in the build but as we only do ONE testrun per TFS Build that wouldn’t work either. The best would be if we ran MSTest / solution in the TFS Build.

The TFSBuild.proj file

Under the ItemGroup for testing I add the following line to tell MSTest to look for testcontainers with the “.Tests.dll” filemask.

<TestContainer Include="$(OutDir)\%2a.Tests.dll" />

The next step is to introduce a TFS Build override where we specify our testsettings file and do the copying to our deploymentitems folder specified by our testsettings file. There is also delete statement that I was added later because I found an assembly with the correct file mask but that wasn’t a unit test assembly, read more about that adventure here.

  <Target Name="BeforeTestConfiguration">

    <!–We need to set the path to our xml-files and schema-

    files for MSTest to include this can only be done from one

    single testsettings-file.–>

    <PropertyGroup>

      <RunConfigFile>$(SolutionRoot)Dev.CI.testsettings</RunConfigFile>

    </PropertyGroup>

   

    <!–Copy all testinstance-files.–>

    <CreateItem Include="$(SolutionRoot)\**\TestInstances\*">

      <Output TaskParameter ="Include" ItemName ="MyTestFiles"/>

    </CreateItem>

    <Copy SourceFiles="@(MyTestFiles)" 

          DestinationFolder="$(TestResultsRoot)\DeplymentItems"/>

    <!–Copy all schema-files.–>

    <CreateItem Include="$(SolutionRoot)\**\*.xsd">

      <Output TaskParameter ="Include" ItemName ="MySchemaFiles"/>

    </CreateItem>

    <Copy SourceFiles="@(MySchemaFiles)" 

          DestinationFolder="$(TestResultsRoot)\DeplymentItems"/>

    <!–Delete all non unittest-files.–>

    <CreateItem 

      Include="$(BinariesRoot)\**\[Customer].[BtsApp].OrchestrationsHelper.Tests.dll">

      <Output TaskParameter ="Include" ItemName ="MyNonUnitTestFiles"/>

    </CreateItem>

    <Delete Files="@(MyNonUnitTestFiles)"/>

  </Target>

The drawback to this approach is that your testinstance/schema files (often xml-files in BizTalk projects) need to be uniquely named or you would overwrite them in this step.

Lessons learned

  • You should start to think about automated build processes from day one in a dev project.
  • We need to find a way to not use hardcoded paths, maybe we could use environment variables.
  • Maybe we should look into other testing frameworks like XUnit, NUnit and see if they offer better support for this kind of situation

Please let me know what you think of this solution considering the constraints and requirements at hand. The examples for this post can be found here.

Cheers!

Hugo

The perils of using “.Tests” as a file mask for unit tests in TFS Build

Background

As some of you might already have read here I’m currently working with a large BizTalk implementation that has been going on for more or less a year.

So there’s already tons of unit tests in place and one of the first things I’d hade to do is setup some CI-builds to verify that each check-in keeps the integrity of the source control tree intact and run unit tests.

The Challenge

The hardest challenge when I start coaching a team with devs with no automated builds this late in the game is:

  • Talking about the value that automation brings to their day to day development work
  • Introducing automation without changing the already established routines, i.e. keeping balance between the new and established routines

Here is a team that already established a routine for creating unit test projects for instance where they name all their unit test projects [Customer].[BTSSystem].[BTSArtifact].Tests. Not going to change this routine right? I’m going to use it in my CI-builds instead right?

I create a CI-build (read more here) that looks for the file mask .Tests (we’re using TFS 2008) and off we go!

The Error

Some days later I check back in (I’m not at the customer every day, otherwise I would constantly monitor the builds)  to follow up how the builds are going and I found one very odd build that was constantly “partially succeeded”. Learning from my earlier mistakes a read my blog post here about “partially succeeded” builds and found nothing that could help me identify the error. Our build log looked like this:

Using “TestToolsTask” task from assembly “E:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies\Microsoft.VisualStudio.QualityTools.MSBuildTasks.dll”.
Task “TestToolsTask”
Command:
E:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /nologo
/runconfig:”E:\Blds\73\Sources\Dev\[Customer].Dev.CI.[BtsApp].testsettings”
/searchpathroot:”E:\Blds\73\Binaries\Debug”
/resultsfileroot:”E:\Blds\73\TestResults”
/testcontainer:”E:\Blds\73\Binaries\Debug\[Customer].[BtsApp].Maps.Tests.dll”
/testcontainer:”E:\Blds\73\Binaries\Debug\[Customer].[BtsApp].Orchestration.Tests.dll”
/publish:”http://sometfs:8080/”
/publishbuild:”vstfs:///Build/Build/6829″
/teamproject:”[Customer].
/platform:”Any CPU”
/flavor:”Debug”
The “TestToolsTask” task is using “MSTest.exe” from “E:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe”.
Loading E:\Blds\73\Sources\Dev\[Customer].Dev.CI.[BtsApp].testsettings…
Loading E:\Blds\73\Binaries\Debug\[Customer].[BtsApp].Maps.Tests.dll…
Loading E:\Blds\73\Binaries\Debug\[Customer].[BtsApp].Orchestration.Tests.dll…
Starting execution…
TESTTOOLSTASK : warning : The disabled test ‘SomeTest’ was removed from the test run. [E:\Blds\73\BuildType\TFSBuild.proj]

Results               Top Level Tests
——-               —————
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest1
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest2
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest3
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest4
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest5
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest6
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest7
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest8
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest9
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest10
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest11
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest12
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest13
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest14
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest15
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest16
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest17
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest18
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest19
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest20
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest21
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest22
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest23
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest24
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest25
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest26
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest27
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest28
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest29
Passed                [Customer].[BtsApp].Maps.Tests.SomeTest30
Passed                [Customer].[BtsApp].Orchestration.Tests.SomeTest1
Passed                [Customer].[BtsApp].Orchestration.Tests.SomeTest2
Passed                [Customer].[BtsApp].Orchestration.Tests.SomeTest3
Passed                [Customer].[BtsApp].Orchestration.Tests.SomeTest4
34/34 test(s) Passed

Summary
——-
Test Run Warning.
Passed  34
———-
Total   34
Results file:  E:\Blds\73\TestResults\SOME-TFS 2011-04-01 15_29_30_Any CPU_Debug.trx
Test Settings: Local

Run has the following issue(s):
Waiting to publish…
Publishing results of test run SOME-TFS 2011-04-01 15:29:30_Any CPU_Debug to http://sometfs:8080/…
..Publish completed successfully.
Command:
C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe /nologo
/runconfig:”E:\Blds\73\Sources\Dev\[Customer].Dev.CI.[BtsApp].testsettings”
/searchpathroot:”E:\Blds\73\Binaries\Debug”
/resultsfileroot:”E:\Blds\73\TestResults”
/testcontainer:”E:\Blds\73\Binaries\Debug\[Customer].[BtsApp].OrchestrationsHelper.Tests.dll”
/publish:”http://sometfs:8080/”
/publishbuild:”vstfs:///Build/Build/6829″
/teamproject:”[Customer].
/platform:”Any CPU”
/flavor:”Debug”
The “TestToolsTask” task is using “MSTest.exe” from “C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe”.

Microsoft (R) Test Execution Command Line Tool Version 9.0.21022.8
Copyright (c) Microsoft Corporation. All rights reserved.

The type initializer for ‘Microsoft.VisualStudio.TestTools.Utility.LicenseHelper’ threw an exception.
For switch syntax, type “MSTest /help”
MSBUILD : warning MSB6006: “MSTest.exe” exited with code 1. [E:\Blds\73\BuildType\TFSBuild.proj]
The previous error was converted to a warning because the task was called with ContinueOnError=true.
Build continuing because “ContinueOnError” on the task “TestToolsTask” is set to “true”.
Done executing task “TestToolsTask” — FAILED.

As you see all the tests are passing but after publishing the build calls the Visual Studio 2008 version of MSTest.exe. And this got me puzzled. But the error message The type initializer for ‘Microsoft.VisualStudio.TestTools.Utility.LicenseHelper’ threw an exception. got me really thinking…someone even suggested that I was messing with MS licensing, no WAY I would ever do that!

The Solution

The solution was really simple, someone in the devteam had created an ordinary BizTalk orchestration project named [Customer].[BtsApp].OrchestrationsHelper.Tests but because of that it WAS NOT a unit test project so that’s why the build returned that error.

The best solution would have been to rename the project but after asking the developers that renaming would have costs us a lot of time and refactoring so we decided to exclude that particular assembly from our tests. (details here)

Lessons learned

  • You should start to think about automated build processes from day one in a dev project.
  • Partially succeeded builds can also be a symptom of trying to execute “regular” assemblies through MSTest.

Enjoy!

Hugo