How come the add button in Associated Automation isn’t visible in Microsoft Test Manager?

This year is off to a great start with fantastic first week and an even better last day of that week working with my fellow Avega Coach, Marcus Hammarberg .

Our work yesterday is clearly summarized by Marcus in this post and I think Marcus has made some real groundbreaking thoughts on SpecFlow together with TFS and Visual Studio and I’m very glad to have helped out some parts. A shout out goes to Håkan Forss as he gave both of us some valuable feedback too.

So what’s this post about you might ask? Well during our work we stumbled onto a gotcha with the way Test Manager and associated automation works.

As you can see in the picture below I’ve created a Test Case that some tester later gets to run.

1

Suppose the tester runs this Test Case and finds a bug. Now we want to assign this bug to a developer who fixes the bug and associates a test with the test case in such way that the test case gets automated.

So we change the automation status to Planned as you can see in the picture below:

2

Problem

As you can see the tester can’t set Automation Status in Test Manager to Automated because that state is missing from the list and if we zoom to the right side of the tab control we find that there’s no way of associating any tests either.
2.5

Solution

Being that this was the first time I looked a this particular feature of Test Manager I must say that I got a little intrigued to say the least. After some Internet searching we still came up with nothing but when we opened up the same Test Case in Visual Studio we finally saw the add button (marked with … in the picture below):

2.51

Digging deeper

So it got very clear to me then that this behavior was by design but I still felt that I needed to go behind the scenes to see how this is solved because in the end Test Case is just your average Work Item Type which is defined in XML.

Looking at the Automation Status field in the Test Case WIT XML you’ll see this:

3

As you can see in the XML you’ll never see the automated state if you haven’t assigned a Microsoft.VSTS.TCM.AutomatedTestId value which is exactly what you do when you associate a test with the add button like the picture below:

7

Looking at the Associated Automation tab in the Test Case WIT you’ll see this:

4

As you can see the Associated Automation tab implemented with a control called AssociatedAutomationControl. After some investigation with my favorite reflection utility .Net Reflector I found this section marked in red:

10

It looks like depending on which ServiceProvider that is used when creating the AssociatedAutomationControl the Add button will be visible or not, TADA!

To sum things up:

  1. You can’t associate automation with unit tests within Test Manager
  2. You can associate automation with unit tests within Visual Studio
  3. All this is because the ServiceProvider associated with the current AssociatedAutomationControl controls the visibility of the Add button in the Associate Automation tab.

Cheers!

Hugo

10 Replies to “How come the add button in Associated Automation isn’t visible in Microsoft Test Manager?”

  1. Well done Hugo!

    Talk about deep dive. Still strange I think but at least now we know why.

    If you think on why they have designed it that way i can only think that the reasoning is that the automation is created by a developer and hence the developer should set the associated automation field.

    That doesn’t rhyme to well with the Specification by example thoughts on process where you write the specs and scenarios together but then again we’re not there yet.

    It probably has to do with the Test Manager tool being tightly tied to an old school view on what the tester does.

    1. First of all thanks!

      Secondly, now that I’ve done this digging I find it quite logical. To even associate a unit test with a test case in Visual Studio you need to have an open and compiled test project in your solution. Otherwise the Associate Test dialog will be empty.

      Thirdly, getting all this to run in an automated fashion from Microsoft Test Manager is whole other blog post it seems; stay tuned folks!

      Finally it would have been nicer to read the specs for Microsoft Test Manager instead of reflecting it 😉

  2. Thank you for the post!

    I am a little confused about how to actually make ‘Add Associated Automation’ option available in Test Manager. Are you saying it is not possible and it can only be done via Visual Studio?

    Regards,
    shuyi

    1. Thank you for your comment.

      At the time writing the post and with no Service Packs and/or Feature Packs installed it was NOT possible to make the ‘Add Assoiciated Automation’ available in Test Manager, only in Visual Studio. And furthermore I don’t know If this has been fixed in the 2012 version or any later Feature Pack.

      Regards,

      Hugo

Leave a Reply

Your email address will not be published. Required fields are marked *