QA Corner: Microsoft Test and Lab Manager

The last couple of weeks I made a close up view on the Microsoft Test and Lab Manager (MTLM). It’s a new tool coming in bundle with Visual Studio 2010. It’s is not yet released and that’s why I tested this tool with version Beta 2. The MTLM is a tool that accomplishes Visual Studio with a testing component that is needed to fully develop a project within “the Microsoft way” during a full software development lifecycle. You can integrate your test suits and test cases with MTLM, run them or even automate them. So no need for 3rd party tools anymore? The purpose of this evaluation was to show that.

First of all if you want to use MTLM it is necessary that you have a Team Foundation Server with version 2010 running. It is not possible to stick with an older version of TFS like 2008 if you are already using one. It’s not because Microsoft desperately want you to buy their new TFS version, it’s because MTLM uses some new features exclusively included in TFS 2010. You also need to be sure to run MTLM with administrative privileges, to have Windows Automation API3.0 and Windows Media Encoder installed on your system. After you checked that you are ready to start.

The overlying concept of MTLM you have to understand before starting: Test Cases, the smallest instance are separated into test steps. Those Test Cases can be grouped in Test Suits, which can be either static or dynamic. A static Test Suit has much in common with folders, dynamic Test Suits are more like filters to group specific Test Cases together. Test Suits run under a Test Plan, which is the highest instance in MTLM. With Test Plans you can assign the underlying Test Cases to specific Team Projects, define the test environment and enable test monitoring tools. For example MTLM offers screen video capturing of tests, screen shots of bugs, system information collection, event logs and the like. Features to make not only the testers but also the developers life easier because it simplifies the process for the developer to understand occurring bugs. This is where I see the real potential of this software. Fully included in a company software development lifecycle it shortens the gap between testers and developers. The included bug tracking systems that automatically collects all bug information (like screen, logs etc) contributes its part to this.

Designing and writing test cases is little effort. Just define the steps and the expected results for each test case and it can be executed by testers, even if they don’t have much developer background knowledge. To automate a test MTLM offers Action Recording. It stores each executed user input so that it can be replayed automatically afterwards, which can take a lot of boring work off testers. MTLM can be used for UI testing windows applications and web based applications likewise. While I encountered no troubles for testing windows applications, automation of web based applications can be tricky, frustrating or even impossible sometimes. For instance MTLM cannot be used with Firefox, period. Firefox is not supported by MTLM. Using a Test Case designed for Internet Explorer also cannot be used in another browser like Opera and vice versa.

Visual Studio also offers a new feature that is called CodedUiTest. It can be used to import automated Test Cases from MTLM into Visual Studio which are automatically converted to code. The general setup of a CodedUiTest is similar to a TestProject and can be executed likewise. Visual Studio also offers to edit the automated code (to insert asserts and the like) or even build a CodedUiTest from scratch using the CodedUiTest builder that much like MTLM collects user input information and thereafter generates the code needed.

Microsoft Test and Lab Manager seems to be a well though-out tool to increase effectiveness of a companies testing afford and is designed to shorten the gap between testers and developers. It offers a lot of functionality but naturally is limited to Microsoft products. The lack of support for Mozilla Firefox is disappointing and automation of web based applications can be frustrating and sometimes time consuming. The look and feel of the application takes time getting used to, and the application often feels clumsy when trying to automate a test because one false click means to redo the whole test case. Deleting the false step is not possible. It is Microsofts first release of their own Automation Framework and it’s still in beta phase so naturally the product isn’t perfect yet, but it’s certainly far from bad. I’m keeping track of MTLM and wait for the final release for a second look.

Click here to cancel reply.

Jan201015

Jürgen Strodl

0