Das Tool hat den einfallsreichen Namen CAT.Net, was für Code Analysis Tool steht. Bei der Analyse bestehender Projekte hat das Tool einige Probleme aufgezeigt. Es zeigt, ähnlich wie der Stack-Trace, wie ein möglicher Angriffspfad im Code aussehen kann.

Das Tool scheint aber noch nicht ganz ausgereift zu sein, da es extrem viel RAM benötigt. Die Analyse eines größeren Projekts konnte zum Beispiel nicht vollständig durchgeführt werden da der RAM ausging.

Ich denke, dass dieses Tool bei jedem Web-Projekt zum Einsatz kommen sollte. Je früher man ein solches Tool bei der Entwicklung einsetzt, umso gezielter kann man Security Probleme umgehen. Ausschließen kann man sie natürlich nie ganz.

Zum Download:

• CAT.Net V1 für VS 2005/2008
• CAT.Net V2 beta für VS 2010

Die folgenden Bibliotheken und Tools könnten auch hilfreich sein:

First of all if you want to use MTLM it is necessary that you have a Team Foundation Server with version 2010 running. It is not possible to stick with an older version of TFS like 2008 if you are already using one. It’s not because Microsoft desperately want you to buy their new TFS version, it’s because MTLM uses some new features exclusively included in TFS 2010. You also need to be sure to run MTLM with administrative privileges, to have Windows Automation API3.0 and Windows Media Encoder installed on your system. After you checked that you are ready to start.

The overlying concept of MTLM you have to understand before starting: Test Cases, the smallest instance are separated into test steps. Those Test Cases can be grouped in Test Suits, which can be either static or dynamic. A static Test Suit has much in common with folders, dynamic Test Suits are more like filters to group specific Test Cases together. Test Suits run under a Test Plan, which is the highest instance in MTLM. With Test Plans you can assign the underlying Test Cases to specific Team Projects, define the test environment and enable test monitoring tools. For example MTLM offers screen video capturing of tests, screen shots of bugs, system information collection, event logs and the like. Features to make not only the testers but also the developers life easier because it simplifies the process for the developer to understand occurring bugs. This is where I see the real potential of this software. Fully included in a company software development lifecycle it shortens the gap between testers and developers. The included bug tracking systems that automatically collects all bug information (like screen, logs etc) contributes its part to this.

Designing and writing test cases is little effort. Just define the steps and the expected results for each test case and it can be executed by testers, even if they don’t have much developer background knowledge. To automate a test MTLM offers Action Recording. It stores each executed user input so that it can be replayed automatically afterwards, which can take a lot of boring work off testers. MTLM can be used for UI testing windows applications and web based applications likewise. While I encountered no troubles for testing windows applications, automation of web based applications can be tricky, frustrating or even impossible sometimes. For instance MTLM cannot be used with Firefox, period. Firefox is not supported by MTLM. Using a Test Case designed for Internet Explorer also cannot be used in another browser like Opera and vice versa.

Visual Studio also offers a new feature that is called CodedUiTest. It can be used to import automated Test Cases from MTLM into Visual Studio which are automatically converted to code. The general setup of a CodedUiTest is similar to a TestProject and can be executed likewise. Visual Studio also offers to edit the automated code (to insert asserts and the like) or even build a CodedUiTest from scratch using the CodedUiTest builder that much like MTLM collects user input information and thereafter generates the code needed.

Microsoft Test and Lab Manager seems to be a well though-out tool to increase effectiveness of a companies testing afford and is designed to shorten the gap between testers and developers. It offers a lot of functionality but naturally is limited to Microsoft products. The lack of support for Mozilla Firefox is disappointing and automation of web based applications can be frustrating and sometimes time consuming. The look and feel of the application takes time getting used to, and the application often feels clumsy when trying to automate a test because one false click means to redo the whole test case. Deleting the false step is not possible. It is Microsofts first release of their own Automation Framework and it’s still in beta phase so naturally the product isn’t perfect yet, but it’s certainly far from bad. I’m keeping track of MTLM and wait for the final release for a second look.

Selenium in fact is no tool itself, it consists of 3 Tools by name Selenium IDE, Selenium RC and Selenium Grid, that kind of build on each other to provide more functionality. Selenium IDE is a plug-in for Firefox, that allows to records users GUI input, save and replay it in test suits as test cases, thus allow an easy creation of automated web-based GUI tests. Easy to use often also means limited in terms of functionality by nature, that’s why Selenium IDE also offers automatic code generation for various programming languages including C#.NET. This short video provided on the Selenium homepage demonstrates the features of Selenium IDE most suitable: Selenium IDE Intro

The C# Code will be generated for use in the NUnit testing framework, but can be alerted with ease to work together with Visual Studio Unit testing for example. This is where Selenium RC (Remote Control) comes into play. In fact Selenium RC consists of an API that allows the programmer full control over the Selenium Scripting language and a server component that can open, control and close various web browsers like Internet Explorer, Firefox, Opera and Safari via JavaScript input. Selenium IDE and Selenium RC enables the tester to develop an automated test suite that runs on different operating systems and different browsers. While the test cases are easily recorded and converted the tester does still remain full control in terms of alteration. I think that this combination is what is making this tool so popular and why it has a broad community base.

The down terms of Selenium are that it is tight to web GUI testing. Selenium IDE is essentially a Firefox plug-in this means that the tests can only be automatically recorded when using Firefox, not Internet Explorer – of course tests can be run in IE too. I’ve also experienced some difficulties concerning pop ups or IE 8, which is very fussy about Cross Site Scripting Security. The fact that selenium in fact controls browsers via JavaScript input that is processed in the browsers built in JavaScript interpreter makes the IE 8 somewhat resistant to Selenium and it took some time to work around this IE8 “feature”. Going into my Selenium evaluaton in detail would of course go beyond the scope of this blog, but you can read more about Selenium in my upcoming evaluation if you are interested.

Next i’m going to focus on Microsoft’s Test and Lab Manager that comes with Visual Studio 2010. We’ll see how Microsoft handles test automation…

These keys are:

1. Know your Requirements: Thoroughly knowing and understanding the “System Under Test” is one of the most important factors that will impact the success of the AST implementation.
2. Develop the Automated Test Strategy: Within the automation strategy, we define the scope, objectives, approach, test framework, tools, test environment and schedule requirements related to the automated testing effort. It’s about the goals and non-goals.
3. Test the Automated Software Testing Framework: Automated Software Tests are human created and thus error prone as well. The ASTF has to be tested that it behaves like expected.
4. Track the progress, and adjust accordingly: ASTF could reach its limits in a while and maybe needs adjustments/improvements or to be exchanged.
5. Implement the AST Process: Implementing a successful AST effort requires a well-defined, structured, but of course lightweight process with minimal overhead. How to automate tests using minimum effort? And of course…
6. Put the right people on the Job: Because what would we do without our employees?

The HSG QA process is not marked in time thus is doing the next step to further improve this very aspect of software development – introducing automation into the software testing process within the scope of the project named “Implementing Automated Software Testing”.

Because developing our own ASTF is of course out of the scope, I’m going to evaluate existing ASTF candidates in the first step (“Picking the right tool for the job”). The ASTF names are Microsoft’s Test and Lab Manager (MTLM) included in the Visual Studio 2010 beta 2, IBM’s Functional Tester and Selenium – three ASTF that claim to offer a wide language and platform compatibility while easy to use.

During this process I’m also trying to acquire as much information on automation testing as possible. One of my major sources for this task is the book “Implementing Automated Software Testing” that is recently complementing our library. Needless to say that i’m keeping you up to date in this Blog on what’s going on in my “QA Corner” during the next months.

Folgen davon:

• Bisher wurden lediglich 145 Millionen Dollar, bzw. 7% der bereits beantragten 1.9 Milliarden Dollar, ausbezahlt.
• Das Programm wurde nun diese Woche, 2 Monate vor dem eigentlich angekündigten Ende, gestoppt, weil ihnen das Geld ausgegangen ist.

“Workers have reviewed about 40 percent of the applications filed, andmany have been rejected and then returned to the dealer for possible resubmission.” “dealers were not told why their applications had not been approved and were having to review the entire form to determine what went wrong.”

Hätte die Regierung eine Woche in ein Usability Review des Formulars gesteckt wären die Verwaltungskosten, die bereits jetzt nach Schätzungen des Ministeriums für Transportwesen und Verkehr bei ca. 100 Millionen US$ liegen, um einiges reduzierbar gewesen, die Ausfüller, in diesem Fall die Händler, wären weniger frustriert und die Kohle hätte um einiges schneller ausbezahlt werden können.
Selbst die NYTimes hat dieses Thema[1] sehr kritisch aufgegriffen.

so long
Michel

[1]http://www.nytimes.com/2009/08/21/business/21clunkers.html?_r=1