Teknologika

This post is the third in a series of posts, building a GUI test automation framework. In the first post, I outlined the key design goals of an automation framework. In the second second post I talked to the need for a control file. Since the original posts way back in early 2006, a lot has changed. As a result I am going to re-focus (and finish) these posts on how you can structure and develop a set of simple tests using WatiN.

Ok there are some pre-requites if you want to run these examples. I am going to assume that you are running Visual Studio 2005, SQL Express 2005, nUnit 2.4.0 and we are using the asp.net job site starter kit as our test application. I was planning to use the .net petshop 4.0 for this article, however it wouldn’t install with Windows Vista.

Ok assuming you are up and running, let’s get started.

The general structure that I like to follow for this type of project is to have a single solution broken up in to several projects. The standard projects that I like to have are Controls, Framework and Tests. In this example I am also going to add a fourth project to hold the site URLs.

Once I have these projects in place I set up the classes so they are all using single namespace that is then subdivided in a similar fashion such as: JobSite.Controls, JobSite.Tests and JobSite.URLs.

You can download the solution with the basic structure from here. I have set up a Codeplex project http://www.codeplex.com/WatinJobsiteSample/ to store the source code for the examples in the rest of this series of blog posts. In the next post we will get seriously back on track and start putting our test framework together.

As you can tell from my blog, I am a big fan of WatiN. One thing that WatiN doesn’t do is test non IE stuff. Well as part of .net 3.0 Microsoft has now shipped API’s that allow testing of Windows apps, including both legacy Win32 and WPF applications. There is an MSDN article on how to do this here.

So what does this mean? It essentially means that commercial tools are dead in the long term. If it is possible for a single developer to build tools that work better than commercially available tools costing hundreds or thousands of dollars, which it currently is. It won’t be long until they are not purchased any longer.

Another interesting phenomenon is that companies are trying to advertise their tools by adding comments to blogs, such as this one saying things like, you should try tool xxxx.

Sorry but when your commercial tool costs more and does less than the open source alternative, it is time to look for a new product to develop.

Another major irony is that if the open source tool has a bug, or doesn’t have a feature I need, then you can simply open up the code and fix it or add the feature. Most commercial vendors take 4-6 months or more to investigate, test and resolve issues.

So if you haven’t yet, download a copy of WatiN, WatiNRecorder, nUnit and C# Express, then see what the future is like and join in by submitting back features.

One final word, Webiussoft and Inscif, please stop trying to promote your commercial tools by adding comments to blogs about WatiN, we aren’t changing !

WatiN 1.0 has been released, you can get it here. The full release notes are here. This release includes a license change to the Apache 2.0 open source license.

Over the years we have noticed a phenomenon where developers can become reliant on testing. This addiction is a good thing, as it leads to a much better end product being developed. As a by product we have identified 5 stages that developers go through in their test addiction.

Stage 1 : Denial or “We don’t need testing, we are shipping fine without it thanks.”

Developers in stage one are under the illusion that you can simply compile you code and then ship it. Most developers at this stage are seeking compliance with the works on my machine certification. The particularly dangerous developer at this stage is the one who takes offence when his god-like development skills are revealed as capable if causing bugs just like everybody else.

Stage 2 : Annoyance or “The ‘ Pointy-haired boss’ has hired some outside testing company”

Someone from on-high who the developers think of as a pointy haired boss has forced some testers onto the project. It may come as a surprise, but some managers aren’t stupid and don’t believe a developer who says that they are “almost finished” or “95% done”. At this stage, testers are seen as little more than an annoyance who is forcing developers to do extra work like daily builds, smoke testing and tracking bugs.

Stage 3 : Acceptance or “I suppose this isn’t a temporary thing is it?”

At the acceptance stage, the developers have accepted that this new trend of testing may actually catch on, and that it isn’t just a passing fad. Developers will now be used to looking at their issues queues in the bug tracking application, and realise that, yes, they do need to tell the test automatiors that they have renamed the login button on the home page of the application.

Stage 4 : Dependence or “I am at one with my testing brothers and sisters”

After the acceptance stage comes dependence. At this stage the developers on a project have come to rely on the feedback loop that comes from fully engaging with the testers on your team. They get their code verified BEFORE they check it in, they rely on the suite of automated tests to verify that their changes have not broken and they may even from time to time raise the odd defect themselves.

You feel guilty doing any coding without unit tests. You half consider adding tests to the changes you just made to your autoexec.bat! (Thanks Russ !)

Stage 5 : Withdrawal or “Oh my god, what do you mean you don’t have any testers?”

The withdrawal stage when a developer who is used to being at level 5, changes jobs and is plummeted back into a team at stage 1. I’m sorry, but the prognosis for this situation is not good, and the only real solution is to progress your team through all 4 stages right from the very beginning. Who knows, if you suggest it, they may think that you are beginning to sound like a pointy-haired boss.

If you want to get your team to stage 4, feel free to drop me a line.