Category Archives

Test Management

The 5 stages of the developer-test relationship

Over the years we have noticed a phenomenon where developers can become reliant on testing. This addiction is a good thing, as it leads to a much better end product being developed. As a by product we have identified 5 stages that developers go through in their test addiction.

Stage 1 : Denial or “We don’t need testing, we are shipping fine without it thanks.”

Developers in stage one are under the illusion that you can simply compile you code and then ship it. Most developers at this stage are seeking compliance with the works on my machine certification. The particularly dangerous developer at this stage is the one who takes offence when his god-like development skills are revealed as capable if causing bugs just like everybody else.

Stage 2 : Annoyance or “The ‘ Pointy-haired boss’ has hired some outside testing company”

Someone from on-high who the developers think of as a pointy haired boss has forced some testers onto the project. It may come as a surprise, but some managers aren’t stupid and don’t believe a developer who says that they are “almost finished” or “95% done”. At this stage, testers are seen as little more than an annoyance who is forcing developers to do extra work like daily builds, smoke testing and tracking bugs.

Stage 3 : Acceptance or “I suppose this isn’t a temporary thing is it?”

At the acceptance stage, the developers have accepted that this new trend of testing may actually catch on, and that it isn’t just a passing fad. Developers will now be used to looking at their issues queues in the bug tracking application, and realise that, yes, they do need to tell the test automatiors that they have renamed the login button on the home page of the application.

Stage 4 : Dependence or “I am at one with my testing brothers and sisters”

After the acceptance stage comes dependence. At this stage the developers on a project have come to rely on the feedback loop that comes from fully engaging with the testers on your team. They get their code verified BEFORE they check it in, they rely on the suite of automated tests to verify that their changes have not broken and they may even from time to time raise the odd defect themselves.

You feel guilty doing any coding without unit tests. You half consider adding tests to the changes you just made to your autoexec.bat! (Thanks Russ !)

Stage 5 : Withdrawal or “Oh my god, what do you mean you don’t have any testers?”

The withdrawal stage when a developer who is used to being at level 5, changes jobs and is plummeted back into a team at stage 1. I’m sorry, but the prognosis for this situation is not good, and the only real solution is to progress your team through all 4 stages right from the very beginning. Who knows, if you suggest it, they may think that you are beginning to sound like a pointy-haired boss.

If you want to get your team to stage 4, feel free to drop me a line.

Test Management Testing3 comments

Software Development Metrics – Defect Tracking

In modern motor racing, such as formula one, each car is wired with a multitude of sensors that detect almost every possible influence or reaction on the racing car that it is possible to measure. Ride height, suspension travel, wheel spin, steering input and many other measurements all get recorded and analyzed in an effort to squeeze some more performance out of the race car. During a race engineers will monitor the engine performance and fuel consumption in real-time to help the tactical decisions that are made during a race, or slow the car down to avoid an impending engine failure. After the race is long over the data is analyzed to help tune the setup of future events and help evolve the car in an effort to win future races.

Software development can benefit in exactly the same way from this type of analysis and monitoring. Unlike a race car it isn’t necessary to purchase specialised measuring equipment to record data, as it is mostly already being recorded. However, like the racing counterpart it is necessary to spend some effort to yield some useable information and valuable results.

One of the most valuable tools that is available to a Software Development team are the statistics and metrics that can be recorded during a project, as it progresses, or in a post mortem at the end of a project. These different types of measures provide tactical and strategic benefits to an organization during a project. Tactical metrics are produced at a regular interval throughout the life of a project, (usually daily) and provide a close to real-time view into the health of a project.

Tactical metrics are typically produced by the test manager and include the defect trend graph, The defect trend is the single most valuable indicator that I use on my projects, it is the closest thing to an electrocardiogram (ECG) that I have seen for a software project. This graph plots active defects over the life of the project and provides one of the most valuable insights into the progress of both the testing system and the quality of the product itself.

So what exactly is an active defect? An Active defect is any defect that has not been closed, regardless of where it is in the defect cycle. For example, if a fix has been completed and is ready to be verified by a tester, is still counted as an active defect.

Other graphs that are quite valuable include reports that identify the most unstable components in the system, and test execution reports that indicate the amount of test coverage.

Another key graph is the “worm”, which shows daily how you are actually progressing compared to the original plan.

Test Management0 comments