I was hit by a car today, and I was one of the weirdest situations I have ever been in.

I was walking down the main street where I live, in front of some parked cars and one of them decided to drive away and not check if there was someone currently walking in front of the car.

From my point of view, I was just walking along and felt a sharp pain in my right knee and thought, what the frack was that? Next thing I know, I look down at my leg and there is a car bonnet attached to it, so that was a very weird sensation, and not something that I wish to repeat.

From “Making Windows XP Start Faster” at http://www.pcmag.com/article2/0,1759,1768883,00.asp

This post started at Microsoft WebBlogs: NNNNOOOOOoooooo….!.

Michael Howard, co-author of Writing Secure Code, posted on his weblog about a pc magazine article that suggested you should disable a couple of services to increase performance.

Two of the services listed under “Stopping Unneeded Startup Services”

Automatic Updates: This service enable Windows XP to check the Web automatically for updates. If you don’t want to use Automatic Updates, you can disable the service. You can always check for updates manually at the Windows Update Web site.

Windows Firewall/Internet Connection Sharing: If you do not use these features, you can disable them.

NNNNOOOOOoooooo….!

I thought that his post was simple, to the point, and I filed it into the “Who in their right mind would do that?” part of my mind. That was until I saw this today and almost fell off my chair…

Warning: The following content may offend some readers

D:\WINDOWS\ehome>doRunner.bat

D:\WINDOWS\ehome>

..\Microsoft.NET\Framework\v1.0.3705\CasPol -s off
Microsoft (R) .NET Framework CasPol 1.0.3705.6018
 Copyright (C) Microsoft Corporation 1998-2001. All rights reserved. 

 Success

... do some stuff  ...

D:\WINDOWS\ehome>..\Microsoft.NET\Framework\v1.0.3705\CasPol -s on
 Microsoft (R) .NET Framework CasPol 1.0.3705.6018
 Copyright (C) Microsoft Corporation 1998-2001. All rights reserved. 

 Success

Please excuse me for a moment, I am going to be sick …

Who in their right mind disables code access security for every .net application on a system, just to make their application work?

I heard somewhere that MS are considering either removing Caspol.exe all together from the Whidbey release. I would hate to see Caspol.exe disappear altogether, as the framework configuration control panels seem to break to often for my liking, and I need to resort to Caspol.exe to make things work. However please, please, PLEASE remove the -s off option. I am happy to live with whatever pain it causes, if it stops cowboys opening up my system to the world just to make their life easier.

If you have only walked in the dark, you will have never known the clarity that light brings. – me

This post is the fourth in a seven part series covering my seven tenets of software testing.

I was giving a presentation once to the CEO of the company that I worked for about the current state of play within our our organisation. I was Development Manager at the time, so testing was not my primary focus. During the presentaiton, I couldn’t resist including a couple of testing related slides. The first slide showed an example defect trend graph, which I used to illustrate the sort of information that should be generated by the Test Manager to assist with the day to day decisions. The second slide was the same graph with the data removed so that only the two axes remained, illustrating the lack of information available when there aren’t any testers logging issues.

Steve McConnell used a brilliant analogy in Code Complete¹, where he compares testing to a bathroom scale when you are trying to loose weight. Steve (or should that be Mr. McConnell) states that the scale does not help you loose weight at all. The scale is merely an indicator of your progress towards your goal.

In my way of thinking, to extend Steve’s analogy, a test team is more like a weight loss clinic. The statistics and metrics that they produce are like the weekly weigh in, and blood test results that tell the real story of how you are progressing.

Government health warning: Metrics can be addictive

I don’t smoke, but testing metrics are like a cigarette habit, once you are used to having them, it is almost impossible to give them up. You may be able to go for short, painful, stints without them, but you know it is a case of when they will be back, not if.

Metrics can provide insights and answers to curly questions such as: When will the product ship? The simple answer is to average the number of bugs fixed per day, and divide the total number of bugs by the average. That is approximately how many days until you reach zero bugs. So, if you are fixing 5 bugs per day and you have 200 active bugs, the earliest that you will ship is in 40 working days time. If you want to ship sooner, you will need to stop adding features and focus on fixing more bugs. The same information can be used in reverse to calculate a maximum allowable bug count. Say you only have 40 days until your desired ship date, and you are fixing 5 bugs per day as in the previous example. If you active bug count is over 200 today, you will probably miss your target. This number continuously decreases so in 2 weeks time, with 30 working days to go, your bug count should be at the 150 mark if you are going to hit your ship date.

Interpreting the results sometimes makes you feel more like a statistician than a test manager. But trust me, it is well worth the effort.

References

¹ Steve McConnell 1993, Code Complete, Microsoft Press.

I was reading a post on the The Unofficial Apple Weblog which concluded with the following line.

Now go finish your cereal before it gets soggy.

I have discussed the subject of reading blogs at breakfast with Chris, so I know that I am not the only one who reads their daily dose of blogs in this way. In fact, I would guess that there are a huge number of geeks that read technical blogs instead of having Sunrise or Today on in the background.



Oh by the way, a powerbook and wireless network solves the soggy cereal problem.

From notgartner.wordpress.com: Mitch Denny’s Blog: Importance of testers who code.

The Braidy Tester makes another great testing post. Given the amount he just wrote about tracking fixing up defective unit tests – it just proves how important the role of “tester who codes” is on a development team. Its depressing that there aren’t more of them around outside of the Redmond sphere.

As this is pretty much what I do for a living, and have done for years now, I thought that I should add my two cents worth as a comment on Mitch’s blog.

Mitch,

There are a few of us out here in OZ, a Gold Partner (that I used to work for), is owned by an ex Microsoft Test Manager that specialises in providing just those types of services, (www.devtest.com).

In Australia the term “Software Tester” carries quite a negative stigma compared to “Software Developer”, especially in the open job market. Try explaining to a recruiter that, no you aren’t a developer, but yes you have used .net since it was beta 1.

So many good people that would otherwise happily work as a tester who develops, move to straight development, otherwise they remain on a lower salary tier for no good reason.

Interestingly, with Test Driven Development, the average developer knows a lot more about testing than they used to, but as I have said in a previous post, nothing finds bugs like an experienced, knowledgeable tester.

Testers who can analyse code to write tests that highlight common coding errors like; boundary conditions and poor input validation, are much more effective than someone who performs black box testing, with no knowledge of how the internals of a system works.

Via Robert Hensing’s Incident Response Weblog…

Secunia.com have some statistics that show a stark contrast between Red Hat ES 3.0 and Windows Server 2003. Red hat comes in with 136 advisories and W2K3 comes in with 44 advisories. But here’s the kicker, the windows numbers have come in a period that is 18 months longer than red hat. On a per-month average Red hat tips the scales with 34 advisories per month compared to 2 per month for W2K3!!!

So it looks like anyone who is serious about security (and that should be everyone), should take a leaf out of Microsoft’s book and adopt their own version of the trustworthy computing initiative.

I also think this finally disproves the theory that an open source project will be more secure, simply because source code is available for scrutiny.

Micahel, aka the Braidy Tester’s, weblog is full of insightful posts and well worth reading. In his latest post; Grading on a curve, he mentioned what he referred to as the iceberg principle. Whilst I had never heard that one before, it certainly rings true based on my experience.

The Iceberg Principle – “For every bug you find there are nine related bugs lurking close by”

Steve McConnell also supports this in Code Complete¹ were he presents the following statistics:

80% of errors are found in 20% of a projects code

Now the fun really begins, as Indigo starts to see the light of day. Mitch Denny blogged about it here, and there is already an MSDN article based on a pre-CTP release, which, according to Mitch, will be available at the end of March.

This post is the third in a seven part series covering my seven tenets of software testing.

To start off, I’ll give credit where credit is due. I first came across this tenet in Microsoft Secrets¹ some years ago. Whilst the book is starting to show it’s age these days, it is full of some great little gems of information, and in the past I have made Chapter 5: “Developing and shipping products” required reading for members of my team.

The key idea behind the tenet is that testing starts the day development starts. This is a conscious move away from the waterfall approach where testers don’t get to start their testing until the developers have hit code complete. Starting testing so late in the process creates a situation where the true state of the product only becomes visible in the last third of the project or so. I don’t know about you, but if something is going off the rails, I want to know about it as soon as possible, so I can take some corrective actions before things get really ugly, and expensive to fix.

There are several techniques that can be utilised to start testing earlier in the process, adding significant value to the project.

Buddy Testing

In a perfect world where budgetary constraints don’t exist, (like say for the testers of the computers on Star Trek), a testing “buddy” is assigned to each and every developer. At the end of every day, the developer submits their code and hands over a private release to their testing buddy. The buddy tests the newly crafted code in its semifinished state, and provides immediate feedback to the developer, rectifying any issues before the code is integrated into the main build.

This practice is apparently in wide use throughout Microsoft, and I am led to believe that the ratio of developers to testers in times past was approximately 1:1. However the ration may be more or less these days depending on the product, the quality bar and the amount of automation that is being used.

Well I don’t work for Microsoft, and this ain’t Star Trek, so how can the rest of us utilise this technique?

There are a couple of ways that this approach can be adopted in the absence of unlimited testing resources. Firstly, a tester may be allocated to a number of developers, say, an entire feature team, and they test the feature as it develops, instead of only Joe’s code.

In the complete absence of testers, a developer could pair up with another developer who has sufficient objectivity and emotional detachment from the code that they are testing. (Typically this would need to be a developer working on a completely different feature). To encourage the buddy testing practice, issues found as part of a private release, won’t be entered into the defect tracking system, allowing the developer to resolve the issues as quickly as possible.

Test Driven Development (TDD) and developer unit testing
In the last couple of years, TDD and the nUnit style test harnesses have changed the unit testing landscape. nUnit formalises and automates the unit testing techniques that the better developers were doing in times past. This style of testing is a great technique to improve the quality of the code, and definitely should be utilised in one form or another.

The challenge however, is the developers emotional attachment with their code, Particularly when it comes to performing negative (destructive) tests. As a development / testing professional I can only say that nUnit based unit tests are a great thing, but, they are no replacement for someone who has no emotional attachment to the code, pounding away at it. This becomes particularly important as API only testing becomes less and less effective, (finding fewer and fewer bugs) as the product matures.

Daily build and smoke tests
Discussed in the previous tenet, the daily build and smoke test is a key foundation process for a development team that is serious about producing quality code. This practice that should always be implemented if at all possible.

Pre-Checkin tests
Whilst the daily build and smoke test is great at identifying when something is broken, the technique has a fundamental flaw in that the smoke test will not prevent the breakage from occurring in the first place. If a developer performs a smoke test after they do a local build, but before they submit their code then the problem may be caught before the main branch is broken. The challenge with pre-checkin tests is that they can significantly increase the amount of time that a developer will spend submitting their code. You can expect a lot of resistance from developers for this type of process. Especially if they are used to working on a small team and just checking in to VSS whenever they like. If your developers are used to following a controlled check in process that becomes necessary on larger projects, this should be easier to implement.

Performance testing and application profiling
Application performance is almost always an issue, and the judicious use of a profiler early on can help identify issues that may come home to roost later. Also just stepping though your code in the debugger can provide some valuable insight where time is being spent, although this becomes harder and harder the larger your code base becomes.

Code Reviews
Code reviews are another technique that can significantly improve the quality of software that is being developed. Code reviews can vary from a quick informal review to a full blown inspection. The costs and results will vary along with the formality, but at least some form of review should be scheduled during the development process.

Overall there are a number of different techniques that can be applied to an application as it is being built, and judicious use of the resources that are available can improve the quality of software, from the start of the development cycle.

References
¹ Michael A. Cusumano and Richard W. Selby 1995, Microsoft Secrets, pg. 294, Harper Collins.

via James Newkirk’s blog …

Microsoft’s patterns and practices team have just released the Enterprise Library.

The enterprise library contains the source code and unit tests for seven application blocks including:

• Caching
• Configuration
• Cryptography
• Data Access
• Exception Handling
• Logging and Security

At last the data access application block will finally support an OleDb provider as well as Sql Server, so Microsoft should be able to use it consistently across all their sample applications.

Also kudos to MS for including unit tests with the library.