At my customer the good old problem of needing to run tests as different users came up again. I have tried to solve this before and came up with parts of the login dialogue handler.

The problem is this: when IE starts it uses information in the HKEY Current Users registry hive to decide who it should launch IE as. This prevents using an impersonated user token to start IE.

The other way is to use System.Diagnostics.Process start to start the process (or run as), but then you can’t control IE as it is running as another user.

Well as it turns out, the solution was staring me right in the face all along and the answer is to use BOTH of these methods, not one or the other.

First we need to launch a new IE process using System.Diagnostics.Process start with different user credentials, then we need to use impersonation to fire up watin in a another thread, attatch to IE and we are away. I have included the code which needs a Refactor and tidy up in a big way, but it works.

It won’t work on Vista with IE 7 because of the attatchto bug, but that is fixed in IE8, Also it is also leaving a process behind when the test finishes.

The impersonation code is from this Microsoft blog http://blogs.msdn.com/jimmytr/archive/2007/04/14/writing-test-code-with-impersonation.aspx

Enjoy,

[TestMethod]
    public void TestMethod()
    {
        SecureString password = new SecureString();
        password.AppendChar('p');
        password.AppendChar('a');
        password.AppendChar('s');
        password.AppendChar('s');
        password.AppendChar('w');
        password.AppendChar('o');
        password.AppendChar('r');
        password.AppendChar('d');

        ProcessStartInfo psi = new ProcessStartInfo();
        psi.UserName = "localtest";
        psi.Password = password;
        psi.UseShellExecute = false;
        psi.LoadUserProfile = true;
        psi.FileName = "c:\\Program Files\\Internet Explorer\\iexplore.exe";
        psi.Arguments = "about:blank";

        Process proc = new Process();
        proc.StartInfo = psi;
        proc.Start();

        t.Join();

        proc.Kill();
    }

    private static void DoWorkAs(object o)
    {
        User u = o as User;

        IntPtr hToken = IntPtr.Zero;
        IntPtr hTokenDuplicate = IntPtr.Zero;

        if (Win32.LogonUser(u.UserName, u.Domain, u.Password, 2 /*LOGON32_LOGON_INTERACTIVE*/, 0 /*LOGON32_PROVIDER_DEFAULT*/, out hToken))
        {
            if (Win32.DuplicateToken(hToken, 2, out hTokenDuplicate))
            {
                WindowsIdentity windowsIdentity = new WindowsIdentity(hTokenDuplicate);
                WindowsImpersonationContext impersonationContext = windowsIdentity.Impersonate();

                // domain\username
                Console.WriteLine(" Thread 2 : " + WindowsIdentity.GetCurrent().Name);

                IE ie = IE.AttachToIE(Find.ByUrl("about:blank"));

                ie.GoTo(@"http://www.google.com/");
                ie.TextField(Find.ByName("q")).TypeText("WatiN");
                ie.Button(Find.ByName("btnG")).Click();

                Assert.IsTrue(ie.ContainsText("WatiN"));
                ie.GoTo("about:blank");

                //revert
                impersonationContext.Undo();
                Console.WriteLine(WindowsIdentity.GetCurrent().Name);
            }
        }
        if (hToken != IntPtr.Zero) Win32.CloseHandle(hToken);
        if (hTokenDuplicate != IntPtr.Zero) Win32.CloseHandle(hTokenDuplicate);
    }

    public class User
    {
        public User(string u, string d, string p)
        {
            Domain = d;
            UserName = u;
            Password = p;
        }
        public string UserName;
        public string Domain;
        public string Password;
    }
    public class Win32
    {
        // P/Invoke snask
        [DllImport("advapi32.dll", SetLastError = true)]
        public static extern bool LogonUser(
            string lpszUsername,
            string lpszDomain,
            string lpszPassword,
            int dwLogonType,
            int dwLogonProvider,
            out IntPtr phToken);

        [DllImport("advapi32.dll", SetLastError = true)]
        public extern static bool DuplicateToken(IntPtr ExistingTokenHandle, int
           SECURITY_IMPERSONATION_LEVEL, out IntPtr DuplicateTokenHandle);

        [DllImport("kernel32.dll", SetLastError = true)]
        public static extern bool CloseHandle(IntPtr hHandle);

    }

I am pretty proud of our WatiN stack, it is in it’s third major version now and is an absolute joy to work with.

After seeing some other’s code samples online I thought that I should share some of the details so people can improve what they are doing. We recently hit a new one day record with one of our SDE/T’s writing 72 new test cases in a single day.

How it works

Our stack is broken down into four key layers, the test, a logical layer, a physical layer and then WatiN itself. If we are working on a Winforms application, we replace WatiN with an in-house developed stack that uses the Windows UI automation API’s. Our test data is stored in a SQL express database and we have a generated data access layer, which keeps the cost of creating CRUD methods very low. One of our tests looks like this:

[TestMethod]
public void VerifyCustomerCreation()
{
    Logical.Customer.CreateNewCustomer("Default Customer");
    Verification.Customer.VerifyCustomerCreated();
}

And that’s it. The tests are written in the business domain of the application that we are testing, not in a pile of clicks and set values. The next level down the “logical” layer is called by the tests and looks like this:

public static void CreateNewCustomer(string customerName)
{
    Physical.Customer.GoToNewCustomerPage();
    Physical.Customer.EnterNewCustomerInfo(customerName);
    Physical.Customer.SaveNewCustomer();
}

We build our logical layer up by combining smaller atomic actions, this allows us to heavily use the DRY (don’t repeat yourself) approach.

Our physical layer is where the rubber hits the road and it looks like this:

public static void EnterNewCustomerInfo(string customerName)
{
    ControlHandler conroller = new ControlHandler();
    TestData.Customer customerDetails = TestData.Customer.GetCustomerDetails(customerName);

    controller.SetValue(NewCustomerPage.txtFirstName, customerDetails.FirstName);

    controller.Invoke(NewCustomerPage.btnSave);
}

The control handler fully abstracts WatiN, or another automation tool so we interact with any control by typically only calling either SetValue, Invoke or GetValue.

To define our controls we have an in-house developed tool where we drag a magnifying glass (Spy++ style) onto a page and it then captures all the controls and gives us a generated static class.

This stack is the result of a lot of investment in our time and represents, in my opinion, state of the art when it comes to web test automation. But then again, I am a bit biased because I wrote it :-) .

So I haven’t worked on my WatiN Mac os x port for a few days, but I got back into it last night. The main hurdle at the moment is that the System.Diagnostics.Process class is not fully baked on Mono 2.4 on the Mac platform, because it doesn’t implement the /proc filesystem that linux does.

The correct thing to do would be to learn how the Mach kernel process structure works and fix mono, but I haven’t programmed in non managed C since 1990 and I would be very dangerous, especially at such a low level in mono.

So the first approach was to use the parts of mono that were working, namley Proces.Start() to call a shell script that I wrote to /tmp, then redirect standard out to a file, then parse that file and act accordingly.

StreamWriter sw = new StreamWriter("/tmp/watin-listprocess.sh",false);
sw.WriteLine (@"ps x | grep" + processname + " | grep -v grep > /tmp/watin-processlist.out");
sw.Flush();
sw.Close();

Process shScriptProcess = new Process();
shScriptProcess.StartInfo = new ProcessStartInfo("/bin/sh","/tmp/watin-listprocesses.sh");
shScriptProcess.Start();

A hack, yes, in fact a very messy hack. So messy there had to be a better way.

And there is …

Enter Monobjc. Monobjc is a managed wrapper around the Mac os x Cocoa API’s. A quick search pointed to the NSTask and NSPipe classes as a way to start a process with NSTask, and capture standard out with NSPipe, then parse the result.

A much more technically correct solution.

public static void Main(string[] args)
{
// spin up the objective-c runtime
ObjectiveCRuntime.LoadFramework("Cocoa");
ObjectiveCRuntime.Initialize();
NSAutoreleasePool pool = new NSAutoreleasePool();

// Create our process
NSTask task = new NSTask();
NSPipe standardOut = new NSPipe();
task.StandardOutput = standardOut;
task.LaunchPath = @"/bin/ps";

// add some arguments
NSString argumentString = new NSString("-ax");
NSArray arguments = NSArray.ArrayWithObject(argumentString);
task.Arguments = arguments;

// We have liftoff
task.Launch();

// Parse the output and display it to the console
NSData output = standardOut.FileHandleForReading.ReadDataToEndOfFile;
NSString outString = new NSString(output,NSStringEncoding.NSUTF8StringEncoding);
Console.WriteLine(outString);

// Dipose our objects, gotta love reference counting
pool.Release();
}

So this needs to be fully baked in and then put into my Watin on mac os x port and then it should work stand alone, without shell script assistance.

I didn’t think it would be this easy, but it’s working. I can now run a basic WatiN test on mac os x using mono.

The code is here http://code.google.com/p/mcwatin/. All credit to the WatiN guys, all I did was remove stuff :-)

Here’s what you need to get this running

Mono 2.4 and MonoDevelop preview for mac os x from here
The source code from http://code.google.com/p/mcwatin/
Firefox 3 with JSSH installed.

start firefox up with the following command line

/Applications/Firefox.app/Contents/MacOS/firefox -jssh

Then run the test. It will work but fail as exceptions are being thrown in the dispose method.

1. Get a subset of WatiN to compile with no PInvokes done

2. Get McWatiN to successfully start firefox. done

3. get McWatiN to run a basic test. done

4. Replace all the windows and IE stuff that was removed with code from Watir.

Building: McWatiN (Debug)

Building Solution McWatiN

Building: McWatiN (Debug)
Performing main compilation...

Build complete -- 0 errors, 0 warnings

---------------------- Done ----------------------

Build successful.

Two words that mean so much … this time they refer to WatiN built using MonoDevelop on Mac OS X.

1. Get a subset of WatiN to compile with no PInvokes done

2. Get McWatiN to successfully start firefox.

2. Replace all the windows and IE stuff that was removed with code from Watir.

My presentation at the Microsoft SDC open day alongside Devtest CEO Sarah Richey is now up on Microsoft’s site.

MePresentingAtMicrosoft.jpg

http://www.microsoft.com/australia/services/microsoftservices/sdc_openday.mspx

For my presentation you need to open the “How we do: Testing” video and I am in the last half following Sarah.

I am currently thinking about developing and delivering a WatiN training course. Is this something that anyone would be interested in ? If so, drop me a line via the comments on this blog.

Next Monday will be interesting, as I am presenting at the SDC Open day alongside Devtest CEO, Sarah Richey.

Here’s the blurb from the Microsoft site.

In challenging economic times, software development projects that are critical enablers to achieving business goals, will be looked at favorably and expected to deliver the results with a high degree of certainty. At the SDC open day, Microsoft’s own software development arm the Solutions Development Center (SDC) will present and share with you its formula to achieving certainty and success in project delivery.

You will hear from the SDC project teams on how they apply agile planning and development processes to deliver the right solution. How metrics-driven project management is used to keep the project on track. How daily builds and deployments help to maintain the project “heart-beat”, and how the SDC incorporates unit testing and automated testing into the process to ensure ongoing quality. .

You will also learn how Microsoft development tools and technologies are put to use throughout the software development lifecycle to enhance individual and team productivity.

Finally, you’ll have the opportunity to step inside the SDC facility and learn how the workspace is used to promote innovation, team collaboration and knowledge sharing.

I only have 10 minutes so it will be a “short and sweet” session, and attendance has closed, but there will be ~130 people in attendance.

Generally there are two key factors at play here: requirement delivery and bug fix rates.

The simple way

The simple answer is to average the number of bugs fixed per day, and divide the total number of bugs by the average. That is approximately how many days until you reach zero bugs. So, if you are fixing 5 bugs per day and you have 200 active bugs, the earliest that you will ship is in 40 working days time. If you want to ship sooner, you will need to stop adding features and focus on fixing more bugs.

The same information can be used in reverse to calculate a maximum allowable bug count. Say you only have 40 days until your desired ship date, and you are fixing 5 bugs per day as in the previous example. If you active bug count is over 200 today, you will probably miss your target. This number continuously decreases so in 2 weeks time, with 30 working days to go, your bug count should be at the 150 mark if you are going to hit your ship date.

The advanced way

In the 1950’s there was a little project that was underway at the Los Alamos National Laboratory in New Mexico. The manhattan project, required some advanced simulations, and they named the method they used after the Monte Carlo casino, which can be found at Turn 4 of the modern Monaco Formula 1 Circuit. There are essentially 4 steps to performing a Monte Carlo simulation:

1. Define the inputs in to the problem.
2. Randomly generate inputs for your problem.
3. Apply the random values you generated to your problem.
4. Repeat step 3, enough times to get statistically valid results, and then combine and analyse them.

Applying this method to our problem the steps become the following, assuming a data set where the bugs fixed – bugs found for a 2 weeks period results in the following data : 2,3,1,4,0,2,2,1,2,3

1. Determine the average and standard deviation of bugs being fixed over time. For our example the average is 2 and the standard deviation is 3.16.

2. Using Microsoft Excel, use the following function to generate valid random data =NORMINV(RAND(),Average,StandardDeviation).

3. In an Excel worksheet use the values that you generated in step 1, to workout the number of days to ship, I used an IF function and then COUNTIF to count if the days to ship was greater than zero.

4. In excel, copy the above 100 times and then plot the results.

ESDRawPlot.gif
ESDShippingChance.gif

The first chart is the scatter plot of the simulations. The second one shows the standard deviation of the estimated ship dates, and shows the chance of shipping in X number of days in the future.

The interesting thing about this model, is that even for 20 defects, and an average of fixing 2 per day the minimum days to ship is 3, but the maximum is 37 !

I saw this today whilst analysing automation failures, and it made me laugh.

Assert.AreEqual failed. Expected:<10904.73>, Actual:<10904.73>.

So this is what happens when you compare two different types of double precision numbers …