What is website browser compatibility testing?

The goal of browser compatibility testing is to ensure that the site renders without error on the target web browsers. Some minor rendering differences are expected from browser to browser, or within different versions of the same browser.

The image below shows the rendering differences between Internet explorer 5.5 (which is ghosted) and Internet Explorer 6 for Microsoft’s homepage.

What you can clearly see is that the main page elements in IE5.5 are clearly moved to the left. You can also see that the navigation bar is not working at all, and that silverlight is not supported on IE 5.5.

These types of rendering differences will occur typically across major versions of browsers, but as the same rendering engine is typically used on all platforms for multi-platform browsers, the same version of the same browser will typically render the same result on all platforms that it supports. This reduces the need for testing the same version of a browser on multiple platforms, however simple sanity checking of browsers cross platform is recommended if the configuration is already available.

So which browsers should I test?

To determine where you should focus your testing effort, let’s look at the browser market share at the time of writing (December 2008)

Source: http://marketshare.hitslink.com/

Generally a browser should be tested if it meets one of the following criteria:
• It is used by a significant portion of a sites users, and is in common, widespread usage.
• It is the default browser on the latest version of Windows or Mac OS X.
• It is a newly released browser which is expected to quickly gain a significant portion of browser market share, e.g. IE 8 or Google Chrome

After reviewing the data against those criteria, the following browsers are suggested for inclusion for testing most sites:

What bugs am I likely to find?

To decide what we need to test we need to understand what is likely to break. The current batch of web browsers have a set of commonly known bugs and differences. If you understand these differences, you can go a long way to understanding why pages render differently in different browsers. Internet explorer has a large number of CSD layout issues and rendering bugs. A full list is available here.

You can see visually how these bugs manifest themselves with the Acid 2 test. This is a test that is designed to see if a browser can render CSS correctly.

With its market dominant position, most sited will be built for IE first, and as a result, browsers that don’t have these issues, will now have layout issues. This is clearly shown below in this comparison of different browsers running the browser Acid 2 test.

If your sites are developed on Macs, you can almost guarantee there will be issues in internet explorer. If your site is developed on Windows, the opposite applies and Safari, Chrome, Firefox and IE8 will probably be broken.

One more thing

Your site will never look the same in EVERY browser. It is simply not possible. Here is how microsoft.com looks in lynx.

Ironically, this is exactly how search engines see the world.

The latest nightly builds of webkit, includes the new improved web inspector. It includes great new features such as:

- A Javascript debugger
- A Javascript profiler
- A resources panel so you can find where your page takes time to load
- Support to look into HTML 5 databases

For all the details check out the post here, then download your latest sword and get testing.

If you want people to test your beta software you need to make it as easy as possible to obtain and test. WebKit makes testing their nightly builds an absolute snap.

All you need to do is to download the build and then run the exe. That’s it. Webkit exists as a dynamic library so the application opens up your existing safari and loads the new browser engine inside.

This allows your existing “released” Safari to sit side by side with the nightly build. To add icing to the cake, the icon is a gold colour instead of silver, so you can easily tell which one you are running.

Back in 2005 I wrote a series of seven blog posts called The Seven Tenets of Software Testing. These posts have been buried deep in this site, so I have added a new page – Tenets of software testing – that links to all these origional articles, hopefully making them easier to find if you are new to my software testing blog.

Need to test Windows Communication Foundation ? There is an article in the July issue of MSDN magazine on how to do it. The article is here http://msdn.microsoft.com/en-us/magazine/cc700338.aspx.

Anne-Marie Charret has posted about her desire to keep test documents when doing Exploratory testing.

On my current project we are delivering in excess of three thousand requirements, and with each requirement spawning up to six test cases, we were simply drowning in a sea of tests.

To try and survive this, and complete the project with my sanity intact, something had to change. There were a lot of changes, however the main change that I instigated was to introduce the option of performing exploratory testing when the requirement was delivered. Documenting the tests as we performed them, instead of a huge up-front planning process. This ensures that we still maintain a record of our test case to requirement traceability

Well did it work? I am happy to say, yes. Instead of spending most of our time in front of Microsoft Team Foundation Server, we spent the time in front of the application finding bugs.

Need to do browser compatibility testing for IE 5.5, 6, 8 and 8 ?

IeTester should do the trick. It’s a stand alone application that contains all 4 rendering engines.

I find myself referring more an more to the wikipedia software testing portal. It is a great source of testing terminology and a good reference. If you haven’t seen it before I suggest that you have a look around.