Advanced Testing Techniques
The goal of browser compatibility testing is to ensure that the site renders without error on the target web browsers. Some minor rendering differences are expected from browser to browser, or within different versions of the same browser.
The image below shows the rendering differences between Internet explorer 5.5 (which is ghosted) and Internet Explorer 6 for Microsoft’s homepage, in December 2008.
What you can clearly see is that the main page elements in IE5.5 are clearly moved to the left. You can also see that the navigation bar is not working at all, and that silverlight is not supported on IE 5.5.
These types of rendering differences will occur typically across major versions of browsers, but as the same rendering engine is typically used on all platforms for multi-platform browsers, the same version of the same browser will typically render the same result on all platforms that it supports. This reduces the need for testing the same version of a browser on multiple platforms, however simple sanity checking of browsers cross platform is recommended if the configuration is already available.
So which browsers should I test?
To determine where you should focus your testing effort, let’s look at the browser market share.
Generally a browser should be tested if it meets one of the following criteria:
- It is used by a significant portion of a sites users, and is in common, widespread usage.
- It is the default browser on the latest version of Windows or Mac OS X.
- It is a newly released browser which is expected to quickly gain a significant portion of browser market share, e.g. the latest IE, Firefox or Google Chrome
After reviewing the data against those criteria, the following browsers are suggested for inclusion for testing most sites:
Reason for inclusion in test matrix
|Google Chrome (Latest)||Most common browser in use today.|
|Internet Explorer 9.x||IE 9 is the new IE 6.|
|Firefox (Latest)||Latest version of Firefox|
|Safari (Latest)||Default browser on Mac OS X.|
|iPhone & iPad (Latest)||Uses webkit, the largest single mobile browser.|
What bugs am I likely to find?
To decide what we need to test we need to understand what is likely to break. The current batch of web browsers have a set of commonly known bugs and differences. If you understand these differences, you can go a long way to understanding why pages render differently in different browsers. Internet explorer has a large number of CSD layout issues and rendering bugs. A full list is available here.
You can see visually how these bugs manifest themselves with the Acid 2 test. This is a test that is designed to see if a browser can render CSS correctly.
With its market dominant position, most sites will be built for Firefox first, and as a result, if not tested, other browsers will now have layout issues. This is clearly shown below in this comparison of different browsers running the browser Acid 2 test.
If your sites are developed on Macs, you can almost guarantee there will be issues in internet explorer. If your site is developed on Windows, the opposite applies and Safari, Chrome, Firefox and IE6 will probably be broken.
One more thing
Your site will never look the same in EVERY browser. It is simply not possible. Here is how microsoft.com looks in lynx.
Ironically, this is exactly how search engines see the world.
The latest nightly builds of webkit, includes the new improved web inspector. It includes great new features such as:
- A resources panel so you can find where your page takes time to load
- Support to look into HTML 5 databases
For all the details check out the post here, then download your latest sword and get testing.
If you want people to test your beta software you need to make it as easy as possible to obtain and test. WebKit makes testing their nightly builds an absolute snap.
All you need to do is to download the build and then run the exe. That’s it. Webkit exists as a dynamic library so the application opens up your existing safari and loads the new browser engine inside.
This allows your existing “released” Safari to sit side by side with the nightly build. To add icing to the cake, the icon is a gold colour instead of silver, so you can easily tell which one you are running.