Browser compatibility testing

BrowserCompat

The goal of browser compatibility testing is to ensure that the site renders without error on the target web browsers. Some minor rendering differences are expected from browser to browser, or within different versions of the same browser.

The image below shows the rendering differences between Internet explorer 5.5 (which is ghosted) and Internet Explorer 6 for Microsoft’s homepage, in December 2008.

MicrosoftDotCom.jpg

What you can clearly see is that the main page elements in IE5.5 are clearly moved to the left. You can also see that the navigation bar is not working at all, and that silverlight is not supported on IE 5.5.

These types of rendering differences will occur typically across major versions of browsers, but as the same rendering engine is typically used on all platforms for multi-platform browsers, the same version of the same browser will typically render the same result on all platforms that it supports. This reduces the need for testing the same version of a browser on multiple platforms, however simple sanity checking of browsers cross platform is recommended if the configuration is already available.

So which browsers should I test?

To determine where you should focus your testing effort, let’s look at the browser market share.

March 2013 Web Share

Source: http://en.wikipedia.org/wiki/Usage_share_of_web_browsers

Generally a browser should be tested if it meets one of the following criteria:

  • It is used by a significant portion of a sites users, and is in common, widespread usage.
  • It is the default browser on the latest version of Windows or Mac OS X.
  • It is a newly released browser which is expected to quickly gain a significant portion of browser market share, e.g. the latest IE, Firefox or Google Chrome

After reviewing the data against those criteria, the following browsers are suggested for inclusion for testing most sites:

Browser

Reason for inclusion in test matrix

Google Chrome (Latest) Most common browser in use today.
Internet Explorer 9.x IE 9 is the new IE 6.
Firefox (Latest) Latest version of Firefox
Safari (Latest) Default browser on Mac OS X.
iPhone & iPad (Latest) Uses webkit, the largest single mobile browser.

What bugs am I likely to find?

To decide what we need to test we need to understand what is likely to break. The current batch of web browsers have a set of commonly known bugs and differences. If you understand these differences, you can go a long way to understanding why pages render differently in different browsers. Internet explorer has a large number of CSD layout issues and rendering bugs. A full list is available here.

You can see visually how these bugs manifest themselves with the Acid 2 test. This is a test that is designed to see if a browser can render CSS correctly.

With its market dominant position, most sites will be built for Firefox first, and as a result, if not tested, other browsers will now have layout issues. This is clearly shown below in this comparison of different browsers running the browser Acid 2 test.

BrowserAcidTest.png

If your sites are developed on Macs, you can almost guarantee there will be issues in internet explorer. If your site is developed on Windows, the opposite applies and Safari, Chrome, Firefox and IE6 will probably be broken.

One more thing

Your site will never look the same in EVERY browser. It is simply not possible. Here is how microsoft.com looks in lynx.

MicrosoftInLynx.png

Ironically, this is exactly how search engines see the world.

Advanced Testing Techniques1 comment