Wednesday, October 2, 2013

Importance of Benchmarking Custom HTML5 Browser Platforms

In my current job we are currently porting over our propitiatory TV applications built in O-code over to HTML5. This is been done as our company is gradually making the move from a embedded, broadcast based TV App environment to more of a IPTV environment built using Open web technologies.

As part of this paradigm shift we are working with some low level hardware concept Set Up Boxes that have a custom QTWekbkit based browser. The first thing I did was to run some benchmark tests to get an idea on the HTML5 compatibility of this customised browser.

HTML5 Benchmarking tools and libraries are plentiful on the web, for e.g here are few good ones:

www.html5test.com
- Fast being the most used resource to find out how HTML5 compatible your browser is (Probably because it uses a simple scoring system)

www.browserscope.org
- A very useful benchmarking portal for various browser's capabilities

http://www.browserscope.org/alltests
- Some very good standalone tests to run against your browser

There are also numerous articles and blog posts on the web (lots by Google, YUI etc) on strategies to benchmark and understand your user's browser environment. We are also seeing the growth of RUM (Real User Monitoring) which gives you statistics on the performance of your websites and apps through the eyes of the user.

So why is this important? Why do we need to benchmark browsers? The truth is that this is largely over looked in the open web, as most power app users these days use the latest version of Chrome or Firefox and the fact that we are seeing some major improvements in the IE world, we can be safe to assume that our user's browser platforms have enough 'grunt' to serve up our app nice and fast.

BUT like our company, if you are working with a custom browser then Benchmarking is probably the most important aspect before you start building apps for that custom platform.

This is because:
1) You need to know exactly what you have to work with. So for HTML5 you need to know how compatible you are with the current standards. For e.g. maybe your custom browser platform does not support the HTML5 <video> tag, which would be a big loss if you need to build a TV app :)

2)  You need to be able to maintain a history of your Benchmarks as your custom browser platform will evolve over time, for e.g. the company who supplies you with the custom browser platform will keep giving you new builds as they continue to work on their products, and you need to be able to compare the new benchmarks against your history to make sure you don't have any regression (things being left out or missed).

For point 2, we use html5test.com as it gives you a nice and easy score which we log down and maintain to compare new browser builds.

I should give a special shout out to html5test.com as it's been built really well and the best part about it is that it's open source so you can actually fork the code and run it locally (which is what we do).

So whether you are building for the web or for custom browsers, knowing the platform you have to work with is very important because ultimately the app you build is not meant to impress you or your dev team, but the end users.

So go Benchmark!




No comments:

Post a Comment

Fork me on GitHub