Android and iPhone OS version breakdown

Here are two useful links on the breakdown of actual mobile users for each of Android and iPhone:
  http://developer.android.com/about/dashboards/index.html
  https://developer.apple.com/devcenter/ios/checklist/

The reason I love these stats is that: a) they are meaningful, and b) they are meaningful. They are first meaningful because they record active users (people who access the Google PlayStore, and the Apple AppStore, respectively), as opposed to based on sales of handsets, or stats of some website that is focused on a particular group (e.g. Apple developers!!) And they are meaningful because almost all active users will have visited these particular sites at least once a month, the survey times are 14 days, so we're talking a boat-load of people. Big numbers means reliable stats (all else being equal - see previous comment about the likely lack of bias).

The big disappointment is both Google and Apple choose to show percentages, not actual numbers. And no breakdown by country of the user. (If you know how to get those stats, please let me know!!!)

I saw a thread on Slashdot saying that these stats show that Android handset market is much more fragmented than Apple. It does in fact show that 94% of people who visit the AppStore have upgraded to iOS6. There is some bias there, as I think they might be strongly encouraged to when they visit the AppStore? (Just checked, and it constantly reminds you to update, even if you don't visit AppStore). But, anyway, if the above theory that practically all active users visit the AppStore at least once a month is true, then it is useful information for a developer (i.e. testing on iOS6 should be sufficient).

Android users, on the other hand, get upgraded by their network provider? We still need to test on each of Android 2.3, 4.0 and 4.1.

But even more important for a developer, if you want to do any animation or special effects, is knowing the hardware power of the devices. The Android 2.3 stats suggest that percentage of users are still on relatively old hardware, single core. And that the other 2/3rds of users probably have a bit more firepower. For Apple users I've no idea of the breakdown. (Again, information donations are very welcome!)

Some updating line charts in d3

There are so many lovely d3 demos... but I think this might be more due to the creativity and stubbornness of the people who have chosen to use d3 than because of some innate quality of the library.
(d3 is a charting library, that claims it is not a charting library; instead it likes to see itself as an alternative to JQuery, I think.)

As a case in point, there is this page showing three beautiful examples:
   http://bost.ocks.org/mike/path/

In particular scroll down to the bottom and it is tracking how much scrolling you've done on the page, with the time on the x-axis. But the script for this cleverness is about 110 lines. And it is quite opaque. To put that comment in context I've read the O'Reilly d3 book cover-to-cover, then played with every d3 example in the book.

By the way, another example of scrolling charts, this one with multiple charts overlaid: http://lyngbaek.com/real-time-stream-graph.html

Here is a similar chart done in Rickshaw (which is another library built on top of d3): http://code.shutterstock.com/rickshaw/examples/fixed.html

And another example done in Rickshaw, that is quite heavy-duty: http://code.shutterstock.com/rickshaw/examples/extensions.html

RUnit and callback functions and scope

I have a function (called something() in the below example code) that takes a callback function as a parameter. To be flexible it can take the callback as a Function object, or as a character string, or as a list (which keeps the function name and the parameters together).

But I hit a problem when trying to write a unit test for something(). It works with a global function as the callback (such as mean in the below code), but not for a locally defined function (thingy in the below code):

thingy=function(x){ mean(x) }

test.something=function(){
x=1:10

something(mean,x)
something("mean",x)
something( list("mean",x) )
something(thingy,x)
something("thingy",x)    #Fails
something( list("thingy",x) )   #Fails
}

The problem appears to be that functions in a test file are all loaded in a special environment. And, when I thought about it, that makes sense, unit test functions are supposed to be small and independent, and I don't want to be defining a function at the same level as the unit tests.

I tried a few things, but here is the one that worked:

test.something=function(){ 
assign("thingy",function(x){ mean(x) }, envir=globalenv())
 
x=1:10

something(mean,x)
something("mean",x)
something( list("mean",x) )
something(thingy,x)
something("thingy",x)
something( list("thingy",x) )

remove("thingy", envir=globalenv())
}

So, we explicitly put our thingy() in the global environment, where everyone can find it and make use of it. And we define it inside the test function, no earlier. Removing it at the end is optional, but Murphy's Law states that if we don't then it will end up clashing with something. Late on a Friday afternoon, in six months time, when we've forgotten about this code.

PhantomJS: POST, auth and timeouts

I recently discovered PhantomJS, which (for me) is an alternative to Selenium, with two key differences that (again, for me) make it very useful:

1. It is headless, meaning you don't see anything graphical. That means I can run it on a server, from the commandline, without needing X installed. It also means it causes less load.
2. It embeds webkit, rather than attempting to interface with many browsers, and control them as a user would.

The second point allows me to POST to a URL, which is great for testing how web services work in a real browser. Selenium refuses to offer this because it is not something a user can do with a browser. (The workaround in Selenium was to make a temporary page that uses an AJAX call to POST to the URL, then does something with what is returned.)

There are two things that PhantomJS makes difficult, which I will show techniques for here. The first is that authorization is kind-of-broken. The second is timeouts for requests that never finish (e.g. an http streaming web service). But, first, the basic example, without auth or timeouts, and using GET:

var page=require('webpage').create();
var callback=function(status){
    if (status=='success'){
        console.log(page.plainText);
        }else{
        console.log('Failed to load.');
        }
    phantom.exit();
    };
var url="http://example.com/something?name=value";
page.open(url,callback);

Now here is the same code with basic auth (shown in orange), and a five second time-out (shown in red):

var page=require('webpage').create();
page.customHeaders={'Authorization': 'Basic '+btoa('username:password')};
var callback=function(status){
    if(timer)window.clearTimeout(timer);
    if (status=='success' || status=='timedout') {
        console.log(page.plainText);
        }else{
        console.log('Failed to load.');
        }
    phantom.exit();
    };
var timer=window.setTimeout(callback,5000,'timedout');
var url="http://example.com/something?name=value";
page.open(url,callback);

Don't use page.settings.userName = 'username';page.settings.password = 'password'; because it has a bug as of PhantomJS 1.9.0 (it uses two connections for GET requests and doesn't work at all for POST requests). Instead make your own basic auth header as shown here (thanks to Igor Semenko, on the PhantomJS mailing list for this trick).

For the time-out code I still call the same callback, but pass a status of "timedout" instead of "success" (so the callback could react differently, if timedout was a bad thing - here I treat them the same). So, if the URL finishes loading within 5000ms, then callback is called (by the page.open() call) with status equal to "success". If it has not finished within 5000ms then callback is called (by the javascript timer), with status equal to "timedout".

I explicitly clear the timer immediately when entering callback(). This is not really necessary, as we're about to shutdown (the phantom.exit() call) anyway. But it feels safer because otherwise callback() might be called twice (i.e. if the page loaded in exactly 5000ms); the more computation being done in callback(), especially if asynchronous, the more this might occur. (Well to be precise: that catches the case when page loads in just under 5000ms and triggers the callback before the timer does. But, if the timer gets in first, and then the page loads in just over 5000ms, and callback computation takes a while, then we may still get two calls. I think calling page.close() in callback() might prevent this, but that is untested.)

Finally, here is the same code using POST instead of GET:

var page=require('webpage').create();
page.customHeaders={'Authorization': 'Basic '+btoa('username:password')};
var callback=function(status){
    if(timer)window.clearTimeout(timer);
    if (status=='success' || status=='timedout') {
        console.log(page.plainText);
        }else{
        console.log('Failed to load.');
        }
    phantom.exit();
    };
var timer=window.setTimeout(callback,5000,'timedout');
var url="http://example.com/something";
var data="name=value";
page.open(url,'post',data,callback);

The differences are shown in red. It couldn't be easier!