I was curious to see how the uptake of Google's Chrome browser would be, considering that they were promoting it on their front page. I looked it up on a special page set up by Net Applications to track Chrome usage before I left for work. It seemed to do pretty well, as it climbed above 1%, passing Opera's alleged market share. Not really surprising considering the massive media coverage it was getting.
When I got back later and reloaded the page, I noticed that it had gone down to 0.5% or so the last few hours. I still left the page open, and returned a little later. To my surprise, the page was no longer showing the same numbers for the same time. It's as if it had never shown 0.5%.
I tried to get my hands on a cached copy of the page to make sure that it wasn't just a mistake on my part, and indeed, it was not. Apparently Net Applications decided to change the numbers after they had been published. …
Edit: It looks like the original image has been removed from the server, so I uploaded both of them here instead:
The numbers are 0.56% and 1.18% respectively for 9/4/2008 11:00:00 AM (EDT).
See the full images in my Net Applications/Chrome stats photo album.
This is not the first or even second time the numbers published by Net Applications have "mysteriously" changed from one day to the next. A few years ago, Opera showed up with up to 5% in their stats. Apparently they figured that this was too high, so the numbers were slashed. Then, last year, Opera for Desktop was climbing above 1%, and Opera Mini was up to 0,6% or so, and climbing fast. Overnight, the figures changed completely, and Opera Mini was down to 0.1% or so, while desktop was at about 0.5%.
Now, what is the reason for this apparent tampering with the stats? I really don't know. It could have a perfectly valid explanation (even though I can't think of one at the top of my head), but Net Applications seems to stay quite tight-lipped about how their stats are measured. For example, they claim that theirs are "global stats", but aren't their customers mainly located in the United States?
In any case, this highlights just how unreliable browser statistics are. While one browser might re-fetch resources from the server fairly often, another browser might use the cache more aggressively, and thus cause fewer "hits", and show up lower in the statistics. This depends on how the stats are measured, of course, but many stats companies refuse to share their actual methodology.
And when their numbers are seemingly manipulated without any explanation what so ever, this raises serious questions about how reliable they are, and whether they should be quoted at all when discussing browser usage.