Posted on June 23, 2008, Chris VG Chartz: A Bunch of Lucky Guessers?
I’ve previously expressed my disdain for numbers cited as coming from VG Chartz, the videogame industry sales-tracking website, calling it “unreliable.” Given the amount of attention it’s been getting — the site has been sourced by publications like Forbes and The New York Times — Gamasutra’s Simon Carless dug into the website a bit to find out exactly how it obtains the numbers it shares. Here are a few choice excerpts from the piece, which you can read in its entirety right here.
…if they are sufficiently out, then VGChartz will retrofit their results – either weekly or monthly – to conform to the more ‘official’ data. But they won’t credit those firms as the source of the retrofitting – they’ll just bump their numbers around without saying why on the site.
…Let’s be clear. I think the concept behind VGChartz is a wonderful one – freely available data to let everyone see how well games are selling. And it’s absolutely true that all data is an estimate – not even major services such as Media Create and NPD get it exactly right.
But VGChartz is staffed by amateurs working in their spare time to estimate sales, and while they are perfectly smart, they are much closer to the SimExchange model of estimation than the Media Create method.
… [Site creator Brett] Walton freely admits the [MGS4] numbers were based on zero actual data for the entire European market, just pure extrapolation. It’s also very unclear how far the estimates for launch were based on real retail data for Japan and North America.
It’s a reasonable figure, of course, because the VGChartz folks are smart people. But it’s not a real figure. It’s a educated guesstimate, and it’s much more of an estimate than the subsequent Chart Track data for the UK, for example.
I don’t have some agenda against the site, as I’m sure someone will accuse me of, but I’ve always taken issue with numbers from the site being taken as absolute canon, or as a complete equivalent to groups such as the NPD. Walton claims that early sales numbers will be labeled accordingly, in response to Carless taking issue with the way the site marketed its MGS4 numbers. But is that really enough?