Iron Man & Retrofitting
One of the most unexpected results in the recent NPD charts for May was the appearance of the poorly reviewed
Iron Man game for PlayStation 2 in the Top 10 of the charts, with 130,000 copies sold.
Thinking about it carefully, with the movie rocketing to unexpected success during the month, it would make sense that the game would sell well. But it's not the kind of game that you're likely to estimate in the Top 10 - and indeed VGChartz did not,
estimating 53,000 units in sales, according to VGChartz staffers.
But what's surprising is that
Iron Man for PlayStation 2 has been adjusted in
its official VGChartz page so that its first four weeks of sales (encompassing May) add up to 111,000 units.
Clearly, these numbers have been changed after NPD debuted, showing a couple of things. Firstly, if you were a journalist, you could have cited VGChartz as saying
Iron Man was a flop on PS2, selling half as many units - when NPD vibrantly disagrees. In addition, and more interestingly, it shows that VGChartz trusts NPD over their own prediction data by retroactively changing things to better match
Apparently, this has happened before, because in a
FAQ about North American VGChartz numbers, Brett Walton addresses this precise subject:
"Do we adjust our data? Not as such. Do we adjust our methods then? Yes - which will of course alter some data. On what basis? If we believe that a particular data set differs significantly from other sources of data (data released into the public domain by tracking firms, manufacturers, analysts) then we do re-check our data and make adjustments to the methods / scaling factors used.
This happens on a fairly infrequent basis - less often than we adjust due to internal data changes - and is something that every tracking firm and analyst does. I personally have no issues with "benchmarking" our data from time to time against other sources of data - as long as it has been made public."
In other words, if they are sufficiently out, then VGChartz will retrofit their results - either weekly or monthly - to conform to the more 'official' data. But they won't credit those firms as the source of the retrofitting - they'll just bump their numbers around without saying why on the site.
As a result, we get to what VGChartz actually is - a strange mixture of a prediction market (as consensus prediction site TheSimExchange is) and a retroactive, but non-credited reflection of charts that have historically been known for having more concrete data.
