Where Have You Gone, Joe AOC?

One of the most annoying things about working in MMOs is listening to armchair quarterbacks talk about subscriptions, play time, and what have you without a speck of data. “The sky is falling!” is one of the most common refrains of a player whose particular little red wagon isn’t being fixed fast enough. Do I sound jaded and bitter? Friend, I’ve been around so long that I remember when “everyone” was quitting Ultima Online after the carebear patch. Yet it’s still there, and doing better with more loyal subscribers than a number of other AAA titles I could mention but won’t.

So let’s take a look at some numbers for Age of Conan. Specifically, let’s use the data we’ve got to test two common hypotheses – one, that players who left WoW for AOC are going back to WoW, and two, that the patches are causing players to stop playing. (Please note that I said “stop playing,” not “quit.” I will not speculate on whether or not players are actually canceling subscriptions. I will simply note that in my time on the front lines during the first year of a new release, if I checked the billing records of twenty people claiming to have canceled, one of them would have actually done so… and half of those that did so resubscribed within the week.)

(46 days of data ranging from June 5 to July 21. The population is GamerDNA members who list AOC as one of their games. Blue is AOC, red is WoW, green is other.)

You can see that on June 5th of this year, just a hair under 80% of those GamerDNA players listing AOC on their Xfire feed logged in to play it. On July 21, that number was just under 40%. But here’s where they didn’t go in huge numbers: Back to WoW. On June 5th, the percentage of players with AOC on their list who logged into WoW was 9%. On July 21st, it was 12%.

Where did they go, if not back to WoW? Well, according to our data, all over the place. GamerDNA members listing AOC instead went and played the following titles, in order of popularity.

There were other titles, of course, but I’ve chosen just to show the top ten games of the group not playing either WoW or AOC during the date range of the first chart.

It seems clear that if players left WoW for AOC, they aren’t going back even if AOC isn’t what they had hoped it would be. Instead, they’re playing other games – particularly titles that guarantee a certain amount of immediate fun and action. So, our first hypothesis is WRONG!

Now, let’s look at how that top chart works with patch dates. According to what felt like an entire afternoon of googling – why does the patch notes section of the community site only list the most recent patch? I hate that – I have the following dates for patches in the 46 day period:

June 12, 16, 23, and 26

July 10, 17, 23

After staring at the chart until I’m crosseyed, I have a few tentative conclusions. Now, bear in mind that I have not looked at the notes, or the board traffic surrounding the individual patches, and am drawing these conclusions entirely by the numbers. The patch notes on June 16th and June 23rd made people happy (at least temporarily, see the last conclusion). Instead of the dip in logins that I would predict for a patch day (if the servers are down, people can’t log in, and drops on patch days are to be expected), the numbers rose.

The patches of June 12, and July 17, inspired temporary protests, as the following days experienced drops in logins of approximately 10%. The issues were apparently more of principle than actual playing problems, as the numbers rebounded.

The patches of June 26 and July 10 were the least popular of the observed period, causing sharp drops in logins on the following day. In both cases, protests were largely symbolic, as logins rebounded back to around 40% of users claiming AOC as a title.

However, something went badly wrong with the June 16th patch, and the initial consumer satisfaction appears to have been entirely based on what the notes contained, and not the reality of what went live. The numbers rose as people rushed to try it, but the weekend immediately following this patch saw one of the steepest drops of the observed time period. In fact, the logins never truly recovered from that patch, or that particular set of changes to the live product. The patch notes on the 23rd convinced players to log back in and give the product another try, but this time the rebound was of very short duration, completely crushed by the patch that went live on the 26th.

Okay, guys, that’s the data. Now, what this skeleton needs is some meat on its bones. What happened on June 16th, in your opinion? Why did the 23rd give you such hope? Why does it seem like nothing since June 16th has really mattered, and numbers have remained more or less constant? Data is never the whole story – it just tells you where to look.