f13.net

Asheron’s Call 2 launched with over 3000 severe yet unrecognized bugs.
...
Many of the crafting recipes were about ten times harder to create than intended. The creatures that dropped the needed parts spawned incredibly rarely, due to an oversight.
Many of the quests could easily become broken if the steps of the quest were done in the wrong order. Fixing them then required the assistance of a customer service representative.
Almost every skill in the game was broken in some way. Some skills literally did nothing; others were too costly or too powerful; some started out super strong and then got weaker as you leveled up the skill.
...
It was the extremely talented Jesse Kurlancheek (a.k.a. “Devilmouse”) who was responsible for the skills in AC2. Because of the agreement Turbine had made with their publisher Microsoft, he was obligated to create at least 600 skills for the game before it shipped, broken into 30 different skill trees. Problem was, he only had three months to design and implement them all. Jesse famously told his boss, “I can either implement the skills, or I can document the skills, but there’s no time to do both.” And he was right. But they chose to implement instead of documenting, and the result was tragedy.

Without any idea of how skills were supposed to work, QA never looked at any of them. To be fair, QA should have at least poked around with them some, but they just weren’t motivated to spend any time on them because the designer would almost always say “no, that’s not a bug, that’s how it’s supposed to work!” So the QA team spent their time wandering around looking for objects that were hovering a tiny bit off the ground, instead. At least those were irrefutable bugs!

The lack of specs was devastating. When the game launched, the live team needed to get the game into a maintainable state. Without specs, we had no idea what Jesse had intended. Worse yet, a few months later, not even Jesse remembered what the intention of each skill was. So we created elaborate analysis software in order to locate skill deviations, and slowly reverse-engineered the intentions, like archaeologists exploring an ancient culture. The lack of specs cost us thousands of man-hours. It made us look like total dicks, too, when we fixed the outrageous bugs we found.

The “reap” abilities in AC2 stole the health of an enemy and gave it to you. However, as is typical of this sort of power, you couldn’t steal more life than you were missing. So if you were only missing 100 health, you could never steal more than 100 from the enemy. Tragically, there was an accidental minus-sign in the implementation. If you reaped somebody and you were fully healthy, instead of doing no damage, you did double damage. Reaps were the most effective attacks in the game, provided you hadn’t been hurt too much. It made PvP in particular a nightmare, and players were howling for us to fix it. When we fixed it, though, several classes became unplayable; it turns out they had only been fun at all because of that broken skill. So then we had to buff those in various ways, until the next major bug was discovered, which threw our rebalancing out of whack again. And on and on, over and over, a constant dance of chaos and confusion, for over a year, before we had created our own specs and could work towards stability.

If only the skills had been affected this way, we could have dealt. But the lack of specs permeated almost every aspect of the game. The quest areas were about half-documented; the monster spec was nothing but one extremely complex spreadsheet. It’s not like the AC2 designers didn’t know how to make specs, of course: they were given impossibly small time windows to do their work, and they did their best.

If our publisher had allowed us, it would have been so much better for AC2 to have launched with only half the classes, but with docs for all of the classes. Then the live team could just implement the docs, rolling out new things every couple months, and make Turbine look super productive instead of super incompetent.

Of course, it doesn’t just happen that you hop onto the Live Team and suddenly you’re making game-design changes. At first there are a lot of smart and talented people at the helm, helping you learn the ropes, making the hard decisions for you, keeping you from doing stupid things. But inevitably they are pulled off to other projects, and somebody relatively junior gets the helm. That’s how I got to be in charge of balancing AC2’s classes.

Fortunately, I had a decade of engineering experience and understood how to tune complex systems. I wrote analyzers, modeled usage patterns, and made corrections.

Unfortunately, my approach did not take the “human equation” into consideration very well.

Learning to Balance the Human Equation

I found that the Feral Intendant class was 30% overpowered, and that’s why so many people were playing a Feral Intendant. Yet somehow, reducing the power of the Feral Intendant to the correct level did not suddenly make the game more fun… thousands of players were complaining and nobody was telling me they were happy about the change. Weird! I double checked my calculations. They were correct. So what had gone wrong?

Turns out that the people who played the other classes available to that race had taken on an “underdog” mentality. The people who played Claw Bearers liked that they were woefully underpowered compared to Feral Intendants. It was like playing the game on Hard Mode. And the people playing Feral Intendants liked playing on Easy Mode. In balancing the game I had failed to understand the needs of the people playing it. I just ham-handedly fixed the equations, instead of solving the problem with the finesse it needed. It was one of my more serious missteps. (And it’s a great example because I think it’s pretty obvious in hindsight. Most mistakes were much more subtle.)

But man, what a fast way to learn! After just a couple years of that, I became a good game balancer. The constant feedback loop helped me learn from my mistakes in a matter of weeks! Compare that to developers on traditional games, who must wait until the sequel ships before they get to try their hand at balance again. That’s why working on a live team is such a fast way to learn your craft: the feedback is so much faster than any other gaming platform, that it accelerates learning by dozens of times.

But AC2 cost millions of dollars to create. Turbine didn’t create it as a tool to help me hone my design skills, that’s for damned sure! How did I get to do it? Simple: the designers who would have done it were burned out of working on AC2, and were called away to work on the important New Project. AC2 wasn’t a blockbuster hit, so it didn’t make sense to use the rock star designers on it. Better to let the B team step in.

On Asheron’s Call 2, we determined that about 10% of the playing audience read our website or forums (it spiked on patch days, to a whopping 15%).

This small percentage of people are not randomly pulled from your userbase. They tend to be very similar to each other and not very representative of the rest of your player base. That means you can’t use them to judge the quality or merit of ideas or implementations. But of course, the ones who do post are your most vocal users, and it would be foolish to ignore them.
...
The pretend quitting: “This is the last straw. The new updates on the test server are a mockery of everything this game was supposed to be about. If this change goes live, I will be forced to cancel all three of my accounts forever. Since I am guild treasurer, I’m sure most of my guild will quit too, and we’re the only decent PvP guild on our server, so PvP on our server will entirely die out. I wish I didn’t have to do this, but they’ve forced my hand.”

This is a classic. On AC2, I did my best to correlate people who said they were quitting to people who actually quit. Almost nobody who said they were quitting actually quit, and the few who did didn’t stay gone long: they entered a rebound cycle and came back pretty quickly. Most often, they didn’t leave at all.
...
Being exposed to the forums causes real damage to developers and moderators. For instance, the QA employee in charge of WoW’s test-server forums is so badly burned out that he routinely deletes threads full of information and bug reports because users just sicken him.
...
Players tend to whine and moan most loudly on the game’s official forums, because they are putting on a show to try to convince developers to change things. Behind closed doors in guild forums or fansite forums, posters tend to be considerably more upbeat. It’s very surprising to see the change of tone. It helps put things in perspective.

Don’t force your developers to read the forums. Instead, have the community send a weekly digest of posts to the team. Make sure the digest has as many upbeat or informational posts as it has complaints (even if the actual ratio on the forums is much different). Remember that forums are not representative of the user base so there’s no reason to expose developers to all the hate and anguish there. A taste is enough to get the idea.
...
Never post while drunk
...
The early AC/AC2 forum moderators at Turbine had a drinking game they’d play sometimes when reading forums late in the evening: drink a shot for anybody who says “slap in the face”, chug if someone says their guild is quitting, and if somebody predicts the game will be dead in a month, the whole room has to drink. I’m sure versions of this game exist at many companies.

While i dont think any of what he said is neccessarily surprising, what continues to amaze me is that this pattern seems repeated in one form or another for almost every failed/troubled mmorpg LARGE SOFTWARE PROJECT WHERE HUMANS AND TEAMS ARE INVOLVED made

The games industry has lots of prima donna superstar developers but very few decent project managers ...

But here’s the weird thing: WoW is exhibiting the same symptoms as AC2 did when I was doing the designing. The B team is in charge.

Interesting posts.  I've never used official forums in any of my time in mmogs, but I was surprised the number was that low.  The concept of a community digest being produced sounds more interesting.  But I think companies like SOE and Funcom have already done things like that by appointing advocates from the community.

That whole experience convinced me that developers and QA need to work closer together and that while it does slow things down at first, the long term benefits are astounding once you get rolling.

At first there are a lot of smart and talented people at the helm, helping you learn the ropes, making the hard decisions for you, keeping you from doing stupid things. But inevitably they are pulled off to other projects, and somebody relatively junior gets the helm.

That whole experience convinced me that developers and QA need to work closer together and that while it does slow things down at first, the long term benefits are astounding once you get rolling.

The game obviously had some huge issues, but none I think were worse than the Microsoft no-chat fiasco.

Of course, if the game wasn't so broken, I likely wouldn't have played it as long as I did.  The grind in that game would have been absolutely nauseating if you couldn't exploit pathing and quest issues.

I really want to read that kind of article for SWG.  Cal?

But reading the postmortem, it may be that the live management team knew from the very first moment that they were dealing with a dead duck that couldn't be saved anyway, I dunno.

I really want to read that kind of article for SWG.  Cal?

I always wondered how "lets not have npc's" managed to fly at an early design meeting for AC2.

Edit I should maybe add I hadn't done more than glance at Elder Game before.  But having read quite a bit now, maybe my selective quoting was unfair.  I don't agree with him on forums and still think he's overly obsessed with balance, but there's a lot of interesting stuff there.

That said, overall I do agree that forums are at their best when they are used as a means for players to communicate with each other rather than a way to scream for Dev Team attention.

I wonder what DAOC's official forums would be like if it had them.

I always wondered how "lets not have npc's" managed to fly at an early design meeting for AC2.

Almost every skill in the game was broken in some way. Some skills literally did nothing; others were too costly or too powerful; some started out super strong and then got weaker as you leveled up the skill.

AHA! I knew the bounty hunters skill "lucky charm" did nothing. For literally years there was rumor and speculation that ranged from "improves hit or crit" to "increases drop rates" to "does nothing" and "is a developer joke".

Now we know... i think  :uhrr:

That whole experience convinced me that developers and QA need to work closer together and that while it does slow things down at first, the long term benefits are astounding once you get rolling.

Interesting.  One thing that comes to mind reading this is that the DAOC model of official communication (The Herald) is better than the Blizzard blue-post style.  The Blizzard style leads players to believe that their drama queening on certain official forums will get their favorite change made, where the DAOC model - blog-style - gives no such illusion.

I wonder what DAOC's official forums would be like if it had them.

AC2 is a weird memory for me. I remember my experience extremely fondly, but for many odd reasons.  It was the first and perhaps only game that I ever rushed to the level cap.  It is also the only game where I actively and regularly exploited game mechanics. 

I've been at companies where QA is in a little room way in the back, and others where QA is sitting in the same cubicle pit as the devs. Not suprisingly, the latter case tends to have higher productivity than the former. How much...I'd estimate at least times a hundred bazillion.

Another hump that desperately needs to get over is the weird idea that everything is some kind of secret, including tools used, vocabulary, user-to-developer relations and stuff that needs to be a little more standardized across the industry. Keep your design ideas secret, actual code, artwork, style, etc, but Gods, stop trying to reinvent the wheel because you're going to fail miserably. Post-mortem is always filled with "Look how this game did it, why didn't we try that?" It reminds me of two film companies that started in Alaska, one run by someone who used to work in Hollywood and another who made his own documentaries. The Hollywood guy is still making movies up there. The do-it-yourselfer never finished a single project on his own.

If your competitor has shitty QA, why give them any help?

The gaming industry in general, but especially Blizzard, are very quiet on the nuts 'n bolts of getting a game out the door. I often wonder what their project managers and designers are thinking over there when something like MxO folds.

The gaming industry in general, but especially Blizzard, are very quiet on the nuts 'n bolts of getting a game out the door. I often wonder what their project managers and designers are thinking over there when something like MxO folds.

I heard a Blizzard Test\QA Manager talk at GDC, at a roundtable, about testing.  It was standard software development testing stuff just not standard game development stuff.

They have career testers there rather than a building filled with +400 at minimum wage for one example of something they do that many game places do not.

Brilliant.  I wish other MMO's would learn that.

I actually enjoyed AC2 near the end.  It was extremely different and creative.

I wish they would bring it back.  :heartbreak:

If they were ever going to do an AC3, they'd probably have to choose a different title.

AC2 is remembered more for being a failure, than as a flawed game that had some really great ideas and fucktons of fun.

I wonder if Turbine would ever want to take the risk.

The catering to explorers in AC1 hasn't been seen since in a MMO, which is really frustrating to me. Perhaps the most fun I've ever had in an MMO was taking off by myself from Danby's Outpost as a medium-level character and trying to make my way to Teth on foot.

I posted some musings about "AC3" back in January. (http://stormwaltz.gamerdna.com/game_post/10j5YCT/an-asheron-s-call-3-feature-list)

EDIT: Before it's mentioned, yes, when I wrote that, I blanked on the fact that UO was skill based...

I had no problem subscribing from Japan with AC2, but Horizon's wouldn't let me. Almost makes you think there could be a God somewhere  :awesome_for_real:

Lots of games get postmortems at Gamasutra and elsewhere, but not so much MMOs, about which people tend to be tight-lipped with a few notable exceptions.

The catering to explorers in AC1 hasn't been seen since in a MMO, which is really frustrating to me. Perhaps the most fun I've ever had in an MMO was taking off by myself from Danby's Outpost as a medium-level character and trying to make my way to Teth on foot.

This is the number one thing I miss in AC and wish would return in an MMO. Heck, if they just reskinned AC so the graphics weren't so fugly and fixed a few problem areas (back when I played, if you didn't have Item or Life magic you were a gimp) I'd resub in a heartbeat.

I still remember I played the game mostly with 2 friends of mine. We'd log on, chat a bit, then pick a random direction and go exploring. Unfortunately exploration is pretty dead in most modern MMOs.

While reading what Citan wrote, I found it completely ironic that he seemed to gloss over the fact that AC2 hype completely lied to the customers on what they were going to deliver.  I remember very clearly becoming excited when we were told that we could rebuild the world.  Literally players were supposed to be able to mine/harvest resources and rebuild Derith.

I found it completely ironic that he seemed to gloss over the fact that AC2 hype completely lied to the customers on what they were going to deliver.  I remember very clearly becoming excited when we were told that we could rebuild the world.  Literally players were supposed to be able to mine/harvest resources and rebuild Derith.   That was what initially brought me in.  Sadly I don't think they ever had an intention to deliver on that promise.