MMO budgets and development times

[Venkman]

SL feature feedback is an example of conch-shell representation. But that's not the fault of the bug submission system itself. It was how it was configured.

CMs work all the time, but the numbers never add up. Even 1% of WoW's playerbase spamming the boards is a lot more noise than a budget-justified CM team can handle. But they do their best, elevate what issues seem big, talk about it at the next roundtable/staff-meeting and then the teams that can do something about it a) consider whether they should; and, b) schedule it into their development cycle. CMs may appear to be doing nothing when in reality they are merely the conduit between players who want immediate feedback and developers who can't possibly deliver it.

I like the idea of Correspondent programs. General flame-fest clique forums reporting into competent volunteer lateral thinkers reporting into CMs reporting into devs. Sorry if every little pet idea doesn't get the personal atta-boy from the developer, but try getting anything approaching even what we have now from any other service industry without first approaching your local Attorney General or the Better Business Bureau Devs don't report to us. They simply provide a service to us.

We don't have it great, but I often find CM complaints to be a case of people getting more wanting more.

[waylander]

Yeah I think DAOC had Team Leads or something. Shadowbane had its Advocates. I think the advocate system worked well for funneling up issues, and every quarter I would ask them to poll their specific forums to see what the top 5 issues were.  Only problem we had was when the player base dwindled due to guilds quitting after their cities were destroyed, and it made it harder to find quality people.  But otherwise we had a budget for advocate leads, and free game time for class advocates that worked rather well for a long time.

I'm not really sure what the other games are doing.

[tmp]

As for actually acknowledging bugs, I'd like to see real bug tracking systems used by beta/test server users but I'm not sure it's possible. Second Life uses JIRA tracking on their stuff and it gets hijacked constantly by board warriors who glory in their ignorance of how software development works.

That's just Prok. She's.... special, to the point of getting SL-wide recognition of sorts.

[IainC]

If it's their job, then why does it often appear that they do nothing at all, or that issues that tend to have broad-ish forum community support run on without comment for a long time? I did say that a good CM can do that job, but it isn't optimal if Coder Joe doesn't pay attention to how the changes he is coding in are being received, or if reasons for the changes are never communicated back to players.

MMOs are service industries, and while I think Coder Joe should spend most of his time coding and CM Mike should spend his time community managing, it isn't a good thing if Coder Joe works in a vacuum without ever realising what impact his changes have on how the paying customers play (or that CM Mike has no idea what Coder Joe is working on). Better Joe reads 50 threads on Stormtroopers and then helps formulate a reasonable solution than cook up a fix without ever bothering to find out what players want, how they currently behave and maybe how well they'll receive that fix.

However, if it makes a stronger point, replace the NOT A CM bit with "someone who is able to start the ball rolling on a fix and to post feedback". Too often I've seen CMs as company mouthpieces who's only authority is to delete offensive forum posts - if it is their job to pass on important information back up the chain (which yes, should be part of their job) then they certainly never comment on it.

I'm going to field this one. I'll start with a disclaimer: I work as a CM but for a publisher not a developer so my experience will be different to someone who gets to have regularly scheduled meetings with the guys working at the coalface.

That said, during beta I'm very busy. I'm spinning a lot of plates simultaneously. I have about 50 or so forums I read regularly for the purposes of doing my job. That takes a long time even if you're just skimming thread titles for stuff that you think you should read. While I'm reading those forums I'm gathering feedback. The guys I have to pass that feedback onto don't have time to read 50 forums a week which is why they pay me to do it for them. Feedback by itself is meaningless without context - who are the people giving this feedback? What sort of background do they come from? What agendas might they represent? All of this has to be collated and summed up in a format that a bunch of people who are frantically busy can digest quickly while also answering all the questions they're likely to have about it.

Then there are the beta testers. Even though the beta forum is just one of the forums I read in my working day, it's probably the most important. These are the guys that are actually playing the game, the fans on the messageboards are speculating and responding to marketing drops, these guys have sharp-end experience but again it needs to be filtered and adjusted for bias. These guys also need to be nurtured carefully, like a plant you can kill it with over attention. I read the posts they make, but I try not to get involved in the discussions except to keep things on topic or to shut down trouble. The reason for this is because I want the testers to tell me what the game is like, not to react to my interpretation of what the game is like or what I think would make the game better. I have my own ideas and they go in a separate part of the forum. From time to time I'll have to remind the testers that I do indeed read every post that they make, I am passing on their feedback to the devs and no, they aren't ever going to see one of those feedback documents. That takes a great deal of time.

On top of all of that, I'm also doing the PR fun stuff to the outside world, I'm the guy during beta telling the fans how great the game is and how much fun we're all having in beta. On live release I'll be the guy doing fire control on drahmaz and explaining to people who aren't playing the game yet, precisely why they should. Note: At this point I'm still doing all the other stuff too.

Finally I have a lot of other hats I need to put on from time to time. One of our stipulations with regard to official communications is that it must be proofread by a native speaker of that language (we operate in five languages). As I also need to know the content of all our official communications, a handy way to kill two birds with one stone is to have me and my colleagues proofread the relevant release for each language. As a lot of the stuff I get is translated from a French original by non-native English translators, it often needs quite a bit of editing. I actually need to play the game I'm fronting for as well at some point and I need to stay current with the stuff that's coming from the developers.

If you made a sparkling observation of beautiful truth and I didn't acknowledge it specifically, I hereby formally apologise. It will I'm afraid, happen again.

[fuser]

That said, during beta I'm very busy. I'm spinning a lot of plates simultaneously. I have about 50 or so forums I read regularly for the purposes of doing my job. That takes a long time even if you're just skimming thread titles for stuff that you think you should read. While I'm reading those forums I'm gathering feedback. The guys I have to pass that feedback onto don't have time to read 50 forums a week which is why they pay me to do it for them. Feedback by itself is meaningless without context - who are the people giving this feedback? What sort of background do they come from? What agendas might they represent? All of this has to be collated and summed up in a format that a bunch of people who are frantically busy can digest quickly while also answering all the questions they're likely to have about it.

Then there are the beta testers. Even though the beta forum is just one of the forums I read in my working day, it's probably the most important. These are the guys that are actually playing the game, the fans on the messageboards are speculating and responding to marketing drops, these guys have sharp-end experience but again it needs to be filtered and adjusted for bias.

 I'm just wondering as a tool to a CM do developers data mine across their forums, feedback, bug database to give a large picture?

 All the beta's I have taken part in was one giant vacuum where /bugs were being reported, threads on beta forums for an issue or feature going no where with inherent bickering that really puts beta tester(s) in a negative feedback loop. I always wondered why its an issue connecting /bug or /feedback to an actual tracking system that has some transparency to a tester they can interact with. It's really terrible to have a ticket closed saying its known issue when you already took the time to search their beta boards but have no other way to check to see if your wasting time(the company and yours).

 Perhaps I'm just still jaded over Lum's GameCo example.

[DrewC]

No game company is going to open it's actual bug database to the public.  We cannot allow players to see all of that information, if only to deter exploitation of bugs.  Yes, ideally, we're going to fix all of those exploitable bugs, but here in the real world where developers like to see their families on occasion and overtime laws are, occasionally, enforced, we can't fix them all right away.

On top of that, those bug databases contain actual trade secrets.  It's not like we're developing the cure for cancer here, but if I had access to Blizzard's bug database the value to my company would be very high.  Like hundreds of man hours high.  No company with the primary goal of making money (hint: all of them) is going to publish that kind of information to the general public.  And please don't try and tell me that a beta community isn't the general public, or sell me on NDAs somehow making this a non-issue.  If you really believe that I have some 'beach front' real estate that I'd like to sell you.

Creating a separate database that tracks player created bugs and known issues is a lot of work.  It might be worth it, certainly it would be nice, but what other features is it stacked against?  Web-based guild management tools?  Web-based blogs for characters?  A WoW-style armory?  No developer can afford to implement every feature that 'would be nice,' and ultimately you have to ask yourself what is going to attract/retain more players.

[IainC]

I'm just wondering as a tool to a CM do developers data mine across their forums, feedback, bug database to give a large picture?

 All the beta's I have taken part in was one giant vacuum where /bugs were being reported, threads on beta forums for an issue or feature going no where with inherent bickering that really puts beta tester(s) in a negative feedback loop. I always wondered why its an issue connecting /bug or /feedback to an actual tracking system that has some transparency to a tester they can interact with. It's really terrible to have a ticket closed saying its known issue when you already took the time to search their beta boards but have no other way to check to see if your wasting time(the company and yours).

 Perhaps I'm just still jaded over Lum's GameCo example.

Yeah, I also read the feedback forms, talk to beta testers ingame (as a regular player, not with an 'official' character) and imcorporate all of that too. Our QA department are the ones who tend to track the bug reports and they field those. I skim them to see if there are deeper issues tied up within but I don't feedback on bugs directly.

[UnSub]

I'm going to field this one.

A key issue I see coming through here is about CM communication with the fanbase. While I think it sounds like you've got too many plates to spin anyway, a lot of what happens on beta forums can be resolved a lot more quickly if CMs / devs actually put their stamp on a thread. Fuser is right - a lot of beta forum posts can get into negative feedback loops or fanboi / h8ter pissing contests. Some positive reinforcement from official channels can easily prevent this (so will the threat of a big stick, so whatever works).

I don't think being a CM is an easy job, but any means. Players are really only just starting to learn what CMs do on a day-to-day basis. However, I do think that communications with MMO players - especially during beta - will likely improve with more transparency and more indication of action. Posting "I've read this and will forward it to the relevant person" means a lot more than dead silence in a thread.