f13.net

I doubt this exists but is there any list of estimated budget and development times for the larger MMO's out there?  Similar to the subscription graphs and such.

Does anyone believe a good, modern, solid, polished top tier MMO can be made "on the cheap" or in a short development cycle?  Maybe on the cheap is not the best phrase, how about for significantly less (i.e. half or so) than WoW spent or in a shorter time than they took.

No.  Looking at TR and other recent dogs, I don't think spending half of WoW's budget even gets you out the starting gate in good order.

I think an analogy would be like creating a summer block buster film  vs. creating a movie for that goes straight to DVD.

I hesitate to even say that any of the old guard could even create a good, modern, decent MMOG. I love those guys but man, their game development skills are approaching prehistoric.

Just because you understand the process doesn't mean you don't suck giant ass at it.

(http://z.about.com/d/familycrafts/1/0/f/o/1/ct2-38_twine.jpg)

It could be done but would have to be somewhat different in approach. WOW is already "on the cheap" in terms of graphics. You could make a solid, polished MMO for cheaper but not following the same formula. You could do a Second Life type of thing or an MMO racing game or MMO Animal Crossing or whatever...but you aren't going to produce a game like WOW for a much lower cost.

It could probably be done but you would need the right group of people working on it otherwise you end up with games like EQ2 which was made for about half of what WoW cost to develop. Less time is not going to happen unless it's a radically different game design than your (now) traditional raid-oriented Diku MMORPG like Margalis said above.

Dash, the genre has fractured so much that the sort of baseline you seek is somewhat impossible, without more definition. It's also something of a competitive advantage for companies, and as such, a fair bit of proprietary information. But, there is enough info out there that we could guesstimate a few things. And maybe a few red names would be willing to chime in.

What is it you're looking to understand? To help answer that, as a place to start here's some stuff off the top of my head (qualified so some can dispute it ;) ):

• 3D virtual world graphics (EQ2 on high end, WoW on middle), 2D Flash (Club Penguin, Dofus), 2D bitmap (old UO)- (type of art drives different types of costs)
• Licensing a graphics engine (Unreal, ala Vanguard) versus buidling one (err, I think WoW?)- (recurring fees, renewals, etc)
• Full persistent public space environments (EQ1) or instantiated (most everthing since, including later EQ1)- (affects server tech)
• Licensing server tech and middleware tools versus building (not sure why you'd do this of course)
• Size of projected accountbase
• Projected concurrency during peaks and valleys
• Fully player-customized experience (SL), mostly player-customized within a game-directed framework (SWG), mostly game-directed RPG-esque experience (GW)

Some quick numbers:

• EQ2 was stated to have a development budget of $25mil, two years before it launched. Some of the stuff built for EQ2 has been extended for use on other games in the SOE library.
• Club Penguin was stated to cost roughly $3mil. They licensed the Smartfox server system, built it in Flash. They got bought this year for $350mil (with another $350mil if they hit certain performance metrics).
• WoW had a $75mil development budget. Then above and beyond for marketing, sales, localization, etc. (and no, they don't collect 8.5mil * $14.99/mo, but are still burning money for heat)

Hope that helps get ya started.

That's wrong and will continue to be wrong no matter how many times you guys repeat it.

Your response is uninformative and will continue to be uninformative no matter how many times you think just saying "wrong" is enough.

I've talked about the cost to develop WoW many times in the past. You just ignore all facts that don't fit with your view of things.

I've always wondered about the production schedules dev teams keep. How often do MMOs release when they are originally expected to do so? Seems to me, it never happens. Is there an example of a Massively Multiplayer Game that released on time, rather than going by a "It will be ready when it's ready" state of production?

Tabula Rasa was most recently quoted in the New York Times as being at least 20 million (http://www.nytimes.com/2007/11/02/arts/02tabu.html?_r=3&ref=technology&oref=slogin&oref=slogin&oref=slogin) and an older article in the Korean Times thinks it's more like 100 million (http://www.playnoevil.com/serendipity/index.php?/archives/1624-Noted-Excellent-Article-on-NCSofts-Business-and-Prospects-at-Korea-Times.html).  I'm honestly not sure which it is, but it's interesting when considering Tabula Rasa was over six years and one complete scrapping to neigh nothing when they decided that multi-horned unicorns probably weren't going to sell.

Bah.  If you ask me (and why would you) it's not about the money, it's about the talent.  Getting distracted by the dollar signs is the road to ruin any kind of art, games included.  Runescape was neigh free and it doesn't seem to be wanting for players.

I agree with that last part, but I'm pretty sure the current talent pool is mostly stagnant.

Currency difference resulting in such a wide spread? Or maybe one number was development and the other included everything else (though that's quite a broad split to account for marketing and distribution).

And the money brings the talent. It doesn't need to always, but you're not getting a TR for $3mil no matter who you have on the credits roll.

No. I just don't have the luxury of refreshing the board all the time. Glad I can hit it when I can because I usually learn something. And usually I can learn something from you. Often though it's a noise thing.

My rough, unsourceable understandings of MMO costs range from $14 million for CoH, $20 - $30 million for a number of other MMOs and up to $85 million for WoW (which Trippy can provide a link to disprove if he wants).

I can't remember the name of it, but one MMO was developed and published pretty much entirely from one guy's house. That probably would have been a lot cheaper than the above examples, but also a lot more limited.

It seems that most MMOs take somewhere between 3 and 5 years to come out from announcement to launch, but that's not a guarantee and pre-production can add a lot more time to a project. (Also, imo MMOs shouldn't launch websites proclaiming their existence until they are 90% content complete and a month away from the external beta.) My non-dev mind finds it hard to consider that MMO development taking more than 3 years as being a good thing, since every three years sees a large difference between hardware, software and infrastructure from when the game was mapped out.

In order to cheapen / shorten the development cycle, you'd probably have to either buy out an existing MMO and revise the content as you see fit and / or rely on third party MMO middleware. These things are only just starting to happen in the genre.

On dev time: depends on how much you dedicate to testing it and at what stage (constantly tune a good system at Alpha, toss the system for a new one to start over, just add content to a system pretty much done, etc)

Sherwood Dungeon was stated as being developed by one guy. I don't know what server tech he used, but the game client is Shockwave.

The cost to launch WoW in NA and South Korea was around ~$65 million. This includes development costs, all the server farms, network operations, billing and other CS infrastructure and so on. This is straight from a top ex-WoW team member who was initimately familiar with the budget. I'm estimating just the development costs to be roughly $50 million (it's probably less than that since setting up 3 separate server farms is not cheap) as you can tell from my post above comparing EQ2's development cost with WoW's.

It's true that money might bring talent but, sadly, money seems unable to create talent - no matter how much the suits wish this were the case.  If the history of MMORPGs has taught me anything, it's that investors can't tell the difference between a game artist or a game hack even to save their pocketbook.  (I could be fair and say that gamers are fickle, but where's the fun in that?)  That's why would-be developers tend to talk fast and hope nobody notices (http://www.mmoginfo.com/pc/Dawn/index.html).

So before heading to bed, I decided to do a quick search to see where you've talked about the development cost. Found this one (http://forums.f13.net/index.php?topic=7760.msg211904#msg211904) from Aug 2006 where you say pretty much the same thing.

But I felt then the same way I feel now: how can you decouple the cost for developing the game itself from whatever was invested for the technical backbone? Most companies aren't SOE where they can "flip a few switches" to roll in a new MMO to a robust billing and CSR infrastructure. So everyone has some amount of "all the server farms, network operations, billing and other CS infrastructure and so on." We could go further and start guesstimating spend on marketing and advertising, localization, the licensing program with other companies, the sorts of things, again, everyone has to do in some form.

At this level nobody just get's a pallet of cash to go build a game, so you can't just focus on the tools and talent that built the game when comparing "what it cost" different companies to make them.

Well, ok, you can, but I don't think that's as useful within this particular thread. Dash seems to want to know what it would cost a company to "make" a game, and "making" a game is, well, I already said it.

Edit: and yes, I know this deviates the same way I usually deviate from the one single datapoint you usually like to talk about. I do it because that is how things are.

The talent pool is not stagnant as much as the people hiring the talent are.

Most devs developing MMORPGs today are also-rans who have a track-record of faiure. The funny thing about Blizzard devs is that many of them had no track record (for MMOs) at all. Yet that turned out swell.

A good developer is going to trump a bad MMO developer, every time. Maybe the one or two guys doing the fundamental server work need to have very relevant experience but the rest don't.

You don't need MMO experience to program a graphical client, an auction house, quests systems or anything like that. The distinction between an experienced MMO developer and a vanilla developer is vastly overstated.

Thanks for the replies.

I just find it difficult to believe that people would expect to make a MMO competative to WoW in quality for less money and in less time.  I'm not saying you need to emulate WoW in every respect but should you expect to make a game of the same quality and depth in basically the same mold (raid DIKU MMO) for 20 mil in 3 years when WoW took say 70 mil + and 5 years?

I assume companies are thinking "No, we're not" and are looking to make simply good games in the traditional development model (read: buggy, incomplete) and for a piece of the pie that Warcraft made so big.  I guess what I'm saying is budget and dev time set my expectations. 

They may expect to but recent history shows pretty conclusively that they can't.  No one is even within an order of magnitude of WoW's sub numbers.

Here's one thing to keep in mind about WoW's numbers though: afaik they span a lot more territories than most other MMOs we talk about here hit. I think that's important because it shows that while any one company beating them worldwide is probably impossible for any of the games on the radar, that doesn't mean they can't best them in one territory, and be deemed a "winner" because they are only in that one territory.

It's kinda like some analysts (like this guy from Merrill Lynch (http://www.ibtimes.com/articles/20061114/playstation-wii-xbox360.htm)) in 2006 talking about the console war. Some actually felt Xbox 360 could win the worldwide console war simply by winning North America alone, due to sheer number of units sold.

You don't need to beat WoW everywhere in order to be beating WoW at all. So it's not a total loss if you don't have more money and more time. imho anyway.

I've always wondered (these days) why dev studios don't start with smaller (well done) game worlds, then expand consistently with new planets/areas/content.  Less focus on making 15 worlds/areas 'different' initially, and instead have 5 well done worlds.  Shorter dev time, build more solid game systems, get to live and continue development of content and other systems with incoming revenue.  You'd have to have flexible engines/tools to do it, but hell, that should be the focus to begin with.  You can somewhat pace advancement by tinkering with xp rewards as you go along.

Hardcore players will *always* outpace your content, and be screaming for more, so why worry about them?

They are all being a bit dumb... they are trying to create a $30,000,000 MMO that will pull in 8,000,000 customers. It's not going to happen. It's very very hard to have that broad of an appeal. Better to make dozens of lower development cost games (EVE for example) with dozens of different themes. Use the same engine, scripting tools, billing infrastructure for all. Then you can save on development costs and hit every corner of the market at once.

Part greedy/stupid, part MMO-dev snobbery: "Well, those RTS developers over at Irvine managed to throw together an EZ-mode mmog with crappy graphics for X tens of millions, surely we hardcore MMOG veterans can put together a better game with bleeding-edge graphics, traditional MMOG gameplay and more content for X/2 tens of millions. After all, we've done it before!  :uhrr:"

Yeah thats the other thing though.  I remember Brad saying he was proud to get 35 mil or whatever it was for Vanguard since that's a lot of cash for an MMO, yet still way behind WoW.  Then they decide to throw every system known to man in, plus new ones.  Housing, naval combat, flying mounts, diplomacy, leet graphic stuff, etc etc.  Let's forget management, could that possibly have worked? 

I think Vanguard could have had a good base, but no chance of competing with WoW because it was too buggy and clunky, whereas a LOTRO was smooth but lacked depth.  I think WoW is so smooth it's Console-like, i.e. it generally "just works", and thus is mainstreamed. 

No... about 10 years ago Brad made a game... through blind luck it was an awesome game.

He relied on the same luck with Vanguard. He found out the hard way, lightening doesn't strike twice.

When I win the lottery, I'm going to make the best MMOG ever.

Stop stealing my dreams.

The genre evolved. He guessed wrong, on what people would want and on what was capable of being built in the time frame he had. And no, I don't believe another year would have made a difference. The focus was alway on more cool shinies rather than on making what was there already polished.

When people talk about "quality", that's what they're talking about. You need to stop inventing at Alpha and focus on polishing through to launch. At this point, you're only going to get credit for what works.

You are being too kind. Brad is clearly an inept manager who cannot separate what he likes in a game from what makes business sense. That doesn't mean he doesn't have good ideas, but, as many have written here over the years:

Ideas are easy. Execution is hard.

QFT

Keep in mind that many of the big-name MMOs currently on the market have undergone major redevelopment during their production schedules, in some cases completely scrapping years of work that had already been done. Needless to say, this adds significantly to the final price tag for these games.

The MMO industry is still really young, and no matter how many cautionary speeches are given at conventions by those who have been through the process, it's still really easy for a company to misjudge the scope of what it's trying to do. This can result in the cutting or reworking of significant amounts of code, art, audio, and design. Not to mention the fact that gameplay and marketing assumptions made early in the development process might change during production, and someone high up in the company could come along and decide that the product needs to change its focus considerably.

That said, it's possible for a company to plan and execute an MMO without such hindrances; it's just often the exception rather than the rule, especially when big-name publishers are involved.

One mistake people usually make in software is to look at a product and say "hey, this should only have taken a year to make!" Which is true, if you knew the exact endpoint when you started and worked via a logical progression.

But as you say, there is a lot of stopping and restarting. The product I work on has been in development for 5+ years but I could write the entire thing from scratch in a year by myself.

Personally, I tihnk that Brad McQuaid did manage to pull off a better EverQuest.  Why not?  He had the experience and the knowhow to make it happen.  However, what he didn't realize is that (as DQ has said), the genre evolved.  Making 'a better EverQuest' wasn't good enough anymore.  Certainly not for those who were sick of EverQuest, and certainly not in the face of Blizzard's 'better EverQuest', World of Warcraft, which was geared towards the far more fertile casual audience.

This is not to say that I necessarily believe that a good MMO developer is needed to make a good MMO.  The World of Warcraft example is one.  The Final Fantasy XI example is another.  To wax optimistic a bit, I do believe that there could potentially be a lot of talented developers out there yet.  The only trouble is that we haven't heard of them yet and neither have the developers.  When there's a Serek Dmart there will also be a Will Wright.  What's the differentiating factor?  It's already been mentioned:

Put another way, the devil is in the details.

Hopefully one of the characteristics of these talented developers is the understanding that WoW did not, in fact, target the casual audience, they targetted the whole audience.  I'd be willing to bet that every reasonably conconctable demographic that plays diku-style MMOs is predominantly playing WoW.  The hardest of the hardcore certainly among them.

LotRO is quite like that, though I don't know how well it turned out for them.

Sidetracking a little, can't help but wonder now if there's any "Plan 9 from Outer Space"-like MMO out there... or rather which one would fit. In term of development process, of course.

>can't help but wonder now if there's any "Plan 9 from Outer Space"-like MMO out there

I don't know, but there are plenty of "Glen or Glenda"'s out there.

But we've already mentioned Vanguard. :oh_i_see:

 :roll: Do you consider the term "screwed the pooch" a compliment? Because that's what your sentence indicates.

Dude, Vanguard wasn't even finished. There's no comparison.

The parts that VG got right were less EQ1 and more indicative of one of the lesser-leveraged directions even DIKU games could be going. Something that EQ1 and WoW both do well is start players off with very compelling racial backgrounds. You feel like you're from that culture. DAoC went too far by arbitrarily requiring three times the amount of development to simultaneously support three completed compartmentalized races. And the rest don't go nearly far enough, including EQ2. Some don't seem to have learned that one of the primary barriers to alt'ing, and thus to extending an account, is boredom wtih having to do the exact same content over again (waves at TR as the most recent offender).

In that sense, I feel VG did contain some nods to how the genre evolved, specifically by doing something the genre has evolved away from. But all of that is irrelevant because of the complete lack of realistic time management. I'd say that's the part he didn't evolve at all, the understanding of how to manage a project through completion.

If by "targeting the whole audience" you mean accessibility, I agree completely. At the root, IMO, that is the source of their success.

How quickly we forget that EverQuest, too, wasn't finished.

Ah, my first introduction to the /stuck command and falling through the world.  Good times, good times.

EQ1 could exist in the form it did only because of the time in which it launched. You got credit for trying then because the rules weren't established.

VG was trying to do the same thing years after that no longer was allowed by the market.

Going back to where my quote was pulled from there, we're in agreement, DQ.

As this thread shows, MMO development costs are as varied as the groups that produce them.
Nowdays there are several middle-ware solutions that can speed the process like the HeroEngine, BigWorlds, and even Gamebryo.

One of, if not the largest, costs involved with an MMO is simple asset creation. Especially if you're making a modern/futuristic/post-apoc mmo. But there are enough teams in China, India, and Russia to farm them out to for a decent price.

Speaking of middleware, there are now at least 3 companies (OGSi, for one) that have bundled CSR work, community relations, billing, and the ancillary functions needed to round out an MMO.

But let's have a little fun by working out our own budget for MMO "X", developed solly in America, and starting from scratch.

Administrative Costs

Office Infrastructure
   Rent
   Furniture
   Supplies
   Utilities
   Fun Money / Misc Cash box
   Signage

Salaries
   CEO - 60k
   CFO - 60k
   CTO - 60k
   Lead Programmer 60k
   Art Lead 55k
   Content/Writing Lead 55k
   World Building Lead 55k
   Sound Lead 55k
   3x Junior programmers (including networking, client, and database programmers) 55k
   10x asset modellers  35k
   5x animators  35k
   5x texture artists 35k
   5x world builders 35k
   3x content writers/ quest scripters 35k
   2x Press Relations people 35k

Technology Costs
   39 workstations @ 3k
   4 development servers (hardware) @ 4k
   Software licenses 30k-50k
   5x Wacom tablets
   Recording equipment 15k
   Bandwidth for development servers
   Bandwidth for live servers
   Engine license - between 200k and 500k

Publishing Costs
   DVD pressing and printing
   Box printing and handling
   Digital download server costs

Advertising
   Print
   TV - hire Morgan Webb on G4 to tell people it's the best thing since sliced bread
   Intarwebz - $1,000 per 15,000 impressions on IGN
   Game conventions - 5 conventions per year x 3 years of development x 8k-20k each

Ancillaries
   Customer Service
   Paid GMs
   Community Manager
   Back-end billing

and all that's just off the top of my head. It really couldnt be done well for less than 6 million - and that's assuming you subcontracted all this out to the cheaper subcontractors overseas.

Fun to think about, eh?




   

CEO at 60k? Man. I'm keeping my cushy middle management job  :-)

As to the other salaries, the Gamasutra magazine runs an annual salary estimate Feature. It's darned good reference.

I think your cost estimates are wildly inaccurate.

Meh, it's not like dollars are worth anything nowadays anyway. Maybe that's why NCSoft keeps funding US development teams - it might actually be cheaper to do that than do development in S. Korea! :lol:

Heh, much of this discussion is a little far down the road.  One thing that is lacking in the overwhelming amount of releases is the pure, simple fact that games just don't function when they are released.  Some recover, most never do but manage to have enough money coming in to support itself.     

Obviously mmos and single player games are still making enough profit in many cases to keep everyone involved in the process happy apathetic.  Until that changes all the talk of time and money remains moot.


 

Keep my net admin job or take a pay cut to be CEO... hmm...

Yeah. I wasnt trying to be scientific. Unfortunately the MMO industry is one of the worst when it comes to costs.
There should be some interesting numbers coming out before the end of the year from OGSi about the technological costs of everything from CSRs, bandwidth, etc in a white paper they're developing.

Most CEOs make more than 60k of course, but imagine being a fledgling startup with actual morals for a moment  :-o  :awesome_for_real:
Hopefully you wouldnt want to kill your budget pre-launch with a fat check and you'd want to be an equal with the guys you were in the trenches with. And then there's the back-end money that you've hopefully contracted for.

What I posted was just an early-morning ramble off the top of my head (see quote where I say off the top of my head) to spark conversation and get people getting deeper into the details.

This book (http://www.amazon.com/Developing-Online-Games-Insiders-Guide/dp/1592730000/) is out of print and about 5-7 years out of date but still has a lot of insights into what goes into backend MMO production, if you can find it.

Here's a more cynical take on Margalis' point. Note that all of this HAS ACTUALLY HAPPENED in various places. I'm not making ANY of this up.

Designer Guy plays PopularGameOfTheMoment, wipes on a bad pull, curses, says "crap, I know even I can do better than those idiots! Screw it, I'm gonna do it."

Designer Guy's friend with a trust fund says "OK", hiring begins.

About 6 months later GameCo, flush with $2 million in venture capital from Trust Fund Guy, opens up a shiny new office in downtown Frobozz. Aeron chairs and 22" LCD monitors for everyone! Designer Guy's other friend, Bob, is hired on as CEO because he smiles a lot and talks pretty. Hiring continues.

Bob does the rounds of game publishers, explaining their great idea for this totally original game that will smack everyone in the face. The usual MMO publishers snicker and pass, because Bob's company bought Aeron chairs and LCD monitors before actually making a working prototype. However Bob finds a kindred soul in a publisher, Activariplay, who wants to break into the MMO industry, because hey, how hard can it be! GameCo gets 5 million more funding and some vague milestones over the next few years. Hiring continues.

About 6 months after Activariplay's cash infusion, Designer Guy is somewhat worried that they still don't have a working prototype. This is because they haven't yet been able to find *any* server programmers, because anyone with any multiplayer game server experience at all is immediately issued 40 houris and all the opium they can smoke and chained to a desk by the first game company to find them. There also isn't a working client because Client Programmer Guy thinks licensing client tech is for babies and he is positive he can write his own 3D engine from scratch, just like that punk Carmack.

A year after Activariplay's investment, the first milestone is due. The collective staff of Gameco spends two weeks frantically injecting amphetamines so as to avoid sleep and trying to get ANYTHING up and running at all. Their milestone presentation consists of some NC-17 concept art by Art Guy One, a few half-assed character renders Art Guy Two managed to outsource, and an hour long Powerpoint presentation by Designer Guy where he tries to describe his vision for his game through sheer force of will. Activariplay declines to fund Gameco any further, mainly because all the Gameco principals are visibly twitching from all the amphetamines.

Bob goes back to TrustFundGuy and offers him a 40% stake in the company in exchange for more funding. Bob and DesignerGuy do the publisher rounds again. Hiring continues, although anyone with any experience at all in the gaming industry takes one look at Gameco's flush office and no actual work, talks to a few of the cynical artists smoking in the back, and run away in terror. DesignerGuy proclaims that the entire MMO industry is corrupt and incestuous anyway and begins recruiting kids fresh out of college. (The smarter ones of those do some due diligence and pass on Gameco too.)

Another year passes. It's now been three years, GameCo has burned through about $15 million of other people's money, and all they have to show for it is a lot of mildly pornographic concept art, character renders done by artists working on buffing up their portfolio for applying to real companies, and various fragments of a 3D rendering engine done by Client Programmer Guy who is still convinced he can write his own 3D engine Any Day Now. GameCo is now out of money and Bob and DesignerGuy have thousand yard stares on the few days they come into the office.

Big Bad MMO Publisher, who has seen the Gameco story many times before, swoops in and says that they'll assume GameCo's debts, pay off Trust Fund Guy, and fund GameCo's game, in exchange for outright ownership of all of GameCo's assets and IP. Bob, who just wants the nightmare to end, doesn't bother consulting anyone else and signs on the bottom line. BBMP promptly fires all the artists and Client Programmer Guy, parachutes in a deeply experienced and deeply cynical development team in the home office, and informs the shell-shocked yet somehow still employed Designer Guy that his brilliant and innovative MMO design, thanks to production realities, is being scaled back to "just like BBMP's other products, but with a few more pixel shaders". Designer Guy contemplates suicide or more drugs, and settles on more drugs.

Two years later, GameCo's Game is released. F7 proclaims it a failure and epitomizing what's wrong with the game industry today. It has hundreds of thousands of subscribers, and is considered a moderate success internally, especially since it really only had a 2 year development cycle once adult management took over, but its original fans who were attracted by Designer Guy's drunken rantings on message boards feel betrayed that the dynamic virtual world promised them turned out to be yet another moving the bar from left to right while keeping the other bar from moving from right to left game. DesignerGuy and Bob quit GameCo and form RealGameCo, where the dream will never die, word, y0. They attract $3m in initial investment based on their success in producing Game (despite the fact that Game was actually produced literally in spite of both Bob or DesignerGuy) and the cycle begins again, complete with Aeron chairs.

Bob McQuaid?

Sounds like Horizons.

Brilliant. 

That's funny. I don't think ATiTD cost quite that much.

I think the first ATITD had about six people total.

Hi Larry Us

@Lum:

Man, that is so freaking accurate--and while his description implies "indie" roots to the game, this is also the exact strategy many established studios have, which just bigger orders of magnitude on the dollar signs.

I've seen it at work myself as well--you're spot on!

Well, at the very least, the process gets us the mildly pornographic concept art.   :lol:

Damn.  Truth is on the Interweb after all.  Brilliant.

The brilliant part:  That is an MMO success story.

lum thats good stuff.  I've no doubt that sort of thing goes on.

That's my perception of some of the bigger games especially the part about "once adult management took over". 

@Lum:

Scott, that's totally inaccurate.  You're doing the public a disservice by blatantly lying like that.

They'd be 30" monitors.

Hey, I had to build my own desk on my first day at SomeOtherGameCo.

I had to build a bookcase my first day at Mythic.

I'm told they still have that abortion of carpentry stored somewhere, where they can point to it as an example of why some people should never be allowed near power tools.

Build as in planing wood or build as in IKEA?

Big difference.  :wink:

Not if you're an employee for a really real life company that's making you build your own goddamned office furniture. That's amateur hour shit there.

I don't know... making your employees build their own furniture sounds like the game industry I know. 
With any luck, they'll design a good furniture building simulator in the process.
Before long, trade skills come to be.

You've never worked for a small company then, the type that makes up most of America's economy? "Many hats" doesn't just mean one does both programming and some art work  :wink:

Can an a garage MMO be made? Yep. Minons of Mirth proves all you need are 3 people who are dedicated.  The problem is that people on indie dev teams dont usually stick with a project - most of the same work is done on mods, but those have shorter lifecycles because everyone knows it's about learning or pumping up your demo reel. But the tools are there for it to be done on a shoestring.

Actually, I have. I've worked at the companies that make you build your own furniture, or had me set up networking computers. Same company where the paychecks bounced every pay period for the last 6 months I worked there. Most of those companies do not succeed, because it's fucking amateur hour. The ones that do succeed don't do that kind of Mickey Mouse shit for long.

Anywhere from 20-70$ a square foot depending on where you start the business.
The furniture supplies, utilities is a ever growing never ending expense.
No such thing as fun money...
Signage? In the grand scheme of things a totally miniscule amount.

Holy crap! Take the top three salaries and at a minimum triple them, a minimum. That's if you're hiring someone that's accomplished and reputable. Someone that has achieved before and knows what the hell they are doing.
Lead Programmer is probably half of what a real lead would make. Same with the art lead. Other leads are a wide range of numbers depending on the credentials.


If this is a real MMO, not a huge one but a real one, I'd argue that you are dealing with at least 5-10 more engineers.
You basically have 20 artists. I know we are very early on in the process and we are currently at 13 split between character, environment and animation. I would imagine an art team ending up at 40ish or so, probably more.
3 "content writers", you mean designers? Ugh I am saying that's horrifically low. Also, 35k is an entry level non experience salary. Your "asset modelers" "texture artists" and "animators" are all artists. Good ones, talented ones, start at 60ish and get up in the 100s for the truly good to great ones. Designers run the gamut. Great ones make more than double what you are talking about, others make much more than that.
Press relations people? For a true MMO that you are taking to market and hoping to be successful it's a number so far beyond 2 I am not sure where to start. Not to mention your 'press relations' people are incredibly well paid, well connected and ridiculously important assets to a successful launch.

About half the real cost of fully developed workstations, 1/10th the cost to license the software, 1/20th or less of the cost to build out even a small modern audio dept.
Your engine license? Book that licensing one engine STARTS at a million dollars. That's no flat fee either. That's a starting point and for the most part that's not going to get you licensed with an engine that's actually shipped a game.


MASSIVE expense here. Expense you cannot fathom until you actually sit down and discuss a deal involving one or more aspects of this.



I'd argue that to author an MMO, not a HUGE IP MMO, but even a middle of the road amount of content is at the least a 3 year 30+ million dollar job. If you want to do it right. That to me is INCREDIBLY TIGHT and doesn't include any sort of marketing budget. I had no idea the massive undertaking 'making a cool game' was when I opened the doors last year. Scary? Hell ya. Daunting? Absolutely. Doable? You bet. But understanding the financial landscape was something that had I not done it BEFORE we signed off on this startup would have destroyed us.

I am in complete agreement that a fully built, tested and thoroughly designed MMO is now the equivelant of making a major motion picture with star talent. If you want to do it right it's going to cost you, but doing it right means a windfall of epic proportions if you team up with the right people and companies.

This made me giggle.


We're still talking about online gaming, rite?

C-crews do not come cheap no matter what the field. A CFO, CEO, CIO, COO do not need to be experienced so much in gaming as their respective specialties. The proper human resources working for each of the executives must be damn good as well. These people can most certainly be attracted to gaming houses with potential. That means a realistic model and a worthy paycheck.

Hehe ya, the online 'game' that's turned into a multi-billion dollar industry. WoW has made some see the pot of gold at the end of the rainbow as something attainable by anyone putting an MMO into the game space. The peoples salaries being referred to are people that do make a very nice paycheck because demand for the good ones is at an all time high, as it should be. Corral all the incredible dev team talent you want but if you don't have people driving the bus that are passionate, dedicated and totally committed, in addition to other worldly talented, you are on a bus that's never leaving the parking lot.

Even Curt Schilling is arguing with DarkSign now.  :awesome_for_real:

Edit: 60k for a CEO or a lead programmer is absurd, as is the CEO getting paid the same as the lead programmer.

Edit2: In business "achieved" has a different meaning from everyday usage.

Guys.

My comment referred to the "achieved" part. Not the cost or worthiness.

Fuck, I work for the government as a programmer and I make much more than 60k, and while there's big money in government contracts it is NOT at the peon level. And my company bills the government for at least twice that for my stellar services. :) And while there's a "Lead" on my title, I don't kid myself and think my skills are nearly as good or indispensable as a lead programmer for an MMORPG, which has many times my department's budget and potential profits.

I didn't see a figure for a DBA, but good DBA's -- and you want really good DBAs, preferrably with MMORPG experience (any sort of real-time, high precision DB work would suffice -- like bank or stock transaction gigs), runs well into six figures. Hell, the Oracle DBA down the hall makes twice my salary.

I know at the peon level the pay for programmers and such in the games industry is peanuts, due to a million applicants for every open job, but for the critical stuff -- not so much. The people with the skill sets you need are in high demand.

Good to see previous posts didn't scare gehrig38 off.

... we'll wait until 38 Studios release something before we do that then   :-)

There's a much easier way to estimate development costs which is to use the method VCs use to estimate "burn rate" for software startups. In the San Francisco Bay Area is used to be $10K per headcount per month. Not sure if that's changed in the last few years since I've been out of the startup scene for a while. Boston is about the same cost of living as the Bay Area so 100 people for 3 years comes out to $36 million or close to what Curt estimated. Now of course it doesn't quite work that way since you never start with 100 people off the bat so it's more like 10 people to start for 6 months or so and then things ramp up from there and it'll actually take 4 years instead of 3 which is how you end up with WoW-sized budgets. Note that the 10K per person is not the average salary -- it's the average salary plus benefits plus facilities (per person) plus all the other overhead to run a typical software business.

@lum:

you do know they still ask 140 dollars for that book ( used ).
Maybe Ive got to sell my used books to amazone then.
new way of funding my superb idea's on how to create the wow killer :)

Having done the convention circuit for a software company for a couple years - figure more like 3 - 4 times that per convention just for floor space, assuming you plan to have more than just one of the little table booths in the third hall. Plus the expense of sending every executive, marketing monkey, IT team... yea pretty much half the company, whether they have a reason to be there or not.

And don't forget the Aeron chairs and 30" LCD monitors. I'm not sure how you mount those on buses, but talented people will find a way.

And yeah, Mr. Schililng's salary and expenditure figures are closer to the norm. Although if you triple the lead salaries, like he recommended, you get $150K/yr. If I made $150K/yr I would be dictating this post to a lissome young houri while she fed me grapes.

Eh, a quality game can still be done on the cheap…

Don't need more than half dozen individuals… …no office necessary, just rent a 2-3 BR apartment (or home if cheaper), some doors and cement blocks can serve as computer desks… …most important assets are skill and time — talented art guy + programmers + sales guy/gal… …of course you need to eat while you're developing this wonder-ment and you + family (a detriment because they can keep you away from work!…) need health care and your bills gotta be paid…

My Mom is the landlord for the ATiTD fellows and their digs are far from spectacular though they are right across from an Eat-n-Park and down the street from Vinnies and Pizza Palace…

The thing about game development is that you are expected to be committed to it to the point of doing without the comfort of, say, a pleasant work environment.

It's one thing if I have a stake in the company and will realize some of it's income as my own.  It's a totally different one if I'm just a hired gun.  Underpay me AND have me work in a shit hole?  Nope, not gonna happen.  Not when I can get a job tomorrow that will pay 'the going rate' in a professional, clean and well equipped environment.

You're absolutely right. However, someone who can get the sort of money that is spent on an EQ2 or a LoTRO isn't looking to "just" do a near-hobbiest experience like ATiTD (which I very much agree is a great game, it's just wasn't ever intended to be a AAA title).

Way to mischaracterize. He's not arguing like you and I are. And I stated that all my figures were conservative, spit-balling, whackadoo bullshit.
Are you so intent on being a douche that you have to just make up bullshit?

Well, assuming game developers get benefits -- health insurance, vacations, holidays, etc -- odds are your actual "cost" to the company is a third again to half again higher than your salary. Of course, I bet you don't get overtime, so that probably works out for the company. :)

But when you budget, you have to budget for ALL the costs associated with an employee -- their salary plus the basket of goodies associated with them.

So even if 150k is a dream salary for you, if you're making close to 6 figures, you're probably costing the company close to that amount.

DarkSign: If you really want to play with costs, go download COCOMO II and toy with it. It's free, it's fairly self-explanatory, and at least for direct development costs will give you a semi-decent range of numbers, especially if you dig around and find accurate input ranges.

Will do. Much appreciated.

Trying to assess benefits is another factor for sure. One of the things I promised members of this family was total commitment. Our benefits package is total health coverage. No deductible full dental, vision and more. That tends to run somewhere near 35-40%  of the employees salary. So ya, when you factor in payroll, and you are projecting, I always added, at a minimum, 40%.

That is a damned good way to get and keep quality employees. Many people don't account for this when comparing jobs; just looking at salaries doesn't cut it.

I agree. You should be commended for that kind of commitment to family issues.

Coincidentally, I wrote some things (http://geldonsgaming.blogspot.com/2007/12/mmorpgs-for-niche-market.html) on this and decided I'd rather developers make small budget niche titles for me than big budget titles that only a bloated casual gaming market populated by apparent idiots can afford.

My company -- which has by far the smallest turnover among a bunch of similar contract-based companies around here, has a similar motto. Their healthcare is solid -- not as nice as that, but top-tier for big companies. What they do, however, is offer practically unlimited sick days (you DO have to get a doctor's note if you're out past a certain point, and they do automated audits to find out if you're routinely sick on Fridays. But if you get sick, they want you to get better -- not worry about missing work). They offer outstanding educational benefits -- two classes a semester, don't have to be related to your job, and they pay for tuition, fees, and books (up front). And you get a large bonus for any degree, as long as they're more than two years apart. Their vacation offers are generous, and they refuse to carry them over -- they insist that you use all your vacation each year. (It's a big problem in this industry, where people will bank 400+ hours of vacatioin -- usually in case they get really sick).

There's a bunch more -- their salaries are slightly higher than the average, for instance. But mostly they've worked out that they keep good employees if they treat their employees well, rather than as people that can be replaced at a moment's notice.

It's amazing what some companies will or won't do to keep employees.

I interviewed a guy once who quit his job because his company didn't appreciate the employees. The only example he could come up with was that they had a free soda thing going then scrapped it. That sort of gesture costs nothing but means a ton to employees.

Obviously health care plans and such cost a lot, but constant turnover is death for a company as is lower-quality workers. In software at least a really good employee is worth five normal employees. Anything you can do to find and retain top talent is worth it.

Curt Schilling is serious about taking care of people, both at work and in his charity work. He is one of the genuinely good guys.

Edit: Even if he did put ketchup on his sock.  :awesome_for_real:

Even with the Everyone Must be Insured law in Massachusetts, he is going above and beyond.

But this is just another example of real changes coming from outside the establishment.

Oh, I could rant and rant. But I'll just say "Agree emphatically"

I need to win the lottery so I can become Johnny GameCo President.  Seems like a fun ride. 

I only studied Engineering in college and got into sales.  I wish I could of parlayed that into game design.  Sounds like an entertaining industry.  Oh wait I just remembered stories of 400 hour weeks for QA slaves.  I'll stay where I'm at, nevermind.

@Lum
This is a month late, but that was a damn funny post.  Cheers.

Even in that respect, there's been a trend towards more reasonable conditions. I think it's because a lot of those young developers aren't so young anymore, are having families now, and want to spend their weekends playing Nintendo with their wee ones. :)

I just thought I'd say congrats to gehrig38 for not being in the news today.

Hehehehehehe.

That's cold.

Amen to that.

Just read your post, and one quote stood out...


Totally agree with that.  I think too that you yourself have to be part of that niche if you want to do it right.  You can't make a niche gardening  MMO unless you are yourself a passionate gardener or can become one.

My theory is that the world has room for just a few AAA type MMOs, and there's a big queue outside to get in that room.  However there's another "room" that's big, huge and somewhat empty.  ATITD, RuneScape, and others are pretty happy in that room where you can hear yourself think and others speak.  Why would you want to be in the AAA room?

Moneyhats.

Well yea that and the hoo - er escorts.

Prestige, clout to get the IP you want, clout to get VC to do it yourself, general market exposure, etc. Stephen Spielberg didn't jump from a nobody to Schindler's List, for example.

Some companies are fine doing the exact thing they a) love to; and, b) are good at. ATITD comes to mind. Eve doesn't though, because CCP used their abilities and success to secure the WoD IP. Funcom is the same in that they used what they learned to create a compelling reason for the Conan IP holders to let them play with it. Blizzard, too, they've done nothing but stretch, by continually showing they can cap genres. I'd love thinking about what'd they do if they decided to go after straight-up RPGs and FPS games.

The larger point though is that there's always room for everyone from indies to public companies to play in a space. You just manage your expectations going in. And then you decide how you want to leverage your success if you have one (maintain vs grow vs stretch).

After reading all this stuff, and a very amusing dev cycle by Lum, I'm glad that I've remained in the player community.  I've had a few offers in recent years, but to me it doesn't seem like the gaming community is really connected with the players at the guild level.  Here we are making community based games where guilds are a major component to that community, and yet no gaming company seems to really care about the issues that make guilds come to or leave a game.

Instead we get grindy POS games that require massive amounts of time, people who can't keep up with their friends have to quit, and eventually the whole guild burns out and moves on mostly due to unfriendly game design that keeps forcing their friends to leave.

Guilds are no longer new to gaming, and many follow their guild from game to game. So when developers make these games a headache for guilds to thrive, the whole guild leaves.  If a game doesn't totally rock, the veteran guild will usually depart within 6 months. So take that $14.99 x 100 = your loss.

The counter argument here is that new guilds always form, and that is certainly true.  But new guilds face lots of challenges and drama, and that can turn people off to future games where it seems like you need a guild/clan to advance. To me its best to support guilds if you plan on making a game that requires a group to progress.

So my contribution to this argument is that dev companies should look at hiring experienced Guildmasters as consultants to a project because just hiring developers who happened to be a member of "x" guild just doesn't cut it. Anyone can be a guild member, but guild leadership can give you a good perspective on how your game design is going to effect guilds in your game.

I agree with that sentiment. Guild-level quests are a start, but even consulting with guild leaders, perhaps in a special forum, to help keep a game on track or adjust the rules.  If MMOs are going to improve it's not going to be from automation or technology. It's going to be getting back to community roots.

Are there enough game-independant guilds out there that will make an MMO successful yet not spurn the casual/hardcore player that plays these games without being in one of those guilds?

For a person not in that situation they might feel spurned or left out or what have you when it comes to those games and they don't feel like they are being payed attention to.  Just look at all the wonderful hardcore/casual bickering that continues today that was pretty ugly during WOW 1.0

I would say that guilds that are multi-gaming guilds are probably the best candidates.  Most multi-gaming guilds usually have hardcore, casual, and extreme casual gamers and they have to juggle the interests of all of them.  Over at Lords of the Dead (http://www.lotd.org) we have a hardcore chapter, casual chapter, and sometimes an extreme casual chapter going.  All of our chapters are lead through a unified guild governance and oversight structure.

So I would say that a developer could probably look at what they feel the ideal group size is for advancing through the game (i.e. WoW 25 man raids), and then find a veteran guild or group of guilds large enough to have a good chance of progressing through their game.   Finding guilds that at least cater to hardcore and causal gamers is pretty important because you will get valuable insight from them about how they use the game mechanics, or how much crap they have to put up with to compensate for poor game mechanics.

As an experienced tester for over 10 years, I usually find that focus groups get dominated by a handful of hardcore guilds who drown out everyone else. So while I do think community feedback is important, I really think dev studios should consider a more professional link between established guilds and their company if they want to build a good game that will help the average guild thrive.

I know a few games that do this with guilds on Test Realms for their respective game.  But during the development stage?  It would scare me away if the dev house was asking random internet gaming guilds for advice.

It is time that 99.9% of the feedback from either guilds or individual players be summarily dismissed.  It only erodes what little innovation and originality a development team may come up with.     

I never said ask random guilds. I said ask established guilds, as in the ones who have been around a long time and played multiple games.  And specifically I'm talking about Guild Leaders.  Put them under an NDA, maybe a consultant contract, etc.

By the time we get brought in now the "Dev vision" has already been coded, and there's no time to fix anything if the game has already been designed to be guild unfriendly. 

Or they can keep doing what they are doing now and making games that make guilds have to go way out of their way to thrive, and then continue to lose accounts in mass when the guilds won't put up with the BS any longer. But as I said before, the downside of that way of thinking is that people are getting a negative perception of guilds in games. So it makes more people think twice before even buying a game that looks like it requires a clan or organized group to progress.

In the end they are gonna do what they are gonna do, but we sit here and bitch all day about devs not thinking outside the box so I gave a suggestion.  No one in here but me ever talks about guilds and game design from the perspective of being an actual guild leader, and then they wonder why guilds end up hating on their game.

So only the gaming Elite can do this?  Sounds snobbish to me.  I think the dev houses should build their game and ignore all the asshatery and forums.  If their ideas are flawed from the beginning there is no amount of fan suggestion that will make the game work.

And I put up that "thinking outside the box" tag line should never be used again when it comes to people saying things should improve and not put forth their own ideas.

This is largely due to developers making games for individual players first, social groups second. Some might consider features for large connected groups of people (guild banks, guild houses, fighting over them, etc), but they expect that most players are not actually so closely connected in this way.

Many have probably run all sorts of metrics to prove it out. If I had to guess, I'd say 80% of all guilds out there have maybe 8-10 active players with a sum total of 20 characters or so. This is the typical size of a small-ish groups of friends that jump from game to game.

At the same time, the idea of bringing in the leaders of the groups could work. But only as long as you ensure you get a proper diversity of players. As part of a meta-game guild myself, they still aren't my only exclusive group because they tend to focus specifically on DIKU experiences, or at least end up spending the most time in them. That tends to narrow the scope of what they'd be most focused on advising to the developer. And down that path lies the whole VG/FoH thing. "We're making a game for you, and here's 2,000 word posts on why!".

I don't think meta-game leaders of DIKU-focused guilds are narrow-minded per se. Rather, they understand that, at heart, the best way to maintain social stability is to constantly have busy people. And the times people are the most busy are between level 1 and the cap, or are achievings at a good rate at the level cap. How many large guilds are praying WotLK comes out in the next few months? I'd bet quite a lot. Right now people are split between being able to raid and all and being cockblocked from doing it. And that split leads to boredom, which leads to drama.

As a result of this, and the ways players have been seen playing in general, we get games of raw achievements with off-focus features coming sometime later. And can we blame the devs? Every game that focused on getting cool crafting systems, music stuff and housing out as quick as possible has been anywhere from uninspired mediocrity to a trainwreck on the combat, questing and general adventuring side. And to a game they have all been of either minor success or on life support.

Horizons...never played it but it focused on crafting and people hated the combat, amirite?

We call those "beta testers".

:rimshot:

I'm sorry, but this just reads to me like "I really think dev studios should listen to me. And ideally, give me money."

You state (correctly) that betas that do guild invites often have hardcore testers drown out the feedback of everyone else. A good CM knows how to filter that, and get feedback from both the hardcore board warrior 5% (which - hey - that would be you, Gus), the somewhat hardcore 10-20% who post occasionally on boards but usually lurk, and the casual 75% that think MMO boards are cesspits.

Another problem is that most MMO dev teams have no lack of feedback. At all. The problem becomes how well that feedback can be implemented. Sadly there is no "make it more fun" dial. If there was I would jack that sucker up to 11. However there are other constraints, usually budgetary and/or time. Even the most hardcore of beta testers never understand that, and get (understandably) frustrated when the bug fixes and feature requests they demand are consistently ignored in favor of things they find unimportant. Because someone else (the producer, usually) is doing triage, and that triage is going to cut in places people don't like.

The equal and opposite reaction to waylander's post.



No time, no money and no understanding from the player base (understandably or not).  Turn, turn, turn.

Lum's avatar almost induced a bout of epilepsy.

 :awesome_for_real: :awesome_for_real: :awesome_for_real: LET'S YOGURTING!!! (http://www.youtube.com/watch?v=587Ex131-20)  :awesome_for_real: :uhrr: :awesome_for_real:

Well at least now I know where it's from. The weeaboo in me :heart: s her KAWAII-ness. Kill me now.

Edit: I fail at emoticons.

No its not a give me money or a listen to me. Its pretty simple that you guys don't develop these social games in mind from the perspective of a guild leader, or look as a guild as one collective unit.   You design games based in individual players, and group/guild features are usually an afterthought.  As such guilds have to make mods, utilities, and other retarded work arounds just to enjoy your games. This happens because you devs usually have no dedicated feedback from guilds until its too far in the design phase to drastically alter your "Dev Vision", and Devs don't like players telling them that their wonderful concept sucks.


Community feedback is great post launch for future planned updates, but I was talking about much earlier in the process.  Making a game the devs think rocks, but their guild community things sucks because its a virtual job is a recipe for bad word of mouth. In case you missed the memo, MMO Release doesn't equal money hats anymore. More people are jaded these days.


Once again by the time guilds or other players are brought into the scheme of things, the game is too far along to be altered based on feedback.  I will say that Warhammer did the right thing by delaying their game for a year and apparently listened to their players. Big studios can afford to do that, but smaller studios pretty much have to march blindly to their doom if they find out too late that people hate the "Dev Vision" for how people are supposed to progress/have fun in their game.

As I said before nearly everyone here is a dev, fansite rep, or somewhere inbetween. You guys have serious tunnel vision when it comes to forcing us into group based progression, and then continuing to handicap guilds (hardcore and casual) to make it a virtual job to play your games. 

So instead devs bake up the game, get community feedback, launch, and reverse engineer it until everyone is happy or their game goes down in flames. Instead they could have just gotten good feedback on their concept initially, and focused on game enhancement rather than reverse engineering.

I find it ironic that companies will go out and hire other so called consultants on game design, but tapping veteran guildmasters the same way for feedback on guild concepts is somehow nasty.  So instead the talking heads keep talking to one another, guilds keep being treated a a second rate entity in the game, and game developers wonder why their guild community goes to crap and negative word of mouth gets spread about their games.

GG Devs, you win while your customer loyalty and return rate goes to hell.

I know my post got lost earlier, but I truly do believe this is because these games are designed for individuals first. If they band together, great! But the scale you're talking about is not a large percentage of the players.

I like how you treat devs as some bizarre otherworldly species that come down from the Design Mountaintop with no contact with the outside world. That sort of rhetoric plays great on message boards, I'm sure, but has no relation to reality. In reality development teams actually mirror the rest of "gaming society", skewing somewhat more hardcore usually. Our design meetings here usually start with a breakdown of Game Whatver's latest patch, which is almost always far more jaded than anything seen on message boards.


Once again, great rhetoric ("IT'S A VIRTUAL JOB!!!1") but removed from reality -- at the early stages of development, there is no guild community because there is no community, because in large part there is no GAME. For the vast part of an MMO's development cycle, the "game" part *sucks* from the standpoint of the player, because what little content has been implemented is broken, unpolished, and likely to put out your eye. Talking about "guild community" at that point is laughable, unless the QA testers have formed their own guild or something.

There is a stage when people are brought in to comment on the early "fun factor" of the game - call it late alpha, early beta, whatever. That has to be handled very carefully, because, again, most of the game is at the poke-your-eye-out level, and an immature tester who's just in it for the free game can totally torpedo your game's success by getting on boards and telling everyone how much your game sucks (even though it's not done and said tester broke about 12 NDAs in expounding on their opinion).

The alpha/beta external tester who can give you an honest, unvarnished, non-ass-kissing evaluation of your game's fun factor while still being able to professionally deal with missing parts of the game itself is a rare and insanely treasured commodity. They are RARE. I have my own personal list that I treasure highly and most MMO devs have similar battle-won lists.


YEAH! FIGHT THE POWER! DOWN WITH THE MAN! Oh, sorry, rhetoric again.

What makes you think veteran guildmasters *aren't* tapped for feedback? Hell, I have about 10 on my contact list I plan to abuse at the earliest opportunity. It's not that the feedback isn't there. It's that you disagree with how it's used. Apparently you have this vision of a "guild game" with "guild progression" and "guild combat" and "guild insert other buzzword here" that no one has yet created out of the many guild-centric MMOs released to date. You may want to make a game pitch out of it, because it sounds like no one is going to do anything you're happy with until that happens. And that would certainly get you on the ground floor of being able to kibitz about the design features!

waylander, what you're saying is that game developers don't know anything about games - only how to make them (and not very well).  The term "armchair quarterback" leaps to mind. 

Based on my own experience from a game that has my name on it, "Focus groups" (aka guilds, "forum vets", "I have been playing since Beta 2" people) in general suck as feedback sources because... 

• Any well organized "focus group" will bitch and moan to no end if you don't do it just as they wanted it - and they will let you know for YEARS  after the release that if you'd just done it like they told you to, it wouldn't suck
• They often can't agree on what they think is important
• They are a minority of the player base who talks the most
• They are in general pretty opinionated (good thing and bad)
• The ones that talk to you the most often are the more juvenile, moody and opinionated
• They rarely can see the flip side of the coin which is developer realities

It's a cop-out to say it as a developer, but some of them should just go make their own damned game and show us how it's done.

Which goes back to what I first posted -niche games made by members of a niche and aimed at the niche,  As a development approach, budget isn't necessarily a huge deal.  It's not the only way to do it, but it does work for some.   If I was going to make a game for the guild/organized player niche, I'd definitely want to be in or get in that scene and understand it.

Waylander, I'm genuinely curious as to what kind of "guild content" you want to see in a game that isn't currently there.

MMOs may be designed for multiplayer experiences (including guilds) but the reality is that that vast majority of players are individuals. Solo play is also immensely popular. So while guilds may want to pretend that they are top dog, a game that forces guilding or rewards guilding excessively just ends up pushing away all those players who can't / don't want to commit to a guild. Individual players would certainly be a larger population group than those in organised guilds, so it would certainly seem wise to keep this larger, often silent group content rather than listening to every request made by the guilds.

That said, the only tangible complaint I saw in Waylander's comments were MMOs not allowing players of different levels to play together easily, thus making a guild's life more difficult. On that note, every single MMO released from this point forward should have a version of CoH/V's sidekick / exemplar system, where a characters of any level can team up with another character of any level and agree on what level content they are going to do (either by sidekicking someone up to the higher level or exemplaring the other character down). Some of the longer content (Task Forces / Raids / Zones) have lvl limits that sk'ing / exemp'ing doesn't get around, but for the most part, it removes this issue.

 :hello_thar:

They are worth their weight in gold, and (I suspect) given "Do you want a job?" calls when openings occur.

Serious question: if these types of testers are so important, why does everything about the typical alpha/beta tesr actively discourage them from joining?

I don't feel like elaborating, I suppose I can if asked but we've talked about it before and it's farily self-evident IMO. From my perspective the process is geared to produce lousy results.

I got a PM about this so I will elaborate. My comments were geared towards major offerings. Their alpha/beta usually have many of the following characteristics:

1. Pain in the ass to actually get the alpha/beta software, must be a paying member of Fileplanet or something similar.
2. Alpha/beta (I'll just say beta from now on) advertised mostly on fan sites as a way to play the game early. (Advertising)
3. Going a step further, alpha/beta being a literal contest of some sort to get into, based on how much of a fan you are.
4. General attitude is that developer is doing us a favor by letting the slobbering fans play the game early for free.
5. Must fill out annoyingly detailed system specs sheet.
6. Minimum system specs for beta often quite high, beta restricted to high-end machines.
7. Poor bug reporting tools.
8. Poor communication with the players about what is and isn't supposed to be working and what sort of feedback they are looking for or even how to report problems at all.
9. Beta not anywhere close to beta quality.

I am a good tester. I am a logically minded person, I develop software. I know how to report bugs properly. I am able to overlook placeholder functionality.

I do not want to enter a fucking contest for the incredible honor of playing your game. You are not doing me a favor, I am doing you a favor by helping you make your game better for free. (As opposed to my standard consulting rate which you can't afford)

I should not have to have hardware that is all less than 6-8 months old to test your game. If so you're doing something wrong or you specifically designed your game to be an utter failure. (Hello Vanguard)

If you give me something "beta" quality that has tons of undocumented but obvious problems I'm going to assume you don't actually care about the quality at all, and I'm going to assume that any issues I run into are issues you already know about and don't care to fix. Don't ask me to test drive a car then give me a rusty piece of shit with four flat tires. Or at least warn me about the tires in advance.

If you can't give me a good way to file bugs or don't even bother to tell me HOW to file bugs then I won't file them. Help me help you.

If you aim your beta testing at superfans you will get superfans, not people like me. Superfans are not good testers.

In a nutshell: take it seriously and professionally. If you can't do that then don't expect me to.
---

For a small company...it's tough. As Lum says getting good feedback is very hard, and for small companies without name recognition probably based on sheer luck and who you know. For a large company getting feedback is still hard but using your alpha/beta as a marketing fluff tool is going to assure poor results.

Most of your points are valid, with one huge, huge caveat - if you're downloading it from Fileplanet, it's not a real beta any more. Call it a stress test, marketing beta, what have you - but by that point features should be locked down and you are for better or for worse telling the world "yeah, this is pretty close to what we're coming out with". That is also why reporting bugs probably take little importance at that point - with tens of thousands of "testers" reporting the same crash bug QA was complaining about 3 weeks ago.

At that point you should be testing for whether or not the infrastructure can handle the concurrency you're planning for, and if the game itself is fun enough for people to play for free (much less pay for). The only bugs that usually surface new at that point are the ones that only happen with tons of people on at once - the easy ones everyone can find, well, were already found, tracked, and are sitting in some programmer's in-box.

System spec breakdown forms should be a thing of the past since most apps include profilers now.

Having been in a large number of Friends and Family betas, I'll say that's not even a "real" beta per se. By that point the people you're catering to are all... friends and family with a swath of friends giving their other friends codes.

Part of me wonders if a real beta ever happens. I just don't like the word. The web 2.0 movement uses it to make people feel special. And so do games. A beta is simply put, marketing. Sometimes a beta is just a beta. Sure. But not once the public (no matter how limited the group is) see it. There are some huge ways I'd like to see these "betas" changed that would maximize publicity (yay PR!) and maximize feedback (yay community managers!). But that's a discussion for another time.

Also, very seamless feedback forms, in game, would change everything. But for whatever reason, people seem to be hooked on supplying nothing like that. Players already make "Betas" hard enough, no reason for devs to make them harder on themselves.

Real betas happen. They're just not for the public :)

For MMOs, haven't all semi-public beta tests really been little more than Stress Tests anyway, even before so many started defraying their costs by offering through Fileplanet? Inviting tens of thousands of people to anything pretty much means you a) know what they're going to do; and, b) know how you're going to handle it.


I consider this a good filtering process for what betas have become: marketing events. You get the people who really really want to be there.

For a "real" beta though, you go to your lists, draft a team by inviting folks, manage them as a QA Lead would, whether they are volunteers or paid, give clear objectives with measurable results, and make improvements where needed. Because even here, by the time you hit Beta, the core of your game pretty much is established, so you're leaning on testers to valid it.

I've definitely seen a change in betas over the years. The EQ and SB betas were longer, drawn out processes where servers went down for days, there were character wipes, and days of patching. You'd see many more messages about the types of bugs to be looking for and you'd spend your days trying to replicate that bug you found yesterday to see if some variation still worked.

Fast-forward to the beta-du-jour where people whine if the server is down more than 30 mins, getting your char wiped would just piss people off, and getting in en-masse isnt that hard.

I have to speak up for Waylander here a bit. I dont think that it's insane to state that an MMO elite (I hate that word but fuck it, that's what it is - not people who are better by character, but have played more MMOs and are "hardcore" another fucking word I hate) can give you better feedback than your WTFOMZBBQ! 10-year old.  Just about everyone in this discussion has either led a guild, been an officer, or knows someone who really knows his stuff and would be able to give "better" feedback than your avg mmo player.  (I put better in quotes because of course devs need some level of feedback from the avg guy who will make mistakes or approach the game as a larger % of the player base will...and that's of course valuable.)

I'd even go so far as to say that they could sit on a council that would discuss things like storyline and game rules. But here I go again...getting on the "wrong side" of the discussion.

I've beta'd plenty of games in my time.  I would always submit bugs when they occur.  I would post my suggestions in the appropriate channels.  The one thing as a tester that frustrates me is that there is no direction.  I don't know what the devs want tested.  I could play a game and say, "You have a huge problem with combat Mechanics... <insert disertation about what is wrong or buggy>."  Then I never get a response from any part of the team.  Has my feedback been heard?  Has this problem been mentioned before?  Are you aware?  Are you ignoring me?  Is it being worked on?  Is it just not in yet?  Is it a placeholder?  Then I see the identical post from other people hours or days later with the same concerns with no response.

Most public BETAs at any stage hardly have enough feedback from the community team that helps steer the concerns of the tester base in the right direction.

One thing I really feel could be handled better is the bug-/feature-request reporting process. Throwing up forum and expecting good feedback from anymore than the 15 people who vocalize the loudest is sort of a fool's errand. I don't think game company QA groups should be opening their Bugzilla or Test Track Pro databases to the masses of course. But I do think you really need to filter the information on the front end better if you expect than just randomly finding patterns in the static.

To me, the more text an end user has to type to explain the situation, the more likely you'll get many unique descriptions of exactly the same event. That's a lot of reading to substantiate what you already knew, when if you had more filters on the front end, you could know more readily what bugs there are, how many are being affected by it, and how much of an impact it has on the playability.

I'm a big fan of pulldown menus, radial buttons, 1-5 scales, and automated data collection (zone, location, targetoftarget, spatial conditions, etc) and then a text field that allows 256 characters at most.

It wouldn't let you collect everything, and isn't good for the meta stuff like look and feel. But that's the level at which forums become a good thing.

But I'd filter that too.

I wouldn't let "testers" onto the private forums until they met a few conditions:

1) Downloaded, installed, patched and logged into the game.
2) Played the game for 5 hours minimum, with actual activity (not AFKing)
3) Filled in a "first timer feedback" form, clicking voting buttons, entering lots of comments, having that read and then approved.

If you're only method for talking to your testers is the forums, then fine, let them read the FAQs and Dev info. But I've lost count of how many times tester impressions of a game are set when they're reading the other-tester feedback while downloading the game. Too much bias. Let them get their own impressions into the system and THEN release them into the wild.

Random comments:

- "Pain in the ass to get into a beta" - Would it be preferable to let anyone in, in which case 90% of the people that download it will stop playing after a few hours. Then, someone could ask the people that left quickly why they left and fix those problems?

- Direction for testing - What kind of direction should be given, and how? A one-line bulletin at log-in? One-paragraph? One-page?

- Bug reporting UI - What about something built right into the game?

- Feedback about whether one's bug is fixed and/or that you're being listened to - I know why this happens, but find it silly that such a simple feedback mechanism slips through the cracks.

Direction for testing: Be more strict about focusing player attention, particularly early on when specific systems are being added to the game. Late beta is that marketing period when you can let just anyone in to play the game for free.

Bug reporting UI- absolutely. My thoughts earlier were all about filtering the players feedback through an ingame dialog. EQ2 did this ok, with "report bug" options on almost every contextual/radial menu. Earlier, SWG tried this. Most others rely on some web-based thing, which immediately reduces the number of players who'll give feedback.

Feedback- This I don't consider as important. I appreciate everyone wants that ding of ego-stroking personal dev feedback. But to me the only important thing is well-documented patch notes.

From my experience in betas -

• Known issues list should be thorough and widely available.
• Inviting the longest serving, highest posting members of your pre-beta community isn't likely to provide good feedback. Roughly 40% of them will find that the beta isn't the game they imagined in their dreams and get discouraged. Another 40% will love it dearly and accept no criticism of it. 15% won't be able to get in due to technical issues. 5% may be able to look at your game and provide constructive criticism.
• It's been said already, but any beta that just puts players into beta and says, "Go play" is going to underperform. Beta testers need direction about what they should be testing and be willing to accept that once that test has been completed, their beta-leveled character is going to be wiped.
• Although it will be in crunch time, devs have to appear in some sort of public forum and explain why they've made their decisions. Yes, players (which they are, unless you are paying them) will be annoyed by changes during beta, but 1) they should expect it and 2) they'll take the changes better when the reasons behind the changes are explained, rather than silently dumped on them. Especially when the change goes against public opinion and / or isn't for something that is over-performing.

I agree with other peoples' comments.

Also, CoH/V has an in-game reporting system - the /bug <title> and /petition <title> commands bring up a window with some basic choices and a text box to enter your report. /petition is for when you need a CS to help you.

For the most part testers have delusions of grandeur. They go in looking to impact ideas in an individually focused manner, but in reality they're just there as part of an aimless mass meant to stress test the execution of said ideas. They want to help design the game where the developers just want them to test their own design.

As mentioned ad naseum, ideas are the easy part yet most testers somehow think they have the Midas touch when it comes to ideas whilst the developers are incompetent. It still amazes me how useless some testers seem to think developers -- folks who often put up with shit wages to do their passion for a living -- are when it comes to actual design, as if the execution is easy and they couldn't come up with good design to save their life. Ludicrously so, even, on the most simplest things; "you really think that a person who does this for a living somehow never fucking thought of that?". This isn't general developer praise either; it's a testament to how useless I've seen many tester 'elite' think of developers (of a wide range of background) and their creativy capabilities.

That grasp of development problems or gameplay flaws being issues of execution rather than issues idea is really the great hurdle that seperates a rare truly useful beta tester and just stress_tester_0432.

So it's no wonder that most betas are so unfocused and intangible. Testers end up harping strongly enough from their own armchairs of flawless design as it is, let alone if you engage and cater to them even more specifically and individually. That said, given the right context and the luck of having a truly constructive and mature beta community, a strong line of communication (and a tangible sense of result) can result in great progress and it's wonderful to be a part of. It's just that having all those stars align is a touch tricky. Many developers just don't end up with that base (or a community team able to filter through to it) to turn to; most couldn't afford to take such suggestions to heart anyhow.

Just because testers think they're better than they are doesn't mean that those that they criticise aren't as bad as the testers think they are.

But don't let that get in the way of your rant.

When I'm BETA testing something, I don't expect a 1 on 1 dialogs with the devs, nor do I think I'm shaping the game at all.  But when I see that something is broken and there is no acknowledgment it's pretty stupid to see 100x posts on the same subject on the forums.

I agree. My goal wasn't to justify either extreme (or to come off as ranting or confrontational; it was genuinely meant as constructive debate) -- whether on developer dialogue or developer creative competence (or the lack there of) -- as much as delving into the middle ground. On either side there are instances of superb dialogue or ridiculous denial but most major betas fall more in a sort of comfort zone where they may still communite with mass, they do so very carefully and often impersonally with said mass.

WoW was the strongest example of that. There was communication (though not quite as strong as, say, StarCraft or WC3x) but for the most part, they only wanted raw stress testing and cared little for personal take on design; they were set in there own and cared little for what anyone said beyond whether or not their design was polished.

The idea I was aiming it was that in many cases, though greater dialogue can sometimes work out given the right context, I can certainly see why many developers would want to try keep the beta tester mass at a certain bay.

A "known bugs" list is a huge thing that I don't see enough in betas. And when it is there, it's always really out of date. If I'm testing your game, while I'm sure the bulk of my contribution is just being there stressing your server, I'm quite happy to report bugs.

Just give me an easy way to give you the detail you want, and a way to make sure it's not something you already know about.

Wasn't stuff like the final version of rest xp, some deep changes to class skillset (invisibility for mages and whatnot) and other pieces and bits of mechanics ... essentially result of testers telling Blizzard where they could stick their original version of the design?

and re: "you really think that a person who does this for a living somehow never fucking thought of that?" argument, the thing is some of stuff you see put in the games really make you wonder exactly that. 'They do it for living' does't carry that much weight when it comes to such hit-and-miss field as inventing ways to have fun.

Yeah, I seem to remember people on the Codex boards complaining that WoW was markedly different from beta and they stomped their feet and pouted about playing.

Features like Mage Invisibility do not fundamentally change the vision of a game. Blizzard was set on the overall design of the game (which is the same now as it was then, and then was a better EQ1), and they care little for what people think of the overall design of the game, because they don't have to. And for the most part they don't really care about the random thoughts of forum trolls either, having said just about that much when reluctantly putting up their Suggestions Forums. In their view, this is why they pay professionals. This is not some focus-group candy-assisted circle-jerk. Not at these development budgets. If they were so dried up on ideas, they'd take another two or three years, shift more staff around, and import new thinking. Why take a chance on the average public that doesn't have a game design job?

As to testing itself, we're circling around the same stuff we did pages ago :-)

If the 'overall Vision(tm)' is simplified down to "better EQ" then i guess it can indeed be said the overall design wasn't changed. But in the meantime they had quite large shift from open world PvP idea to instanced CTF games, stronger focus being put on small group content rather than large scale raids, etc... all of these as result of 'beta feedback' that's being provided daily by their customers. So dunno, from very distant point of view it appears there was design changes to overall concept of WoW. It could be just as well wrong impression though given i don't follow the game too close.


But aren't their lead content designers basically not "game design profeshionals" but pair of outspoken forum hacks from EQ times? In this sense, they not only already took chance on the average public that doesn't have game design job, but keep doing it because hey, it actually somewhat worked out...

It's not as though they went from trolls to autonomous designers :-) A company that size, a group that size, they have a few layers of review and approval.

Otherwise, I get ya about what systems did change due to feedback. I just don't see them as fundamental as not having classes or xp or levels. Open world PVP or instances, it's still level-based with required PVE progression first. What you get when you get there doesn't change the process you used to arrive.

This gets to my main point very nicely.  Like anything else in this world, a good "fun" product takes time and money.  A truly unique and revolutionary idea can mitigate that to a degree, but that's a rare situation.  Obviously, the efficiency and experience of a design team matters as well. 

Overall though, i'm trying to get my head around what we're going to be seeing from MMO's post WoW.  How many game devs can commit the time/money of Blizzard?  Is that the new standard?  If so how can others reach that benchmark when they spent so much?

Clearly there is money to be made, so I'm wondering how this plays out.  If EA and Microsoft types throw their hats in the ring as they seem to be doing, do they box out everyone else?  Does everyone else disappear or do small outfits shift paradigm to smaller niche games? 

Maybe I'm being too dramatic.  I'm a MMO gamer since the original EQ and I never got into WoW although I do appreciate it's console-like level of non-buggyness  and streamlinification of the genre :oh_i_see: ... so I suppose there is still a market out there for other games.  I just dont think I'll get used to the constant comparisons to WoW when what they compare to spent 1/3rd of what Bliz did and had 1/2 the time.  Not sure what people expect.

My armchair general opinion is that anyone who tries to compete directly with WoW is commiting suicide. In order to beat Blizzard, you would have to be another Blizzard, with the track record, expertise and existing video game franchises to leverage.

Now, other games have shown that you can make a different type of MMOG and make a profit. (Eve Online is our big poster child there)

There is no "better WoW". More productive is to break off in your own direction with a game and work your specific angle.

I think we will still see a lot of WoW clones, because developers and their financial backers are glacially slow to realize important signposts in the genere.

There was concern expressed about it but it was rather tame relative to what I've seen in other betas. In any case, though, Blizzard was never going to go live with it anyhow. They were just experimenting and the -75% XP version was either purposefully or accidentally extreme, in that no matter how it was received they were set on their leveling curve -- it was just a matter of deciding on tweaking Rest and whether or not it needed an offset. They would of got there regardless of the forums just based on looking at leveling speed and downtime of testers as a statistical mass. They knew the pacing they wanted.

It's likely the closest example though I suppose since almost everything else save the Talent system revision was tweaking; either dungeon balance or removing dead weight (e.g. original Skill Point system). If anything they've considered feedback a lot more outside of its two betas and rather from their subscriber majority of what sort of content they want.

Which isn't to totally devalue the role they chose to hold beta testers to (something I especially wouldn't care to do) or say there's no communication. They're just very set in their direction, and mostly just open to tweaks to the systems which support it. Everything tangible I can cite anecdotal-ly has always stat tweaking; almost everything I recall from the eight months of forums was as well. Their RTS betas were the same way; there was a lot of uproar (much more than I ever saw in WoW's) about... something with items or heroes or taverns in TFT, I can't remember.. and Pardo basically told them to shove it and focus on tweaking how it works rather than demanding a redesign/removal of it.

Real beta with the public involved only creates even more bad press when you do make a change, so it is hardly surprising devs want to get it all locked down before the public arrive. But at the same time, there have been examples of genuine significant change during betas:

 - WAR's decision to go back and make RvR matter.
 - Mtgo3's decision to start again in order to make the UI completely unusable following Friends and family beta, then during public beta, to delay a second time in order to make the UI marginally usable.
 - I vaguely remember PS going through any number of changes, espeicially around MAX suit capabilities.

This bothers me, and has since I read it, but I just really thought about as to why it bothers me...

It tells me that the same old devs are making the same old games are using the same old testers/feedbackers that have been playing the same old games by the same old devs.

The circle of development is complete.

Crap goes in. 
Crap goes out.

It's more that you find an array of testers who will help you realize your vision than an array of designers to help you create one. At this phase, you're substantiating, not conceptualizing and don't really need a bunch of free-game-seeking neophytes to come along and armchair the whole thing for you.

The conceptualize bit happens in the beginning, when hopefully you've got good talent internalized (otherwise, why have them on staff)? Everyone who enters a late Alpha or Beta likes to think they're helping design a whole game. But they are not. This ain't no democratic process :-)

And they continue to design games where you need an organized group to participate in the Elder game, but somehow the word "Guild" is dirty.  People who actually run and lead guilds try to give them advice on design ideas so we can manage guilds more efficiently, and then help guild members so they don't quit the game.  No, we get told to shut up because think tank's know how to run guilds better than us.  Then said game launches without much thought to guilds other than a cloak, tag, and a chat channel. The game is guild unfriendly so few quality people want to even bother to lead a guild, many guilds die out because the leadership burns out months into the game, and the organized clan community is in a constant state of turmoil.

From a dev standpoint that may be a good thing because people aren't burning through their EPIC raid content. But from a player viewpoint the game sucks beyond a certain point because they need a clan to experience the content, and their choices are few or none.

No one ever really thinks about guilds as a critical community component to a game, and then they wonder why  their server pops decline until there's hardly anyone left (http://www.mmorpg.com/gamelist.cfm/gameId/9/setView/features/loadFeature/1653)

Oh well. That's why we don't have loyalty to companies anymore, why some guilds will never play future games developed by certain companies that consistently get it wrong, and why many guilds will simply jump ship as a full clan to a new game when they get tired of getting the shaft from their current one.

Yeah, what he said. I wasn't talking about design feedback, which should be happening up to and well past release, but simply "this is broken, here's how to duplicate" stuff. Few people WANT to do that sort of work (it is work) and fewer are good at it.

Considering the project I'm on now is my first design gig (I was a programmer on DAOC), I sort of take issue to the "same old people" thing!

I missed where you had actual positive design ideas. (I'm not being snarky - if you had some I missed them) Negative design ideas -- "You should not do this because it sucks! You shouldn't have a grind because grinds suck! You shouldn't have level differentations because levels suck!" -- only go so far. Board warrioring is almost always negative feedback because it's easy and fun (see: most of this board).

Take a specific example: an arcane system of slash commands for editing guild ranks that the server programmer added for debugging which never got improved, ever. This should probably be fixed!

Not helpful feedback: "Your guild system sucks and I can't run a guild. None of my guild will play your game and I hate you and hope your cat dies."

Helpful feedback: "Nobody can understand the slash commands used to control guild administration and they're not documented anywhere save a patch note from 2 months ago (link). What we need specifically more than anything else is a UI screen that lays out guild ranks and permissions. Given what the game supports, it needs to be able to designate which member can do X, Y or Z. It also needs to be able to support editing offline members (the current system only works with members who are online at the time which makes it impossible to demote problem children who know how to game the system)."

The not helpful feedback is funny, but I like my cat. Sorry.

The helpful feedback is good because:

* It's a punch list of action items. Developers love punch lists! It makes us feel like we're levelling up when fixing things.
* It points out specifically what problems are facing people and suggests a solution.
* The action items for the solution are major, but probably doable, assuming the interface is still editable and there's a UI guy with cycles free.
* It points out an exploit of the current system (the online/offline thing) which needs to be fixed before the game goes live.

Another thing to remember is that core design differences probably are never going to be solved via the testing process. If a game has levels and classes and you think level/class systems suck, no amount of design feedback is going to change that because it's a core system of the game which was architected two years ago in a design doc used to pitch the game to a publisher. You might have issues with the margins, such as "The XP curve in this game is way too punishing and there's no way to shepherd/sidekick new players", which could be actionable. You can say "I don't think this game is going to do well because level/class systems inherently are flawed" but your feedback at that point can be boiled down to "you know, this really isn't the game for me. Sorry."

My 2 cents is that at the head of these million dollar projects are the completely wrong type of people. From my understanding, you progress through the ranks over time from usually an artist or programmer, to head honcho. These people dream real hard about creating their own MMO's but in reality they have no business running any kind of business. Especially one with millions of dollars and stake and scores of people to look after.

Sorry Lum I was not neccessarily referring to our discussion in this thread. I should have just stated that that's been an observation of mine over several years. Obviously this forum and this discussion is out of context, but I have seen some very good constructive feedback in my time where people have tried to ask for games to make guilds easier to manage, be able to develop some guild based incentives, etc.  And the basic response was that they usually just ended up with a clan interface, guild chat, a cape, and a tag.

General board warrioring is counter productive I agree.

It is interesting you are talking about guilds needing more attention in MMORPGs. I personally think that guild problems are almost always the trigger issue that causes players to leave a game. They may never say it was because of guild problems they quit, but players will put up with the issues that they say are the reason they are leaving, until their guild gives them a reason. If MMO's can cater to guilds, by making the games built around helping them stay together and work smoother, then there will be significantly higher retention.

A lot of guild drama comes when guilds evolve from leveling and growth to splintering between tier 1 endgamers, really-endgame endgamers, and everyone else. Even that structure can work (with some hierarchy and a thinking leadership). But things really begin to fall apart when most of the guild is blocked from some content (and the really-endgamers splinter off), or new content isn't coming fast enough.

A well-designed intuitive guild management UI and NPC guards that salute as you walk by isn't going to solve those problems :-)


Given the amount of "your [thing] [adjective] [biological part] [species]" that comes through even the bug/request feedback system (be it some Jira/Bugzilla type thing or just forums), how easy is it to spot the actual Helpful Feedback?

Said feedback is great, but the number of times I've seen such feedback 1) never acknowledged by the devs, 2) refuted / argued over / picked apart / derailed by other forum posters and / or 3) repeated several times written in several different ways... well, I don't know the exact number, but it's a lot.  :awesome_for_real:

Point 1 is actually the most important. If a dev acknowledges the feedback as what they want and that it is helpful, then players will follow that example. Ignoring it or reading it silently doesn't actually make the person who gave the feedback (or the people who agree with them) feel like they've made any sort of meaningful contribution, so they stop.

For betas moving forward, every single department should have its own beta forums where players can post their feedback (so: Gameplay, Art / Graphics, Sound, Networking, etc). Each department should have someone - NOT A CM WHO CAN'T DO ANYTHING ABOUT THE ISSUE AT HAND - post a reply to feedback to something they find useful. (On CMs - a good one could do such a job, especially if a lot of feedback is coming in, but they will filter out some of the good stuff too, so it's better if someone who actually works on the problem gets to cast their eye over the posts too.)

Reward good testers with public recognition ("Hey, that was great feedback and we're going to consider it") and they will love you and keep testing for you (until you screw up a design decision that nerfs their favourite class, but dem's the breaks).

A side point - every MMO should have a dev digest, where you can easily see every single post made recently by a dev. It makes dev stalking easier but also means dev comments get widely read since they are centralised.

Then why bother having CM's? That's THEIR JOB. Newsflash: not only can Joe Coder not "do anything about the issue at hand" (because he doesn't get to just randomly pick what he works on based on message board traffic), the time he's spent reading 50 threads on why Panzerelf Stormtroopers are clearly overpowered is time he's not actually... you know... working on stuff.

Some coders can work boards to get and give good feedback. Many can't. It's not fair to make that part of their job anyway because you want to feel like Someone Really Important signed off on your post.

As for actually acknowledging bugs, I'd like to see real bug tracking systems used by beta/test server users but I'm not sure it's possible. Second Life uses JIRA tracking on their stuff and it gets hijacked constantly by board warriors who glory in their ignorance of how software development works (http://secondthoughts.typepad.com/second_thoughts/2007/11/storm-the-jira.html).

I never thought of bug databases as totalitarian tools for world domination...

The biggest problem I see with the SL database is featureitis; Everyone has a pet feature that they'd like, but add enough and the UI soon becomes very complicated. One request I saw was that screendumps should have the options of a transparent sky so that photoediting would be easier. Yes, but the screendump UI suddenly becomes much more complex (and likely to have bugs).

If it's their job, then why does it often appear that they do nothing at all, or that issues that tend to have broad-ish forum community support run on without comment for a long time? I did say that a good CM can do that job, but it isn't optimal if Coder Joe doesn't pay attention to how the changes he is coding in are being received, or if reasons for the changes are never communicated back to players.

MMOs are service industries, and while I think Coder Joe should spend most of his time coding and CM Mike should spend his time community managing, it isn't a good thing if Coder Joe works in a vacuum without ever realising what impact his changes have on how the paying customers play (or that CM Mike has no idea what Coder Joe is working on). Better Joe reads 50 threads on Stormtroopers and then helps formulate a reasonable solution than cook up a fix without ever bothering to find out what players want, how they currently behave and maybe how well they'll receive that fix.

However, if it makes a stronger point, replace the NOT A CM bit with "someone who is able to start the ball rolling on a fix and to post feedback". Too often I've seen CMs as company mouthpieces who's only authority is to delete offensive forum posts - if it is their job to pass on important information back up the chain (which yes, should be part of their job) then they certainly never comment on it.

SL feature feedback is an example of conch-shell representation. But that's not the fault of the bug submission system itself. It was how it was configured.

CMs work all the time, but the numbers never add up. Even 1% of WoW's playerbase spamming the boards is a lot more noise than a budget-justified CM team can handle. But they do their best, elevate what issues seem big, talk about it at the next roundtable/staff-meeting and then the teams that can do something about it a) consider whether they should; and, b) schedule it into their development cycle. CMs may appear to be doing nothing when in reality they are merely the conduit between players who want immediate feedback and developers who can't possibly deliver it.

I like the idea of Correspondent programs. General flame-fest clique forums reporting into competent volunteer lateral thinkers reporting into CMs reporting into devs. Sorry if every little pet idea doesn't get the personal atta-boy from the developer, but try getting anything approaching even what we have now from any other service industry without first approaching your local Attorney General or the Better Business Bureau :-) Devs don't report to us. They simply provide a service to us.

We don't have it great, but I often find CM complaints to be a case of people getting more wanting more.

Yeah I think DAOC had Team Leads or something. Shadowbane had its Advocates. I think the advocate system worked well for funneling up issues, and every quarter I would ask them to poll their specific forums to see what the top 5 issues were.  Only problem we had was when the player base dwindled due to guilds quitting after their cities were destroyed, and it made it harder to find quality people.  But otherwise we had a budget for advocate leads, and free game time for class advocates that worked rather well for a long time.

I'm not really sure what the other games are doing.

That's just Prok. She's.... special, to the point of getting SL-wide recognition of sorts.

I'm going to field this one. I'll start with a disclaimer: I work as a CM but for a publisher not a developer so my experience will be different to someone who gets to have regularly scheduled meetings with the guys working at the coalface.

That said, during beta I'm very busy. I'm spinning a lot of plates simultaneously. I have about 50 or so forums I read regularly for the purposes of doing my job. That takes a long time even if you're just skimming thread titles for stuff that you think you should read. While I'm reading those forums I'm gathering feedback. The guys I have to pass that feedback onto don't have time to read 50 forums a week which is why they pay me to do it for them. Feedback by itself is meaningless without context - who are the people giving this feedback? What sort of background do they come from? What agendas might they represent? All of this has to be collated and summed up in a format that a bunch of people who are frantically busy can digest quickly while also answering all the questions they're likely to have about it.

Then there are the beta testers. Even though the beta forum is just one of the forums I read in my working day, it's probably the most important. These are the guys that are actually playing the game, the fans on the messageboards are speculating and responding to marketing drops, these guys have sharp-end experience but again it needs to be filtered and adjusted for bias. These guys also need to be nurtured carefully, like a plant you can kill it with over attention. I read the posts they make, but I try not to get involved in the discussions except to keep things on topic or to shut down trouble. The reason for this is because I want the testers to tell me what the game is like, not to react to my interpretation of what the game is like or what I think would make the game better. I have my own ideas and they go in a separate part of the forum. From time to time I'll have to remind the testers that I do indeed read every post that they make, I am passing on their feedback to the devs and no, they aren't ever going to see one of those feedback documents. That takes a great deal of time.

On top of all of that, I'm also doing the PR fun stuff to the outside world, I'm the guy during beta telling the fans how great the game is and how much fun we're all having in beta. On live release I'll be the guy doing fire control on drahmaz and explaining to people who aren't playing the game yet, precisely why they should. Note: At this point I'm still doing all the other stuff too.

Finally I have a lot of other hats I need to put on from time to time. One of our stipulations with regard to official communications is that it must be proofread by a native speaker of that language (we operate in five languages). As I also need to know the content of all our official communications, a handy way to kill two birds with one stone is to have me and my colleagues proofread the relevant release for each language. As a lot of the stuff I get is translated from a French original by non-native English translators, it often needs quite a bit of editing. I actually need to play the game I'm fronting for as well at some point and I need to stay current with the stuff that's coming from the developers.

If you made a sparkling observation of beautiful truth and I didn't acknowledge it specifically, I hereby formally apologise. It will I'm afraid, happen again.

 I'm just wondering as a tool to a CM do developers data mine across their forums, feedback, bug database to give a large picture?

 All the beta's I have taken part in was one giant vacuum where /bugs were being reported, threads on beta forums for an issue or feature going no where with inherent bickering that really puts beta tester(s) in a negative feedback loop. I always wondered why its an issue connecting /bug or /feedback to an actual tracking system that has some transparency to a tester they can interact with. It's really terrible to have a ticket closed saying its known issue when you already took the time to search their beta boards but have no other way to check to see if your wasting time(the company and yours).

 Perhaps I'm just still jaded over Lum's GameCo example.

No game company is going to open it's actual bug database to the public.  We cannot allow players to see all of that information, if only to deter exploitation of bugs.  Yes, ideally, we're going to fix all of those exploitable bugs, but here in the real world where developers like to see their families on occasion and overtime laws are, occasionally, enforced, we can't fix them all right away.

On top of that, those bug databases contain actual trade secrets.  It's not like we're developing the cure for cancer here, but if I had access to Blizzard's bug database the value to my company would be very high.  Like hundreds of man hours high.  No company with the primary goal of making money (hint: all of them) is going to publish that kind of information to the general public.  And please don't try and tell me that a beta community isn't the general public, or sell me on NDAs somehow making this a non-issue.  If you really believe that I have some 'beach front' real estate that I'd like to sell you.

Creating a separate database that tracks player created bugs and known issues is a lot of work.  It might be worth it, certainly it would be nice, but what other features is it stacked against?  Web-based guild management tools?  Web-based blogs for characters?  A WoW-style armory?  No developer can afford to implement every feature that 'would be nice,' and ultimately you have to ask yourself what is going to attract/retain more players.

Yeah, I also read the feedback forms, talk to beta testers ingame (as a regular player, not with an 'official' character) and imcorporate all of that too. Our QA department are the ones who tend to track the bug reports and they field those. I skim them to see if there are deeper issues tied up within but I don't feedback on bugs directly.

A key issue I see coming through here is about CM communication with the fanbase. While I think it sounds like you've got too many plates to spin anyway, a lot of what happens on beta forums can be resolved a lot more quickly if CMs / devs actually put their stamp on a thread. Fuser is right - a lot of beta forum posts can get into negative feedback loops or fanboi / h8ter pissing contests. Some positive reinforcement from official channels can easily prevent this (so will the threat of a big stick, so whatever works).

I don't think being a CM is an easy job, but any means. Players are really only just starting to learn what CMs do on a day-to-day basis. However, I do think that communications with MMO players - especially during beta - will likely improve with more transparency and more indication of action. Posting "I've read this and will forward it to the relevant person" means a lot more than dead silence in a thread.