F13 Editorial Scorecard?

[jpark]

If this has been discussed before – no problem if locked/ removed.

Proposal:  A single page ranking by category scores for each MMORPG providing quick reference for those new to F13 who want to find out quickly what the site’s views are on these games.

F13 Gap?   Currently, if one is new to the site or does not follow it regularly, to find out f13's views on a game you can read the review or follow the threads.  This is time consuming and site information is not clearly benchmarked in a systematic way against other games.

Benefit?  This provides quick reference to those new to the site who may cite f13's views for marketing purposes.  It also makes discussion in the f13 forums more focused – so we can see in a single table the conclusion admin/moderators have drawn about the games for further discussion. 

Updates  As user experience with newly released games grows or as older games may undergo major updates, this table or summary page would be updated quarterly to reflect a change in F13’s editorial view of the MMORPG competitive landscape.  This makes the reviews clearly time sensitive, so those that cite F13 for comments have incentive to review editorial views here at least quarterly.

Editorial Process.  Pretty much what we do now:  a game comes out, a review is posted and then we argue until consensus is largely reached on some major points.  Because this table is updated quarterly, this allows the ensuing discussion in the interim to be weighed by the editorial powers of F13 to see if registrants (Foozles et al) have changed their mind on any particular point or score.

Metrics.  Because the summary page is focused on the MMORPG market (eq, eq2, wow, coh etc.) this makes deciding on clear categories a bit easier than “generic’ games.  You guys can decide the categories (combat, social, economy, immersion etc.) but I suggest discipline:  each category should have an average score of 5.5 (or whatever).  If you give a competing product a high graphical score, other products should have lower scores so the average of all product scores is 5.5.  This forces us to clearly benchmark games against each other in light of what is "average" in the market currently.

[Alkiera]

That would, of course, require the admins/mods/whatevers to actually agree...  Which they don't do all that often, especially when it comes to MMOGs.  Witness WoW, where many consider it to be the MMO Jesus would play, if hewasn't already busy... and others can't figure out why ANYONE is playing it.

I was gonna make a similar statement about EQ2, then realized not only are those who are positive toward EQ2 here countable on one hand, I may very well be the resident(and lone) fanboi.  Cest la vie.

Alkiera

[jpark]

That would, of course, require the admins/mods/whatevers to actually agree...  Which they don't do all that often, especially when it comes to MMOGs.  Witness WoW, where many consider it to be the MMO Jesus would play, if hewasn't already busy... and others can't figure out why ANYONE is playing it.

I was gonna make a similar statement about EQ2, then realized not only are those who are positive toward EQ2 here countable on one hand, I may very well be the resident(and lone) fanboi.  Cest la vie.

Alkiera

True enough - but given this site has official reviews it releases on games already - I may be wrong but that loosely implies some sort of consensus to me.

Again it also focuses our discussion.  In ranking the games, we may be clearer by category why we praise or hold in disdain a given game.  For example under "Graphics" EQ2 would rank high on polygon/ technical/ resolution over WoW, while the reverse might be true for "Artistic merit".  In comparing EQ to EQ2 if there is a category for crafting, there may be a consensus that the game has clearly advanced in this particular advancement tract.

In the end - among the moderators /admin, they can decide democratically on the relative rankings.  The rest of us can lobby like mad to change their views :)

[kidder]

It's a nifty idea.  I don't know if a concensus is really needed for a review or not.  Most of the reviews on the homepage are opinion pieces really.  It would be interesting to see an f13 review of a game cited on www.gamerankings.com.  You would have to have some kind of scoring system though, which jpark's idea provides.

I like it.

[Joe]

I was gonna make a similar statement about EQ2, then realized not only are those who are positive toward EQ2 here countable on one hand, I may very well be the resident(and lone) fanboi.  Cest la vie.

Alkiera

Sup, EQ2 buddy?

[eldaec]

That would, of course, require the admins/mods/whatevers to actually agree...  Which they don't do all that often, especially when it comes to MMOGs. 

And not only do most people here disagree with each other, most posters seem to disagree with their own future/past selves. Espeicially re: MMOGs.

[schild]

Be fair, when I'm wrong (ie: CoH), I completely admit it.

That said, I'm not wrong about the DS or Blizzards inability to run an MMOG.

[Velorath]

It's hard to give an MMORPG a fixed score since things like patches, customer service, and server stability can all change things for better or worse (usually worse).  You put up fixed scores and you'll have the fanboi's whining about how you didn't take the Jedi revamp into account, or the fact that paratroopers are being patched in, or that Mythic has promised that player owned mounts and gemcrafting are coming and that shit class you play is being looked at for improvements.  To Hell with scorecards, visitor's can do a search for SWG on the forum and find hundreds of posts going into exacting detail of what's wrong with the game.

[jpark]

It's hard to give an MMORPG a fixed score since things like patches, customer service, and server stability can all change things for better or worse (usually worse). 

That's the opportunity - not a problem.  If you could just post a score and be done with it - there would be no incentive for outside agencies to keep tabs on this site for its views.

Because MMORPGS change over time, f13 scores have to be reviewed / updated regularly.  That keeps "content" on this site fresh / current.  That means those that cite scores from this website in support of their product have to update and keep in touch with this community.

The owners of this site have to decide whether f13 represents a "voice" on products - or just a place we get together and talk.  And rehash.  Forget.  And rehash again.

[schild]

Honestly, as much as we don't like it, some things deserve fixed scores.

MMORPGs do not. Something akin to the editorial scorecard has been in the works for a while, and when it comes out, people may or may not care. Which is fine .

[Velorath]

That's the opportunity - not a problem.  If you could just post a score and be done with it - there would be no incentive for outside agencies to keep tabs on this site for its views.

Because MMORPGS change over time, f13 scores have to be reviewed / updated regularly.  That keeps "content" on this site fresh / current.  That means those that cite scores from this website in support of their product have to update and keep in touch with this community.

The owners of this site have to decide whether f13 represents a "voice" on products - or just a place we get together and talk.  And rehash.  Forget.  And rehash again.

In order for it to be even remotely practical the reviewers would have to be given free trials and characters of appropriate level to experience whatever changes have been patched into the game in many cases.  If Lineage II drastically changes their high-end PVP content for instance, who's going to subscribe and grind through all those levels to update the scorecard?  You'll just end up with crap games getting completely ignored and the games people like and are already playing will be the only ones that get updated, which will probably lead to crying about being biased and that the reviewers are sell-outs.  Well, they'll be saying that more than they already are anyway.

[Rasix]

Be fair, when I'm wrong (ie: CoH), I completely admit it.

That said, I'm not wrong about the DS or Blizzards inability to run an MMOG.

You're not completely wrong, you just aproach the manner in way that you're not going to ever get anyone to agree with you.  Except Bruce. 

[Murgos]

The only way I could see to have realistic sort of rating system for mmogs here would be to add a "What are you playing page" to the profile system and then just show a page somewhere off the front page that lists games by percent of users that play.

The points of failure would be someone has to add new MMO's when they come out and the players maintaining accurate lists.  The players lists aren't too important because the frequecy its updated will be proportional to the number of people playing so you will still get some idea of how well liked a game is even if people hardly ever update thier profile.  The system could be abused by someone registering noobler accounts and parking them in the game they want to look good, so maybe a minimum post count should be required for participation.

It's not a this game is X% system but it will definately show an overall 'site' feeling about a specific game.

For the user portion you would have two pages of code, one containing a form that self submits (for the profiles) and two db tables, one a list of games two columns - a key (mmo id) and game_name, one a list of vote total by mmo id key thats auto-generated once every few hours.  The next part would be a bit tricky, I guess you would need the admin interface to modify a table with a new column for each mmo thats added.  Initially populate it with all existing users and all column values set to 0 then as they vote just update thier row on the table.  This method would also mean that account creation (or approval would have to insert a new user id into the vote tracking table.  This table could get pretty huge so maybe you could break it up into a series of tables named by mmo with user id as primary key and a column of votes.  Actually if you assume votes to be 0 or null then you just need one column for users that voted positive and just insert thier userid into the table when they say they are playing and delete them when they say they quit.  The SQL statement to join the tables would be interesting but not too bad just create it off your master list of MMO names.

I've got enough to do, so don't ask me to build it, was just tossing around ideas.

[jpark]

It's not a this game is X% system but it will definately show an overall 'site' feeling about a specific game.

Speaking for myself - I could see this as an added site feature down the road.

But if the goal of F13 owners is to be heard and build the website's reputation further - you need to be citable / quotable.  To do that you need to conform to standard outputs that outside agencies (press, vendors etc) are used to seeing.  That gets back to some kind of score card - a rating system.

If the goal is to build the reach and scholarly level of discussion of F13 it has to add other elements beyond the current format of discussion.  From pundit to analyst.  From thread after thread of discussion - to resource pages that people in this industry would appreciate from an impartial source (e.g. a single web page indicating MMORPG market share?).

[Murgos]

If the goal is to build the reach and scholarly level of discussion of F13 it has to add other elements beyond the current format of discussion.  From pundit to analyst.  From thread after thread of discussion - to resource pages that people in this industry would appreciate from an impartial source (e.g. a single web page indicating MMORPG market share?).

You didn't just advocate making the canadian an integral part of f13, did you?

[SirBruce]

I'm not Canadian, dammit!

Bruce

[jpark]

If the goal is to build the reach and scholarly level of discussion of F13 it has to add other elements beyond the current format of discussion.  From pundit to analyst.  From thread after thread of discussion - to resource pages that people in this industry would appreciate from an impartial source (e.g. a single web page indicating MMORPG market share?).

You didn't just advocate making the canadian an integral part of f13, did you?

I am "advocating" offering resource information on this site pertaining to MMORPGS (or games in general) to increase its relevance and help focus discussion.

If the owners of this site see merit in that, discussion can shift to what kind of information / analysis can be presented - and who affiliated with F13 is willing to spend the time doing it.

[HaemishM]

As schild says, there is something like this in the works.

Frankly, I think we should a database of MMOG's and each mod should rate the MMOG on a 1-5 scale. 1 middle finger means it's bearable and probably plays like Robot Jesus to the non-jaded. 5 middle fingers for Horizons or Lineage 2 quality. 3 Tears down would represent the amount of crying we've done over the wasted potential of games like Shadowbane.

[jpark]

As schild says, there is something like this in the works.

Frankly, I think we should a database of MMOG's and each mod should rate the MMOG on a 1-5 scale. 1 middle finger means it's bearable and probably plays like Robot Jesus to the non-jaded. 5 middle fingers for Horizons or Lineage 2 quality. 3 Tears down would represent the amount of crying we've done over the wasted potential of games like Shadowbane.

Yikes.

It has a style to it I grant you that.  Problem is that nobody in mainstream media can "cite" your rating system like that given the offensive qualities of using "middle fingers".  If you guys use an iconically vulgar rating system you risk keeping references to your site circumspect.  Could a magazine article or developer publically refer to your rating system to illustrate a point using your score to back up his point?

It's nifty.  But you might consider other stylish means to express yourselves that are incisive but do not preclude reference in formal settings.  

For example:

Rearranging deck chairs on the Titanic?

Come on, it's hardly the Titanic. 

Now the Andrea Dorea, I can see.  A cruise ship struck by a larger vessel that gradually sinks into oblivion.  Perfect analogy.

ETA:  As long as we're on ship metaphors:

WWII Online = Titanic (Sunk on maiden voyage)
Ultima Online = USS Constitution "Old Ironsides" (Antiquated and obsolete, but achieved huge upsetting victories in the past)
Everquest = Queen Elizabeth II (A stately craft, the benchmark for cruise ships)
World of Warcraft = USS Kittyhawk (Leaves a huge wake, targets and destroys lesser ships)
City of Heroes = A Nitro burning cigarette boat (Small, but efficient and focused on what it does)
Star Wars Galaxies = The Achille Lauro (An grand vessel, until everything went horribly, horribly wrong)

You have great metaphors.  This could almost be a ranking system.

Some additional ship metaphors:

Lineage II:  Roman slave galley.
Players toil for months and months for some limited pvp conflict

Wish:  USS Indianapolis
Was torpedoed in the Pacific, but nobody noticed.

I bow to you, sir.  That there is good stuff.

This is not exactly a rating system because it is not a continuous variable.  But it certainly is "sharp".  And could be cited without difficulty.  Middle fingers?  I think you guys can do better.  Your site is counter cultural because of your views and insights.  I challenge you guys to come with icons that are "sharp" but do not rely on the simple tactic of being vulgar to convey your spirited views.

[Samwise]

How about cat turds?  Or poopsocks?

Wait, were we going for less vulgar?

[Shockeye]

Problem is that nobody in mainstream media can "cite" your rating system like that given the offensive qualities of using "middle fingers".

I'm a bit hazy on the reason we'd want to be quoted by mainstream media.

[schild]

I thank you all for this thread. I have come up with a plethora of possible icons. We'll start spitballing this week. See what sticks.

[jpark]

No point in creating another thread so I'll move on to a derail:

This is a place for discussion - have you guys considered making this a site a "resource" as well?  Resources that different professionals and industry segments involved with computer gaming would find useful?

I am not in this industry - but things that come to mind are:

- Links with descriptions of major venture capital firms (or those that invest private equity) involved with gaming company start-ups?  (noting their asset base - e.g. how big they are)
- A diagram depicting MMORPG market share (and forecasts?...)
- A page indicating subscription and revenues by firm / product (when feasible)
- Listing firms typically used for legal advice in this industry (contract law, patents, trademarks)
- Key consulting groups used in this industry - and their basic service (e.g. business plans, competitor analysis etc.)
- A page listing upcoming conference events in different specialties within the gaming industry (graphics, game design, sound, investor roadshows etc.)
- any other service that start-up gaming companies typically must outsource

Some of this may sound like free advertising.  While this content is publically available - it is centralized - on this site - for developers and entrepeneurs to peruse while participating in the forums.  Having this kind of information will inevitably bring search engine hits from others doing market research for firms in this industry.  It helps add to the confluence of activity here of all those interested in where this industry is going and sharing some commentary in the process.

How about cat turds?  Or poopsocks?

Wait, were we going for less vulgar?

People like you, cause unrest :)

[HaemishM]

It was a joke. Sort of. The only reason I can think of we'd want to be referenced by the "mainstream" media that I hate so much is for more advertising revenue. I would take great pleasure in the irony of being mentioned in PC Gamer, a magazine I wish would die in a staple fire.

- Links with descriptions of major venture capital firms (or those that invest private equity) involved with gaming company start-ups?  (noting their asset base - e.g. how big they are)

Does anyone not involved with VC know who these are? Or anyone not looking to get funded? Most of these people don't exactly advertise who they are.

- A diagram depicting MMORPG market share (and forecasts?...)
- A page indicating subscription and revenues by firm / product (when feasible)

Great idea. Except that nobody is giving this information out to anyone that doesn't pay them for it, unless their numbers are bigger than the other guy's numbers. Someone else already tries to do this, badly, with incomplete and unverifiable data. Why should we massage this industry's incestuous circle jerk with numbers no one will give us anyway?

- Listing firms typically used for legal advice in this industry (contract law, patents, trademarks)

See VC firms above. Generally, you have to be in the industry or be invested in the industry to get this information. As for patents and trademarks, the US Patent and Trademark office already provides most of the information you need on this on their own web sites.

- Key consulting groups used in this industry - and their basic service (e.g. business plans, competitor analysis etc.)

I can think of exactly 3. Themis Group, Savant Says Media and NPD. They all have web sites. I'm not going to link to them unless they pay me, or place media through us. NPD doesn't need us, because they don't just focus on games, and charge people high dollar figures to talk about what they think. 

- A page listing upcoming conference events in different specialties within the gaming industry (graphics, game design, sound, investor roadshows etc.)

That's not a bad idea.

- any other service that start-up gaming companies typically must outsource

Also not a bad idea, if we knew of any.

Some of those are good ideas, and I welcome your comments on what we can do with f13. I'd rather think that industry folks will come here because we don't bullshit, coddle or otherwise massage their egos. Providing them an information resource may be a good idea, or it may be just work we do for nothing. The biggest hurdle to that is we just don't know most of what you listed above, and aren't likely to know unless someone comes and tells us.

[SirBruce]

Does anyone not involved with VC know who these are? Or anyone not looking to get funded? Most of these people don't exactly advertise who they are.

Well, there are tons of VC firms, but actually only a few that are seriously involved in game industry investing.  I know some of the major players; you can always do google searches to see the recent investments made in companies like Mythic, Turbine, Perpetual, etc. to see who is behind them.

Bruce

[jpark]

The biggest hurdle to that is we just don't know most of what you listed above, and aren't likely to know unless someone comes and tells us.

Ah well I have learned something right there.  As Bruce mentions - the VC information is available - it just takes a bit of digging (Angel investors though - that would be different - and would not be appropriate to post).

I see two steps to this.  First, you guys (admin, mods etc.) have to decide if you would like to offer "resources".  If so, the second step is gathering it.  Gathering the information does not have to be done overnight - it can be collected over a period of months through forum input.  It actually can provide some clear "goals" to discussion.

- A diagram depicting MMORPG market share (and forecasts?...)
- A page indicating subscription and revenues by firm / product (when feasible)

Great idea. Except that nobody is giving this information out to anyone that doesn't pay them for it, unless their numbers are bigger than the other guy's numbers. Someone else already tries to do this, badly, with incomplete and unverifiable data. Why should we massage this industry's incestuous circle jerk with numbers no one will give us anyway?

The MMORPG market share and sub rate info - the obvious resource at f13 is Bruce - unless someone else is willing to step up to the plate.  Your group (admin, mods) may consider giving Bruce (or anyone else) a chance to present their analysis.  It does not have to be "right" but does have to be presented in a sound matter.  If Bruce is interested in presenting his analysis to you - and if you guys are comfortable with it - I think you should post it.  If someone else has a different analysis on this site - then choose the better one of the two etc.  Anticipating differing views on analysis here, you can have a disclaimer that the said analysis does not represent the views of f13, but is provided as a resource.

On the VC side, I could make some calls & do some research.  For now I cannot commit the intention - but at least I can keep my ears open.  I find it hard to believe one could not eventually track down VC's in your industry (as I track them in my own industry).  Let me know if this is information, if gathered, you would offer on your site.

If you know the "gaps" in the immediate knowledge base at F13, this again provides some structure in which to get some focused feedback from your posters that can be harnessed for constructive purposes.

Again, all this is a suggestion :)

[CmdrSlack]

Quote- Listing firms typically used for legal advice in this industry (contract law, patents, trademarks)

See VC firms above. Generally, you have to be in the industry or be invested in the industry to get this information. As for patents and trademarks, the US Patent and Trademark office already provides most of the information you need on this on their own web sites.
********************

Well, technically, it wouldn't be hard to compile that list.  www.martindale.com could probably be useful, and I've found plenty of IT law firms simply by reading Terra Nova and whatnot.

But I don't really know if you'd want to be a referrer to lawyers.  Not like any read the site.   :-D


EDIT--because figuring out quoting on new forum software is hard.

[HaemishM]

The MMORPG market share and sub rate info - the obvious resource at f13 is Bruce - unless someone else is willing to step up to the plate.  Your group (admin, mods) may consider giving Bruce (or anyone else) a chance to present their analysis.  It does not have to be "right" but does have to be presented in a sound matter.  If Bruce is interested in presenting his analysis to you - and if you guys are comfortable with it - I think you should post it.  If someone else has a different analysis on this site - then choose the better one of the two etc.  Anticipating differing views on analysis here, you can have a disclaimer that the said analysis does not represent the views of f13, but is provided as a resource.

Let me just be the first to say, "No" and "Let's not start that argument again." Bruce and I personally differ as to the veracity of his information is about the nicest way I'm going to say that.

[Margalis]

I think subscriber numbers are not a meaningful resource. If anything they can encourage bandwagon jumping. They may be interesting for analysts or those interested in the business, but they are irrelevant to the players.

What matters is am I have fun, not how many other people bought box Y instead.

I would also stay away from scores and reviews, unless it is going to reflect some diverse opinion. For example, an entire page of one-paragraph reviews. There is no single MMORPG gamer and one review per game would be silly.

"WOW is great" isn't helpful to anyone. What is helpful is knowing that it's more of a game than a world. (For example)

Say what the game does well and does not and let the reader decide what the means. Is lack of crafting a bad thing? Really depends on whether I give a shit about crafting or not.

"We report, you decide!"

[kidder]

I would also stay away from scores and reviews, unless it is going to reflect some diverse opinion. For example, an entire page of one-paragraph reviews. There is no single MMORPG gamer and one review per game would be silly.

"WOW is great" isn't helpful to anyone. What is helpful is knowing that it's more of a game than a world. (For example)

Say what the game does well and does not and let the reader decide what the means. Is lack of crafting a bad thing? Really depends on whether I give a shit about crafting or not.

"We report, you decide!"

I'm of the belief that there is no such thing as a "reporter", anywhere.  I cannot remember the last time I saw an unbiased report on ANYTHING.  Game reviews are opinion pieces, period. 

I find the reviews on this board by Haemish helpful, because I like many of the same things he does about games.  I don't agree with everything, but if I haven't tried a game or am looking for a new game to play, I take his thoughts into account.  Same with some of the other reviews I've read here, or the threads where people ask for tips about what games to buy. 

When I go looking for a title, let's say for example for the xbox, on gamerankings.com, I first look at the overall score then I scroll down and see what companies have reviewed the game.  I'll then follow the links to see what my favorite sites had to say about the game.  I still think it would be awesome to have f13 linked in that list of reviewers.

If the f13 power-that-be don't want to get involved with the "mainstream", that's fine too.  This is my favorite haunt, and after the forum software change it is bursting with more information than I have time to read...although I am trying.

I do see a ton of latent potential for this site, it's pretty damn good already, but it could be more.

[Train Wreck]

Frankly, I think we should a database of MMOG's and each mod should rate the MMOG on a 1-5 scale. 1 middle finger means it's bearable and probably plays like Robot Jesus to the non-jaded. 5 middle fingers for Horizons or Lineage 2 quality. 3 Tears down would represent the amount of crying we've done over the wasted potential of games like Shadowbane.

I like this idea.  It reminds me of Mr. Cranky's bombs.