f13.net

Alright. I'd like everyone to give some feedback on this to make it the best possible system for reviewing a game. It's going to get rather complicated, I'm sure, but here's the skinny at the moment:

It’s simple, once you catch what’s going on. Here’s the breakdown.

Multiplier | Category
3x | Story
2x | Graphics
2x | Sound
3x | Gameplay


Now, if you haven’t figured it out, that adds up to 10. So if we were to review God of War, I’d give it:

Score | Category
85 | Story
90 | Graphics
80 | Sound
100 | Gameplay

Where’d I get those scores? We’ll have to decide on a basis for the documentation, but it would look something like this:

Story
0 = Save the Princess / Save the World / No Story – If it’s non applicable, add a 1x multiplier to Graphics, Sound and Gameplay (bringing them up to 3x, 2x, and 4x respectively). This applies to games like Contra, Life Force, or fighters or puzzlers that are all about the gaming. Every MMOG Ever (Gameplay is also a 0 for every MMOG ever, with the exception of City of Heroes which should get a 25 for at least trying and Planetside for being a vain outlier at 50)

50 =  Half-Life 1 (a subgenre of save the world), Final Fantasy II (save the world with subplots among many characters), basically anything with a subgenre that is remotely interesting or a game that’s so wacky the story compels you to continue (Katamari Damacy, etc)

100 = Planescape: Torment, Deus Ex, System Shock 2

Alright, so we’ll need to do that for each category. Anyway, here’s how the review would go:

Story: 85, 85, 85
Graphics: 90, 90
Sound: 80, 80
Gameplay: 100, 100, 100


(85 + 85 + 85 + 90 + 90 + 80 + 80 + 100 + 100 + 100) = 895(/10) = 89.5

For each of the scores, they would be added to a graph spanning the breadth of the category, from 0 to 100. As such you'd have (in an easily referenced graphical format:

Story:
Super Mario Bros 1 (0) ------------------------------Half-Life 1(50)------------God of War (85)-------------Deus Ex(100)

Games will continue to get added as we score them.

Now at the tail end of overall scores, I have some modifiers that I think are appropriate. (Seperating stability out because it's scored differently from other modifiers)

1. Stability - Give it a score between 0 and 100. Take that score, divide it by 10 and subtract the amount between the score and 10 from the final score. God of War got a 100 in my example (mostly because console games on a whole are perfectly stable, but this one is particularly stable with seamless cutscenes etc). So 100/10 = 10. So you'd subtract nothing from the final score. If it get a 90, the result would be 9 so (10-9)=1. So the final score for the game would be lowered to 88.5. This, however, is not the final score.

More modifiers
Challenge
Control/UI (Control for consoles and UI for PC)
Depth (an extension on story).

I'm thinking we should use a -5 to +5 scale. In other words, right off the bat any one of these can add or subtract points to the final total. For example, I'd give -1 on Bloodlines because melee was wretched. This is a completely opinion based addition. And should be based on "normal difficulty" on any game with options as such. Another example, I'd give God of War -1 for how terrible the final boss fight was under the "challenge" category. It was boring and easy. But +2 on the rest of the game, as such, it would be a +1, but we would be covering all the bases. In other words, they stack. (Which would bring the score back up to 89.5, but all of this would be noted in the review, as such letting everyone know EXACTLY how we came to the final score and what true negatives there were in a game).

The gods of f13.net demand feedback. Appease us!

Being and accountant/Excel-god, if you want a template of it done, I can make it happen and then you can post it on the site. Just FYI.

You just saved Shockeye from busy work.

DING! GRATS!

Level 2!

Well joy, I can go back to doing.. nothing.

I am f13.net's kept man. I am only here for my looks.

Shit, I wanna dinggratz, too! You guys need a theme song or something?

 :mrgreen:

I for one would welcome a F13 themesong.

(Not that it should be sung to REM's end-of-the world or anything or start like this....)

Playstation, mental masterbation, Shockeye's Useless News
Sir Bruce is not banned this week.....

I think a 0-100 rating system is a bit of a reach.  Can one objectively say one game is really an 84 and another is 85?  Does it make any difference to the reader?

No, I prefer a 0-10 system instead.

Bruce

You're right, 8.4 vs 8.5 is much clearer....

Personally, I vote for a rating system A-Z.

Xilren

Sorry, I meant whole integers only.

Bruce

I have problems with 0-10 systems because I don't really see much a difference between 2 and 3 or 6 and 7.

Either it's worth your time or money or it isn't.

I don't really like the 0-10 scale for video games.  A 10 on the 0-10 scale is equal to a 100 on the 0-100 scale leading one to believe that the game has achieved perfection and can't be improved.  This leads to 10's either never being given out, or being given out to games that probably don't actually deserve it.  Now 6-0 are going to be average to really fucking bad so that leaves you with 7,8, and 9 to really give out to the good games.  That's only 3 different levels of goodness, and good games are going to have to get crammed into one of them.  Even one notch difference is going to be a drastic change to how good you're saying a game is.  0-100 offers a lot more flexibility.

I'm pretty happy with three levels of "goodness" for good games.  It's reasonable and useful.  30 levels isn't.

Bruce

I agree, though if you want to have the reviews listed on Game Rankings you'll need to follow that format to a large extent. Generally speaking when I see games ranked on a 10 point scale, I translate the scores into something like:

9+ Highly Recommended
8 Recommended
7 Mixed Bag
1-6 Skip it

Edit: Fixed quote spacing

I myself will vouch for the 100-scale...it more-easily lends itself to quantify small differences between games. There's a huge difference between a 6 and a 7....not so much between a 64 and 70 (or whatever).

First of all, 0-10 and 0-100 are the exact same thing with 0-100 have more wiggle room. So that suggestion goes right out the window for just being contrary to REALITY. An 84.5 is the same thing as an 8.45 but an 88.5 on our system is noticeably better than an 84.5, at least more so than an 8.45 & 8.85. Like I said, wiggle room.

The reason all of the scores are based out of 100 is that it adds more weight to the multipliers on each score - like gameplay and story > graphics and sound.

The question I'm asking in this thread is - have we gone deep enough. I've thought about it and we need to add a visceral fun category. For example Planetside > WoW on visceral fun. Or something that'll light less fires under the ass of this community - BF1942 is viscerally more fun than WWIIOL. And don't tell me I can't compare them, I just did. That and the whole point of this review system is that it's NOT in a vacuum like all the other sites' review systems are. It's completely comparative. And at the end of the day, with proper graphing, I'm hoping we can see why Freedom Force is more fun than say SW:G.

Anyway, I'm rambling now, point is we need a review system, and a solid one. One where you can look at a graph of games and see where the range of games worth purchasing falls. And I'm hoping the system we've come up with will do that.

You mean 60 and 70. Yea, the difference between a 60 & 70 and a 6 & 7 are the exact same thing. Now as I said in the above post, the difference between a 60.0 and a 65.0 is more noticeable than a 6.0 & 6.5. At least to the layman onlooker.

My opinion is:

Six of one, half a dozen of the other.

Roflcopter.

No, I don't, because that's what I would have written if that's what I meant. I was trying to illustrate the point that a 100-scale is better than a 0-10 scale in regards to quantifying differences of degree between games. Sure, a 0-10 scale is the same as a 100-scale system if you use decimal numbers, but that's just silly to me.

Mouth: "Yo, wassup Foot?! Come on in, DAWG, we're open!"

The problem is you are trying to gussy up something that is highly subjective to begin with with some sort of psuedo-scientific methodology so that you can proclaim definitively that one game is better than another -- ain't going to work. Let's take your Planetside > WoW on visceral fun example. While I don't necessarily disagree with you (I like shooters a lot) I would be willing to bet that there are a lot of people who would. And how are you going to prove your case? Measure the relative volume of explosions in game? Count how many pieces fly around when things blow up? It's highly highly subjective.

Adding a "Fun Factor" category to your ratings would probably cover what you are trying to measure but again that sort of thing is highly subjective. To digress a bit before getting back to my main point, some other categories you might want to consider include "Replayability" (sort of related to Depth) and "Multiplayer capability".

Getting back to my original point, there are lots of games I've played that scored really high on review sites that I enjoyed a lot less than games that ranked lower. The GTA3 series is the prime example for me personally. I've tried to like that series -- I even bought the PC version after I tried the PS2 version in case it was the controls that was ruining the game for me, but no, I just don't like those games.

You say that Freedom Force is more fun than SW:G and I'm sure a lot of people would agree with you. However, a lot of people would also disagree with you considering that there are still hundreds of thousands of people playing SW:G right now. Yes maybe the combat in SW:G sucks hard, but there are a lot of people who enjoy playing it as, for example, an economic/social sim (aka "SimBeru"). So who you are to tell them that they should be playing FF instead because FF is more "fun" according to your rating system?

So my suggestion is to not worry so much about coming up with the "definitive" rating system and instead work on writing down a solid editorial process that reviewers must follow. Speaking for myself, what I'm looking for in a review is enough descriptive information about the game (both strengths and weaknesses) to help me decide whether or not I want to buy it. The overall rating is helpful in narrowing down which games I might consider buying (see my previous post) but the description of the game is what really influences what makes me want to buy it or not. For example, gameplay and story are (usually) much more important to me than sound and graphics (I will buy a game every once in a while if the graphics are state-of-the-art ala Doom 3). While you've weighted the sound and graphic scores somewhat like my personal preferences, it's still the case that the way you have it described above a game with really outstanding sound and graphics might rate higher than one with an outstanding story but only servicable sound and graphics (7-10-10-9 = 8.8 vs 10-7-7-9 = 8.5).

I don't want to proclaim anything definatively. I just want to achieve some sort of gaming spectrum against which shit is compared. Hopefully after a year it will result in a decent study of what that year meant to gaming. And so on and so on and so on.

It's a review, not a carved-in-stone truth.  The idea isn't to come up with a mathmatical formula to prove greatness in a video game, the formula presented here just explains how a reviewer determines the final score he gives a game

As to the people who are saying the system is bad because the reviews are subjective anyway....

Welcome to F13.net

The reviews are subjective, extremely subjective. That's the point.

Methinks schild just wants to be on gamerankings or whatever so he can more easily whore himself out to the industry for free stuff, but what do I know, I'm a foozle.

KEKE  :-P

Actually, we want the scoring system for 2 things:

1. We can say we like a game. But really, the only way to properly compare it, particularly MMOGs, is to have them charted out in comparison to what else we liked. The easiest way to do this is - numbers. So we're trying to make a system that best converts to numbers, realizing that there's nothing more insulting than distilling 2+ years of game design/programming/art into a score.

2. A basis by which all games are reviewed. Some sort of baseline standard.

I already get free stuff. And lots of it. Saying what you said is just stupid. Particularly the whoring out bit. Do not even make that joke. We are not about to whore out for any reason. We haven't needed to thus far and we certainly won't be any time, ever, in the future.

Edit: By the way, it does get us on gamerankings, but that's irrelavant. It will be years before we get on there because our archive is too small and we're not corporate backed. We don't need the ranking system until we have a big enough archive, but the reasons noted above are why I'm far very interested in having one.

Jumpy this morning, are we?

It would have been funny to see what would have happened if I tried.

Or scarey. Either or, take your pick.

So why not 0-1000?  0-a bajillion million?

Get a clue.  I'm quite aware that a larger range allows for more possibilities, unless you want to go decimal.  That isn't the point.

Bruce

Common sense.  Acting stupid doesn't help make a point.

It's obviously not common sense, because common sense would use a 0-10 range, not 0-100.

Moreover, he did not claim common sense as his rationale.  He claimed wiggle room as his rationale.  If more wiggle room was better, then 0-1000 would be better than 0-100.

This is all really dancing around the essential issue, and that is how much wiggle room is appropriate.  Can the reviewers justify wiggling enough to say one thing is a 500 and one thing is a 499?  Perhaps not.  But is there is enough wiggle room to say one thing is 50 and another is 49?  schild and others say yes; I say no, and thus that is my feedback on his system, as requested.  Considering the variety in tastes of people of even similar interests, it's hard enough calling something an 8 which will mean a 7 to some people and a 9 to others, and I certainly don't think any reviwer (not simply f13) is qualified enough to provide reviews with a tenfold greater precision.

Bruce

Common sense states that both 0-10 and 0-100 are viable options for reasons people have gone into already.  Schild has stated that his personal preference is 0-100 due to wiggle room.  0-10 isn't enough for his tastes, and 0-1000 is obvious overkill to everyone which is why I say you're acting stupid.  Saying that 0-100 is equal to 0-1000 is like suggesting that if you want to go with smaller numbers instead we might as well go with 0-5 instead of 0-10.  What's the difference between a 2 and a 3 on a scale of 0-10 anyway.  Hell we might as well go with 1-2, where 1 = don't buy it and 2 = good since that's what people really want to know right?

No one is suggesting 0-100 is equal to 0-1000, and you're the one acting stupid by interjecting it.

Bruce

I'm going to say this one time and one time only: stop being a jackass.

That's sound advice.

Bruce

I think a con-system would be more site-appropriate.  Ex: God of War cons RED because it simply reems every sense in your body and still leaves you begging for more.  VtM:Bloodlines cons YELLOW because while it is a buggy piece of shit, it's also a lot of fun.  Why not combine both the funny and the subjective?

That's a novel idea.

So, can we decide on the specs and send them to me, because this is shit I can do to avoid work.



And I need that ok!

Shockeye and I were going to finalize the base stuff tonight. We're still tooling around with some other things. But uhm, he disappeared fromteamspeak about 2 hours ago.

Looks like it'll have to wait until tomorrow. Hopefully we'll have it set in stone tomorrow night. We're looking at somewhere between 6 and 10 modifiers for final scores, and the base scores which result in a pre-modified final are probably going to see what you see above.

I have a question and a suggestion:

Can posters submit reviews within reason? Obviously you guys don't want to be flooded, and will want to read our reviews before posting, but it'd be cool to have some input.

As for the suggestion: To handle the subjectivity issue, why not include some sort of lean factor. Like I think Gamespot does. Basically, for instance, you might give a FPS a lean over a turn based game because you like FPS better. That way people get an idea "ok, if I'm an FPS lover I'll agree with his rating Planetside over WoW but if I'm not I can take his lean into account."

I didn't word that very well but am I making sense?

As far as I know, they're always looking for new people to do work. That being said,  was told that if you want to submit stuff, you need o first submit a pilot review of sorts, then maybe you can do more 'official" things.

Seeing as I hate writing as it is, I'll just be submitting my scores with mini-review within each review's thread.

And I think you mean "bias", not lean...well, maybe you did mean that, but bias is better anyway.

Bias is a good word. Just something to give an idea of the reviewers general thoughts going into the review I guess.

Bumpfuzzle.

Still looking for that finalization fellas. This week is light and I could use something to break up the tedium.

I'll get all the information in for a prototype tonight. After that, some minor changes might be in order.

Looks like Jade Empire is going to be our guinea pig.

Thinking of doing some older stuff. But I know both Shockeye and I are getting it.

The system you choose is pretty arbitrary as inconsistencies will become inevitable with time.  I would say that a 5 point system is more than sufficient and can even become a 15 point system with +/-.  The grading system we use in american schools is such an example (A-F) and is something everyone is familiar with.

Ultimately, the grade or score that you assign is just a catch tag to garner the readers attention enough to actually read the review.  I personally would try to pick a system unique from all other gaming sites.  An A+ --> F- system with the tags changed would be both humorous and semi-unique.  Ultimately, you want to let the reader know that the game is worth playing or not and why.  Siskel and Ebert did this with a binary approach (thumbs up or down) and were very successful.  I mean really... you either like it or not and the important stuff is in the reasoning behind your decision. 

Summary: The simpler the grading scale, the better it will age.  Simple remains consistent longer.

My 2 cents.

Edited for spelling.

Sigh. I never even thought of the A+/A/A-/etc system. Nebu for the win.

I'm going to curl up and die now that I realize that it's 50x better than what everyone is using.

Are you chucklefucks ever going to do something about this?

Oh no!  An ass from the past!

Umm... I meant blast, of course.

Orange Hasslehoff want you.

Would someone please invent a teasing font?  I seem to be making everyone cry lately.  First Cheddar, then Shockeye and now Avatard.  Actually, making Cheddar cry is a bit of a hobby.

Use pink you big silly!

I WAS JUST TEASING!

(http://www.click2music.fi/pink/kuvat/k-pink-3.jpg)

Ok?

(http://www.screamingxenas.com/postcards/callie_in_hell.jpg)