"Oh btw, FFXIV Online in 2010"

[Margalis]

Compared with the previous year, sales at Square Enix were down 25%, operating profits were down 56%, ordinary profits were down 71%, and net income was down 36%.  Sales of Final Fantasy XIV were actually the biggest positive contributer, with 630,000 copies sold (190,000 in Japan, 260,000 in North America, and 250,000 in Europe).

Pop quiz: without going back to google can you identify what time periods are being compared?

[UnSub]

Even the beta was fucking clownshoes, as only select people could report bugs. I wasn't one of the chosen few, so every time I ran into something broken my only recourse was to throw my hands in the air and hope someone ELSE reported it.

[DLRiley]

Wow this game is so bad I can't even find a wiki page...

oh wait I found it, had to type in FFXIV wiki, as oppose to just FFXIV, then scroll down to the 6th/7th link...

SquareEnix I remember when you was cool.

[Paelos]

Even the beta was fucking clownshoes, as only select people could report bugs. I wasn't one of the chosen few, so every time I ran into something broken my only recourse was to throw my hands in the air and hope someone ELSE reported it.

Wait, they held a beta but didn't allow everyone to report issues? WTF is the point of even putting those people in there?

[DLRiley]

Even the beta was fucking clownshoes, as only select people could report bugs. I wasn't one of the chosen few, so every time I ran into something broken my only recourse was to throw my hands in the air and hope someone ELSE reported it.

Wait, they held a beta but didn't allow everyone to report issues? WTF is the point of even putting those people in there?

Well most mmo's beta's simply forward all reports to a comment box that gets deleted every 30 minutes. FFXIV is just being honest about it  

[Paelos]

I guess I fail to see what the point of the beta is then.

And don't say advertising, I get that one, but there are better ways to do it.

[DLRiley]

I guess I fail to see what the point of the beta is then.

And don't say advertising, I get that one, but there are better ways to do it.

Having a beta is simply a part of the tradition of online gaming. In Korea games don't even leave beta to enter into official launch or release, they simply open a cash shop and steadily increase the size of the shop.

[ghost]

I don't think that SE's plan of having dedicated bug reporters is really that bad.  The point of having a ton of people in an open beta really has to be more of a test of the game overall, but do you really need three thousand blazing idiots trying to complain about every aspect of the game?  It would be much more efficient to have a simulated world environment with all the kinds of idiots that you will see once the game is actually released and then have dedicated expert evaluations in game.  Of course you have to have good expert evaluation and listen to what they say, or else you end up with the shithole experience that is FFXIV.  And honestly, if they would just fix the goddamn mouse it would be at least two points higher on all the reviews.  

[UnSub]

It's stupid because not everyone who can post those bug reports will. Or someone might find an obscure bug that becomes more prominent when the game launches, but can't report it through in-game tools so doesn't bother.

It's putting barriers where barriers don't need to be.

[Rasix]

I guess I fail to see what the point of the beta is then.

It's an effective way to stress test your servers and other systems related to your MMO (account creation, billing, etc) in a near retail environment.

Disabling bug reporting is just stupid though.  I'm sure any test manager would want to peruse those reports for things that have possibly slipped through the cracks.  MMOs have a lot of variables and I'm doubting their official QA has the manpower or runway to go through as many scenarios as the actual beta test user base can.

[ghost]

It's stupid because not everyone who can post those bug reports will. Or someone might find an obscure bug that becomes more prominent when the game launches, but can't report it through in-game tools so doesn't bother.

It's putting barriers where barriers don't need to be.

If SE has a system set up to allow a certain number of dedicated bug reporters to get into the game and see how it reacts with a basically full population and they aren't reporting the bugs then they hired the wrong people.  Also, how many people would you have to hire to sift through all the bullshit of open reporting?  It's clear that most MMOs don't respond to every single bug report and, quite likely, round file most of the reports before anyone even sees them.  Another consideration is that in the short amount of time that most betas run you will never uncover all the bugs, particularly obscure ones. 

[Lantyssa]

I'll ask it this way then... if they had no mechanism to report bugs, how would they pick dedicated bug testers?  If they also explicitly said not to post bugs on the boards, it was only for feedback?  How exactly does that work?

[UnsGub]

Also, how many people would you have to hire to sift through all the bullshit of open reporting?  It's clear that most MMOs don't respond to every single bug report and, quite likely, round file most of the reports before anyone even sees them.

Just need one person.

Half the reports are unable due to poor repro and problem descriptions.
45% are badly written fixed dupes.
4% are badly written dupe of known but not fixed dupes.
1% are well written and lead to new bugs, support existing bugs, or provided a 100% repro a "random" bug.

Respond can mean many different things, but in general a beta testing will not get any response besides seeing their bug fixed at some point.  Any system to record a response on everything like that is going to be costly with the overhead to manage it.  Internally I used a system like review 200 reports, 3 new bugs, 10 dupes.  It was summarized by day or period of testing on a version.  Dr Watson captures follow the same type of curve.  Two crashes cover 50% and 45% and the rest are a few dozen issues.

Good career testers typically have 50% of their issues result in a code change.  The response to the other 50% is typically pretty quiet to non-existent.  Move on is cheaper then following up.  Repro steps and clear descriptions are hard.  Just go to utest.com to see the effort to train people on how to communicate properly.

[Paelos]

I get stress testing, but that can only go so far. I understand signal to noise, but if that's the issue you can always filter out the responses.

[ghost]

I get stress testing, but that can only go so far. I understand signal to noise, but if that's the issue you can always filter out the responses.

Most MMOs don't do a good job of filtering.  That is clear from the shitty, buggy launches that we always see.  And most of these have had "open reporting", have they not?  I have no problem with a new system if it works.  Of course  you would have had to start with something that wasn't a steaming pile of shit to start with to see how a new idea might deal with the problems typically seen at launch for most MMOs.  This efectively rules out looking at SE and FFXIV as a reasonable example of whether such a system is feasible. 

[Count Nerfedalot]

Any software development process that is so sloppy/rushed/stressed/overworked/beyond caring that it allows, without severe personal repercussions, a coder to release code to test that doesn't even work in the simplest test conditions is so screwed already that it really doesn't matter if QA finds and reports the problem or not.  It's not going to be fixed on time.

Laying the final blame on an understaffed, underscheduled QA team (that has probably been forbidden to even talk to the coders) for the shitty results from a process that was broken from the initial vague hand-waving "design specifications" through the unrealistic deliverables timeline promised to the suits resulting in hilariously impossible schedules down to the last line of code entered through a bleary-eyed fog at 4AM after several months of deathmarch coding might be a bit of scapegoating.

And that's spoken from the perspective of someone who's done software design, coding and maintenance for over 30 years now but never had the tester's thankless task of sitting under the output nozzle of that particular high pressure shit tube.

But hey, the product shipped, so QA probably all got laid off, the designers and developers moved on to other projects or companies en masse and the managers got their bonuses and a promotion so you can pretty well guarantee the exact same things will happen on the next project.

[naum]

Somebody needs to collect player submitted bugs and glitches, sift through them, enter them into a some sort of bug/enhancement/request tracking facility, even if it's a glorified @TODO spreadsheet.

Really, if that bare minimum is not done, then you are treating your users with most absolute disdain and disregard imaginable.

(I suppose you could "file" all the feedback and not a lift a finger, and that would be as equally reprehensible)

[Nija]

Honestly, I bet you could get a pretty good result out a moderated bug tracker that was accessible to beta testers. Something like Mozilla uses for Firefox, for instance. There are probably hundreds of open source solutions to this kind of thing.

[LK]

One person with a knowledge of the database and a good eye can sift through a large quantity of reports very quickly. SquareSoft just didn't want to do it.

[01101010]

Beta was, to my knowledge and of my opinion, was nothing more then a server stress test. Bug testing was said to be internal and a few choice players...mainly Japanese - if rumors hold true.

[Paelos]

Beta was, to my knowledge and of my opinion, was nothing more then a server stress test. Bug testing was said to be internal and a few choice players...mainly Japanese - if rumors hold true.

Yeah that was what prompted my 

I realize most MMOG players are total cock-knockers, but betas generally bring out the better players who generally follow and care about your game. If the most hardcore are saying it's a mess...

[DLRiley]

Like I said before I won't be surprised if most bug reports get canned automatically at the end of every 30 minutes. The general view that open beta is the time when most of the deal breaking bugs will not only be reported but fixed before launch is laughable. We have a long history of the genre proving otherwise, most likely if the game is that buggy they already know its buggy and working on fixing it or simply throwing their hands up and hoping the magic patch comes down. I think its equally dumb to say that player feedback isn't considered at all but this is usually at the cost of sifting through 10 years worth of spam.

[01101010]

Like I said before I won't be surprised if most bug reports get canned automatically at the end of every 30 minutes. The general view that open beta is the time when most of the deal breaking bugs will not only be reported but fixed before launch is laughable. We have a long history of the genre proving otherwise, most likely if the game is that buggy they already know its buggy and working on fixing it or simply throwing their hands up and hoping the magic patch comes down. I think its equally dumb to say that player feedback isn't considered at all but this is usually at the cost of sifting through 10 years worth of spam.

I tend to agree, but I had bug reporting capabilities in Alpha which was surprisingly missing when closed beta phase 1 started. That is when the beta forums lit up with fireworks and the schism between those that saw the writing on the wall and those happy to let SE shit on their faces. After the first week of phase 1, I knew exactly who we were to SE... stress testers with some phantom beta title.

[DLRiley]

Well at least SE was honest about it 

[Rendakor]

I knew exactly who we were to SE... stress testers with some phantom beta title.

Pretty much this. I did my best throughout phase 2 of beta (when I got in) to help the game; I posted in all the relevant feedback threads after every session, and even started threads in the General Beta forums to help people figure out what was broken/"coming soon", where there were workarounds, etc. After realizing that I wasn't missing the Bug Report button, and seeing the lack of communication from Devs or CMs, I eventually gave up.

[Kageru]

I think a lot of it has to do with the development time-line. A lot of the games don't seem to have any space in their plan for doing any iteration on game features. So even if they know something isn't working, or are being told the same thing by the community, there's just no time to do anything about it. Probably because they are so behind because they got over-ambitious in the design phase that there is nothing on their mind other than making launch.

[Reg]

The problem is, I can't see any evidence that FFXIV got "overly-ambitious" about anything.  Or are there some features I've missed that if they were implemented properly could be considered that way?

[Count Nerfedalot]

Generally, "overambitious" can be defined as "attempting to make an MMO in less than 5+ years".  Having enough time to bake, iterate and polish does not ensure success (TR being the poster child for this), but NOT having enough time pretty much guarantees disappointing results.

[Abelian75]

Generally, "overambitious" can be defined as "attempting to make an MMO in less than 5+ years".  Having enough time to bake, iterate and polish does not ensure success (TR being the poster child for this), but NOT having enough time pretty much guarantees disappointing results.

Particularly when you also decide it should have crazyawesomeomg graphics and also run on consoles.

[Koyasha]

Well, thing is they did manage to make FFXI have crazy awesome graphics, and run on consoles, and have it not be a bug-ridden game, being actually the MMO with the least bugs I have personally experienced.  This is why I find the situation with FFXIV somewhat odd.  It came out to be exactly the opposite of what I figured.  Before beta, I suspected that the game may or may not be fun to play, but assumed that at least it would be technically sound and have no major bugs.

[Numtini]

To be fair, we didn't get FFXI in the US until after the first expansion. No idea what it was like at launch. But other than design decisions and the interface, it was a really incredible game. I believe it's the first one to have footprints and such as you walked. I'm still surprised that nobody has copied the job system.

[Reg]

That's it exactly. FFXI was ambitious as hell and wasn't afraid to try new things like the job system and their unique auction house.  I don't see anything remotely new or ambitious in FFXIV and what they do have doesn't even work. It's like years of experience running an A level MMO actually made them even more stupid than when they started.

[Samprimary]

this is especially why I want a developer postmortem. You can't look at this and not wonder where it started to go so impressively wrong, and what the particulars were.

[tmp]

I suspect the truth is pretty simple, and this is one title where the developers took advice to spend the budget on hookers and blow seriously.

[01101010]

So when and how do we get a moratorium on MMORPGs?