Middleware + MMOG development = vapourware?

[UnSub]

Following the recent news that the release of Gods and Heroes has been pushed back to next year and that the beta for Fallen Earth is not going to be until early 2008 (maybe) following indications of a 2007 beta, I'm starting to wonder: can development studios build middleware tools AND MMOs that use those tools and expect to acheive competent results in both?

Off the top of my head, I can't think of any MMO middleware tool providers who have put out well-regarded games on time as well as producing committed third party middleware. I know that some MMO devs have licensed out their MMO engines / tools after they've released a game, but that's different to saying you are provding a dedicated suite of middleware tools.

Perpetual of G&H has licensed its middleware apps out to Bioware, but has had to drop the dates and employees on their own product. BigWorld was going to use Citizen Zero as its flagship, but that MMO died a long and quiet death, making the third party-developed Dark and Light the most notable BigWorld title. Icarus Studios has been committed to Fallen Earth for something like 4 - 5 years and still doesn't have a beta up and running or a solid date for it.

It seems to me that no company has been able to develop middleware and put out a game in any sort of reasonable timeframe, on top of which the licensing of such MMO middleware appears to have limited appeal at this point. Am I wrong? Is there some success story out there that I'm unaware of?

[Trippy]

Perpetual's Entertainment Platform is not in the same category as something like BigWorld. PEP is the stuff "around" the game -- billing, forums, customer service interface, etc. while BigWorld is a game engine.

The companies that are trying to do both at the same time with unfinished products are doing it wrong. They either need to finish the products and then "extract" out the middleware part or just do middleware. The market for MMO middleware products will continue to grow, however, so even though there hasn't been much success so far that won't be the case forever.

[Kaa]

With Meatplace Raph Koster seems to want to push out the middleware first and then develop a game on its basis. I guess we'll see how that goes.

kaa

[tkinnun0]

If you are not using your middleware to solve your development problems then how can you know your middleware is going to solve the right problems?

[Trippy]

You talk to your customers.

[SnakeCharmer]

When has that ever happened?

[Trippy]

When has what happened?

[tkinnun0]

Customer: <requests a feature deviating from WoW-Diku-mold>.

Middleware vendor: That's a nice feature. We have to prepare a feasibility study evaluating the existence of sufficient synergies with our existing architecture.

Customer: So can we start designing with that feature in mind?

Middleware vendor: <says "we don't know" in not so few words>.

[Abelian75]

Customer: <requests a feature deviating from WoW-Diku-mold>.

Middleware vendor: That's a nice feature. We have to prepare a feasibility study evaluating the existence of sufficient synergies with our existing architecture.

Customer: So can we start designing with that feature in mind?

Middleware vendor: <says "we don't know" in not so few words>.

If you're making MMO middleware that's so specific in nature as to enforce a diku-model MMO (which is itself somewhat hard to define), then that's some rather goddamn specific middleware you're making.

[Murgos]

Attempting to develop a solid, robust middleware package and an MMO simultaneously sounds like a losing proposition to me for anyone but a Microsoft or an EA.  Even then I don't see how the two could be used to help each others development without stretching out the time to deliver both products enormously.

The middleware package is going to require time spent in areas that won't benefit the MMO and vice-versa and so I would expect you are going to hit periods where one team is blocked and waiting on the other.

[tmp]

If you're making MMO middleware that's so specific in nature as to enforce a diku-model MMO (which is itself somewhat hard to define), then that's some rather goddamn specific middleware you're making.

It's like DIKU mud source except with fancy name and you pay for it. Often making things limited in functionality rather than super-flexible is much easier.

[Pendan]

I think you can add Wish to the list of middleware developer and vapor ware game. The server engine was being marketted for the Wish product. Supposedly a chinese MMO company was going to use it for a game before the whole thing came crashing down.

Edit: You also have Hero's Journey which has been in development forever and looking like vapor ware. Simtronics the developers also marketing the engine.

[Abelian75]

It's like DIKU mud source except with fancy name and you pay for it. Often making things limited in functionality rather than super-flexible is much easier.

Yeah, but the DIKU source isn't really middleware, it's just someone's source code for a game they wrote.  To me, MMO middleware would be:  possibly a graphics engine, a server that handles instancing and regions and moving players between those things, and relevancy of entities to other entities, and a client that can interpret server messages.  Beyond things like that, it's a little silly to expect someone else to do anything general enough to work for any MMO.

[Krakrok]

NWN was successful middleware in my opinion. They just fell down in the number of users per server department. And the EMU server softwares seem to be pretty effective middleware.

As far as commercial products I think Multiverse is a big disappointment. They need to bring in a Delphi developer and build a UI for that shit.

[Tairnyn]

I can't think of any that provided both effectively. Wish was a specific game I can think of that spent a lot of effort on their middleware (http://www.zeroc.com/ice.html) but the game was a horrific pipedream that seemed more like a testbed for the middleware than a serious game looking to compete in the market. However, from what I can tell the Ice middlware has been pretty successful so I wouldn't write it off as a total wash. It just doesn't seem feasible to build a solid game off a software infrastructure that isn't already stable and in production.

[Count Nerfedalot]

With Meatplace Raph Koster seems to want to push out the middleware first and then develop a game on its basis. I guess we'll see how that goes.

kaa

last time he tried that, it was called "SWG".  Building on that success...

[Margalis]

One of the dangers is that if the game sucks people will also think the middleware sucks, even if it rules. And if the middleware sucks then the game will suck. So you have to hit a home-run on both.

To me it makes a lot more sense to create a good game based on an architecture that you planned to be modular, then break off the engine down the road. The game will help sell the engine and making the game will give the engine a good test.

[Tairnyn]

I would think that there's more reliable income to be had from a solid infrastructure based on corporate licensing than there is from rolling the dice on the fun to make a game with broad enough appeal to cover the support costs. Building a game from an existing middleware product seems it would be easier than trying to spawn a middleware from a game, especially considering you'd want your infrastructure to support functions that are above and beyond what are necessary for the game you're building. There has to be more cost effective ways than the saturated MMOG market to provide a sufficient demonstration of capability.

Either way, I'd venture a guess that no company has the talent and resources to do both effectively, even incrementally.

[tkinnun0]

Either way, I'd venture a guess that no company has the talent and resources to do both effectively, even incrementally.

I fully believe that if they wanted, Blizzard could extract a solid middleware out of WoW and that this is one enabler of their success. You can guess at its features by comparing a low level mage spell with a high-level instance script.

[Tairnyn]

I fully believe that if they wanted, Blizzard could extract a solid middleware out of WoW and that this is one enabler of their success. You can guess at its features by comparing a low level mage spell with a high-level instance script.

I think that depends on where you draw the line for middleware, which is a discussion in itself. There's the notion of the shell for a skill-based online game as middleware or an abstraction layer to support hardware clustering, security, persistence and scalable distributed access. The most effective approach is going to be one of generality, meaning that even a customer that could care less about games can make use of your product. I don't know much about Blizzard's infrastructure, but I've seen nothing that indicates they've fleshed out a general-purpose middleware that is modular enough to be licensed separately.

[Abelian75]

I think that depends on where you draw the line for middleware, which is a discussion in itself. There's the notion of the shell for a skill-based online game as middleware or an abstraction layer to support hardware clustering, security, persistence and scalable distributed access. The most effective approach is going to be one of generality, meaning that even a customer that could care less about games can make use of your product. I don't know much about Blizzard's infrastructure, but I've seen nothing that indicates they've fleshed out a general-purpose middleware that is modular enough to be licensed separately.

This is kinda what I was getting at.  Blizzard selling its source code doesn't really count as middleware in my opinion, it's just selling the source code for your game.  The same goes for the Unreal engine, really, and certainly the same goes for NWN.  They aren't really middleware, they're... endware (I'll grant that unreal is an arguable case, but I would at least say that it's merely been twisted into use as middleware, somewhat unnaturally).  Havok is what I would consider actual middleware.  It's a physics engine.  It isn't a game itself, just a library you can plug into your game to make it do (really) cool shit.

Basically, what I take issue with is the idea that middleware should be helping to do things like make mage abilities and encounter scripts.  That's way too specific and inevitably inflexible.

[Trippy]

There's nothing unnatural about classifying the Unreal Engine as middleware.

[Tairnyn]

There's nothing unnatural about classifying the Unreal Engine as middleware.

I agree. The term middleware is covers such a large gamut that this discussion will inevitably lead to a conflict of semantics. Building a MMOG game engine from an existing game like WoW would be entirely feasible. In fact, any game that's written using good software engineering practices should provide a well-defined distinction between the game engine and the game content. The issue of building a game and middleware at the same time seems to be more a function of middleware flexibility and functionality than development resources.

[Murgos]

The problem of building the game and then extracting some middleware out of it is that you are increasing the cost/time/complexity of the games development in an attempt to build-in reuse and modularity.  Not to mention compromises in both products due to the nature of the two complex systems.

I believe you would be more likely to produce a successful game if you simply focused on the game without concern for future productization of portions of it.  I would easily expect 1/3rd, and probably much more, higher costs to building the game if you are going down that path.  My own personal experience is that when a piece of software is expected to be reused, and thus adding large amounts of configurability and complexity, the time to deliver and the cost of the software balloons to often 2-3x what it would have been if you had followed KISS and designed it as a 'one-off-get-the-job-done'.

That's fine when you are designing it as a product, not so good when it's just a portion of another product.

[Pendan]

Most of the big MMO game developers appear to have problems with developing a middleware that is useable from one game to the next in their own company. Being able to have your code usable by another company is a lot more difficult than reusing code internally. I think this alone says something about game development.

I don’t know of any evidence the multi MMO game companies like NCsoft or SOE have used much code from one game to the next. Maybe a bit of communication/chat code between SWG and EQ2.  Turbine may be an exception and why a relatively small developer has been able to do a number of big (though not so successful) games.

[Kaa]

The problem of building the game and then extracting some middleware out of it is that you are increasing the cost/time/complexity of the games development in an attempt to build-in reuse and modularity.

Um. If you didn't build in reuse and modularity, you have crappy coding practices and your software will be impossible to maintain. And since a MMORG is an ongoing effort, with patches and expansions and what not, you really really need reusable and modular code regardless of whether you want to sell separate chunks of it later.

Kaa

[Stephen Zepp]

Finally decided to delve in to this one!

Lots and lots of good points, not going to quote them all, but some follow ups:

--building a game and an engine at the same time isn't a particularly effective strategy. At least how we do it is to build an engine base, test it out with a game, and then abstract/"agnostize" effective game systems to a middleware level.

----it's important to reinforce what someone said above, and was also counter-pointed : game systems do not make good middleware/engine systems. By definition, life cycle management, milestone/other metrics, and a host of other reasons, what comes out of a game development cycle is a game, not middleware--and that's a good thing.  I'll give an example: shoreline rendering. Many people would say that "all games need shoreline rendering (if they have water and land of course), therefore my middleware/engine should provide it!. The problem is however that every game is going to want to do it differently (for the most part at least), and while a single game could come up with a shoreline renderer in a few weeks, making it agnostic and flexible enough to work for any game is a multi-month process. You could provide a game specific shoreline renderer as a reference implementation, but then your middleware customers will say "but it's not doing what I want--this engine/middleware sucks!". Leading to my next point...

----in just about every market, engine/middleware purchasers don't want middleware--they want their game already made. That sounds crass--but it's true : people evaluate engines and middleware based on how it implements what they want--not from the perspective of "how much does this save me in time/resources". Very experienced developers purchase properly, but honestly there aren't that many out there--and I've had AAA studios get pissed that we as a company wouldn't change how an exporter worked to meet their needs, and decide to write an engine from scratch because of it.

----re-focusing on the point above : game development says "make a system that meets the game needs". engine/middleware development says "make a system that meets a majority of all possible needs with the maximum flexibility and remains agnostic".

--also reinforcing the point someone else made : when you make both a game and middleware, they become "tainted with each other's flavor". I'll use Horizons for example : it's not very well known that the company sells their backend architecture for billing and customer service, and it's actually quite an effective system--flexible, agnostic, and powerful. Yet because Horizons failed so miserably, it's not being used very much by people that would really benefit from the backend systems. Torque gets nailed by the reverse of this: since it evolved from the Tribes games, Torque is labeled as an "fps engine"--nothing could be farther from the truth. Torque is a completely agnostic, cross-platform game engine, that happens to have both fps and off-road racing reference implementations. Unfortunately, given the point above, people don't evaluate middleware/engines for what they are intended to accomplish--they evaluate based upon how many things resemble what their end game goals are.

Lots and lots of challenges regarding developing engine/middleware and a game to prove that development--we at GG believe in "eating our own dogfood", and we invariably "fix" our dogfood (engine) from the experiences of eating it (making games with it)--but it's much more of a challenge than you would think.

[tmp]

Um. If you didn't build in reuse and modularity, you have crappy coding practices and your software will be impossible to maintain. And since a MMORG is an ongoing effort, with patches and expansions and what not, you really really need reusable and modular code regardless of whether you want to sell separate chunks of it later.

And yet doubt it stopped plenty of MMOs to date from going this very route at the cost of all the later trouble you mention...

I love doing reusable and flexible code because it can save me time, but usually only in the long run. Short-term it means extra work to design and implement what I can estimate to be flexible enough solutions... and every now and then some still can turn out to be not good enough after a while, and require extra rewrites anyway. Given this I can see how in real production environment when sometimes you need to push this or that feature or even game itself *now* the temptation to go the short route and tailor your code exactly to what you need at the moment and nothing beyond... can be too great.

[Mrbloodworth]

Some good thoughts and reads in here.

[Tairnyn]

And yet doubt it stopped plenty of MMOs to date from going this very route at the cost of all the later trouble you mention...

I love doing reusable and flexible code because it can save me time, but usually only in the long run...

Also a good point. Even though it's desirable to have every base covered in the software engineering process, adding new features to a highly flexible architecture usually requires a significant amount of re-engineering to adapt new features that weren't considered at the outset. For example, I feel that the best way for WoW to handle hero classes is to make them akin to the job system in FFXI where players can pick and choose their current hero class and switch between them depending on the needs of a given dungeon or raid. However, it's likely that Blizzard didn't consider this option during the engineering process, making it a monumental coding effort to provide such functionality given the current character storage and access schema. I could be wrong, though. Maybe they have other motivations for choosing the "make a new character of the hero class" option.

Unless you have amazing vision when you design your infrastructure it's likely that major shifts will upset too many invariants to be worth investing in the process. And even if you do choose to make that investment it's likely another feature you didn't consider will crop up and restart the process leading to fatigue and the eventual "hack it in" mentality that gets the job done now rather than done "right". One on hand you want to wait until you have a laundry list of changes that motivate such major changes, but on the other making a large number of changes to an existing infrastructure could lead to a hellish debug process that, combined with a large-scale real-time (often unpredictable) communication environment like a MMOG, may never see the light of day. I'd love to hear the outcry of players when the beta of a new expansion that makes such changes ends up running horribly or, worse yet, breaks the game entirely.

[Alkiera]

To me, middleware should be a more modern, 3D-capable version of LP or MudOS.

Your game engine is what MUD coders of that era would call a 'mudlib', and then content was built on top of that.  Some games distributed their mudlib, so it was available to make games with the same movement/combat/room/NPC systems, just different specific instances of those.

MudOS provides network code, text processing, hooks for a security/access control system and a bunch of utillity-type functions, a long with an object-based language for writing the rest of your game.  The simplest mudlib was just enough to allow the engine to start and open a specific port, to which people could telnet, provide a name, and then sit in a chatroom.  There were a couple commands, mostly 'say' and 'quit', and they and the login system were all written in the scripting language implemented by the engine, so you could look at them, and add code to add, say, character storage, passwords, etc.

It's a heck of a way to start a game (I've tried, it's not easy) but it beats the pants off trying to figure out how to open a port for listening, handling requests for connections, handling the buffering of data back and forth to each client, etc (I've seen that code in Diku-based MUD engines, implemented in pure C.  It ain't pretty, and it's easy to write poorly, and harder to write the most efficient code.)  Better to have that done by someone who was good at it, who can expose some pre-built communication protocols (MudOS supports telnet, raw sockets, and http, even some email stuff like SMTP/POP3, last I recall), though the telnet system is the default, it's pretty trivial to use the http system to write 'pages' hooked into from your games' website to do stuff like a dynamic online-user list, or what-have-you.  Even, say, a CGI-based chat/mud-mail portal, down to using the same user logins as the telnet system.

Heck, built something like that first, add enough to support location mapping, and a good UDP socket comms protocol, then go to town with your client engines.

--
Alkiera

[Murgos]

The problem of building the game and then extracting some middleware out of it is that you are increasing the cost/time/complexity of the games development in an attempt to build-in reuse and modularity.

Um. If you didn't build in reuse and modularity, you have crappy coding practices and your software will be impossible to maintain.

There is a vast difference between maintainable code and code that is meant to be a user interface.  If you want to argue that you should develop every logic block as though it's going to be offered to an end-user I think we should probably agree to disagree now and save a lot of caps, bold and otherwise emphasized fonts..

[Stephen Zepp]

The problem of building the game and then extracting some middleware out of it is that you are increasing the cost/time/complexity of the games development in an attempt to build-in reuse and modularity.

Um. If you didn't build in reuse and modularity, you have crappy coding practices and your software will be impossible to maintain.

There is a vast difference between maintainable code and code that is meant to be a user interface.  If you want to argue that you should develop every logic block as though it's going to be offered to an end-user I think we should probably agree to disagree now and save a lot of caps, bold and otherwise emphasized fonts..

I think the term that you guys are looking for is one that's specific to tools/engines/middleware, and that is "game agnostic".

I can most certainly write modular, easy to reuse code that is completely focused on my game, and therefore not appropriate for a game engine unless it receives additional dev cycles for further abstraction, flexibility, and game agnotiscity (I know I just made that one up!).

[Murgos]

I can most certainly write modular, easy to reuse code that is completely focused on my game, and therefore not appropriate for a game engine unless it receives additional dev cycles for further abstraction, flexibility, and game agnotiscity (I know I just made that one up!).

Thanks, thats the point I was trying to make.

[Abelian75]

There's nothing unnatural about classifying the Unreal Engine as middleware.

Oh, I dunno about that...

Seriously, it really does feel like working directly with a very specific game engine.  A very specific game engine that happens to have been written carefully enough so as to be modifiable without TOO much stress, but in general there's a great deal more tearing pre-existing shit down when working with unreal than I would expect from something that's really middleware.

I'm not dissing unreal in the least, but it really is just a shade away from being someone's source code for a game they made.  The chief difference is that it's pretty good and robust source code (which is a pretty critical difference, granted).  Basically when you get unreal it isn't like, "Here's a 3d engine," it's like, "Here's UT3, have at it."

It is mostly semantics though, as people have said.  And if enough people use something AS middleware, like has been done with Unreal, well, then I guess it makes sense to call it that.  But the main reason I brought this up was because of earlier comments implying that the burden was on a middleware developer to do something like allow for a new kind of mage ability, which imho is far beyond the scope of middleware.