Voice packs instead of VoIP?

[DarkSign]

After reading this article from Massively about how voice chat can destroy immersion, I was thinking...what about voice packs?

Depending on race, gender, personality, would it be feasible to have the computer deposit locally to your body the voice you choose?
You type it into the text bar...and everyone's computer synthesizes it into a voice...young girl if that's who you are...etc etc.

I wonder if that's possible?

[bhodi]

There are filters you can put over your voice, sure. Move it up and down octaves, play around with the timbre.. but they still sound weird.

If you wanted it to replace chat instead of being just a voice chat addition novelty, you can't. Not even commercial packages sound halfway decent. Also, you'd also have to figure out a quick and easy way to tone people out. Text is easy to ignore.. someone singing right square bracket, left square bracket, right square bracket, left square bracket, not as much. Forget about zone-wide chat. The fact is that text is a lot faster, much larger volume of information and no one can talk over anyone else. Voice is a poor medium except when your hands are otherwise occupied.

best song.. ever

[Morat20]

I do expect voice fonts and voice-to-text to start getting a whole lot better over the next decade, just judging by the trends.

If nothing else, things like Second Life, WoW and companies aiming to be the next-generation of things like chatrooms and IM are going to want to offer realistic voice fonts. Leaving aside manginas, a lot of the appeal of things like that was either anonymity or the thought of "Being someone else" for awhile. Being stuck with your own voice or an obvious fake is just going to be a cockblock to successful operation.

Not to mention that realistic text-to-voice and realistic voice-fonts are something that can be used in a lot of places outside pervs on the internet, and make you a good chunk of change in the process.

[DarkSign]

Not to mention that realistic text-to-voice and realistic voice-fonts are something that can be used in a lot of places outside pervs on the internet, and make you a good chunk of change in the process.

That's really what I meant...text to voice that use voice-fonts (Thanks for updating my vocabulary). What's going on in this part of the industry? How soon till we see it in an MMO?

[bhodi]

We're still decades away. There's lots of research going on in this area, but so far, there hasn't really been any breakthroughs. It's a tough nut to crack and I don't think you'll see it in MMOs, period. When software is finally perfected (enough), I doubt the licensing is going to be attractive enough for a company to shell out for it. it's far too complicated for an in-house solution; you simply get too little reward in the MMO environ.

The commercial viability is going to be focused in the disabled persons arena -- people read more than ever, just not books. There's a lot of opportunity there for someone to come in and provide a good text-to-speech converter.

[Morat20]

We're still decades away. There's lots of research going on in this area, but so far, there hasn't really been any breakthroughs. It's a tough nut to crack and I don't think you'll see it in MMOs, period. When software is finally perfected (enough), I doubt the licensing is going to be attractive enough for a company to shell out for it. it's far too complicated for an in-house solution; you simply get too little reward in the MMO environ.

MMO's don't technically need to license it -- users do, so long as it's pretty friendly in terms of interrupting and processing the mic stream. I'd imagine that a pure voice font (not voice to text) would be purchased in conjunction with a mic. The MMO/chatroom/TS/whatever would just get a regular mic feed, as the processing would be done by the client well before anyone else got a chance to use it.

The commercial viability is going to be focused in the disabled persons arena -- people read more than ever, just not books. There's a lot of opportunity there for someone to come in and provide a good text-to-speech converter.

Text to speech is a whole different market, although the technology is roughly the same. Crack one, you're pretty much cracked the other -- or at least done the hard part.

Still, having suffered through the latest automated voice-gunk from the phone company -- they have come a LONG way. :)

[bhodi]

MMO's don't technically need to license it -- users do, so long as it's pretty friendly in terms of interrupting and processing the mic stream. I'd imagine that a pure voice font (not voice to text) would be purchased in conjunction with a mic. The MMO/chatroom/TS/whatever would just get a regular mic feed, as the processing would be done by the client well before anyone else got a chance to use it.

Sure, if people are just playing around with it.

If you want to design a game around the idea, with 'full immersion' voice matching, it's going to be part of the game itself -- I'm not sure how many people would pay for a special mic 'sound like a girl! Sound like your grandpa!' type thing -- how would you market that? What's the target audience, here?

I really don't see it being anything more than a novelty.

[Morat20]

Sure, if people are just playing around with it.

If you want to design a game around the idea, with 'full immersion' voice matching, it's going to be part of the game itself -- I'm not sure how many people would pay for a special mic 'sound like a girl! Sound like your grandpa!' type thing -- how would you market that? What's the target audience, here?

I really don't see it being anything more than a novelty.

I don't think you need to design a game around it -- people do it heavily in Second Life, in RP guilds in WoW and anywhere else, and in any sort of voice-supported chatrooms or games.

Some people would just futz around with it and always be popping up with Robot Voice or whatnot. But I think most that want it -- those willing to buy the mic and all -- would simply set theirs to "James Earl Jones as Kickass Barbarian" or "Hot Valley Girl Elf" or whatnot and leave it there everytime they boot up WoW or Conan or Second Life or their favorite chatroom.

[DarkSign]

You dont need a special mic. You type in what you want to say...and your computer and those who would get the text message would instead hear a voice speaking it.  Basically a girl or old black man version of the robot voice in answering machines.

Although you could take it one step further...voice to text to voice font. Games already have VoIP. Just speak into the mic and it gets translated and spit back out as the voice font version.

[Morat20]

You dont need a special mic. You type in what you want to say...and your computer and those who would get the text message would instead hear a voice speaking it.  Basically a girl or old black man version of the robot voice in answering machines.

Although you could take it one step further...voice to text to voice font. Games already have VoIP. Just speak into the mic and it gets translated and spit back out as the voice font version.

For use in the mass market, you want a true voice font -- something that takes what you're saying and shifts it in a way that sounds realistic. Shifting octaves is pretty easy, and if you can do so in a way that's hard to distinguish from unaltered speech, you're most of the way there for the voice-only people. There's already things like automatic pitch-correcting software and the like (what goes into the studio mics and what comes out on the CD are often WORLDS apart. Whether it's little stuff, like cleaning up wrong notes and just adjusting a slightly off-tune guitar to transforming an off-key singer into perfect pitch, it's almost all processed out the ass).

Doing things like changing accents and the like probably does require going through text first, but I see a lot smaller demand for that -- although I suspect someone will do it because they found it a challenge. :)

[DarkSign]

 

Looks like the HeroEngine already has this voicefont functionality by way of a developer called Diamondware.
Check it out: here

[Ghambit]

Changing your voice before it goes into your VOIP prog. is not hard and can be done at quite high quality (ive got a voice-changng prog. somewhere on one of my boxes).  Why this hasnt been implemented in a game is beyond me.  It should be a slider just like all the other choosable character traits. 

As for text to speech, that's gonna sound weird for a long while and is somewhat pointless if someone can simply talk into their game and achieve much better results.  People dont like to type unless they're good at it; concurrently, they dont like to SPEAK because they feel embarrased and want to maintain anonymity.  Giving them a way to CHANGE their voices (pre-built into the VOIP client and integrated into the game) gives them another cool way to customize their character and makes them more comfortable with speaking in the first place - similar to how people act when they're in costume.

All this tech. has been around a while... just aint used.

As for Diamondware, that tech. has been around a while as well and is basically centered around position-related VOIP. i.e. what/whom you here is dependent on WHERE you are in a gamespace.  The voip client basically automatically connects clients within a certain radius of the user and adjusts output volumes accordingly.  This radius could also increase/decrease dynamically dependent on output volume  i.e. if you scream your radius increases and you can be more heard (like /yell in a chatbox).  Technicallly, you can have thousands of people connected at a time... but in reality, you're not really creating any more connections then a normal run-of-the-mill VOIP server.  I look at it like a software controlled channel switcher. 

Again... it's there, just aint used.   I swear, game designers these days are so behind the times it's ridiculous.   A smart designer would simply code a macro into their resident VOIP program that simply only /whispers voice to people within a certain radius.  In reality, not hard to implement.  OR, you could cheat and create squads (obviously behind the scenes) that dynamically change squadmembers and permissions as people come in range of one another, that way you wouldnt need to redesign much of anything (except to allow people to be in more than one squad at a time).  Throw the voice-changer on the frontend and you're done.

[Fordel]

As for text to speech, that's gonna sound weird for a long while and is somewhat pointless if someone can simply talk into their game and achieve much better results.  People don't like to type unless they're good at it; concurrently, they don't like to SPEAK because they feel embarrassed and want to maintain anonymity.  Giving them a way to CHANGE their voices (pre-built into the VOIP client and integrated into the game) gives them another cool way to customize their character and makes them more comfortable with speaking in the first place - similar to how people act when they're in costume.

I don't think changing the voice will do much to 'unlock' people from not using voice. The actual act of speaking is what opens up the anonymity break, no matter how you sound, emotion and tone still carry through. You can't make people not mumble, or be stop them from being blunt, or have odd enunciation's with words through software. I don't think it's "People will know who I am" that holds anyone back, more "People will hear how shitty I speak" and making them sound like Elves doesn't remove that.


Chris the Stutterer won't Stutter less because he Sounds like Christina the HotElf Stutterer. He also won't be any more anonymous then he was before to the very people he wishes to be anonymous too (the people he wants to talk to).



I do think letting people sound like Hot Elves will make online 'romances' 10x as hilarious and horrifying though.


I also think I use 'quotes' far to often in my writing. 

[DarkSign]

Dont worry. I use ellipses ... too much myself. It just works for a dramatic pause or mental separation.

BioWare's going to use the HeroEngine for their next MMO. If they use the locational sound as well as the voice packs...I'd love to play a game where i heard people's characters talking. I'd definitely break the sound barrier.

The problem then becomes how do you separate TS and Vent from in-game talk. I guess you could have push to talk on both and whichever key you pressed would be the service that got the voice.

One of the ideas i had for a cyberpunk MMO was that you could hack into people's conversations...and better yet guild raid channels. Gawd, it would be magic to hear the GL say "ok go after that healer over by the tree" or "let's meet up at loc X,Y,Z"

[Ghambit]

heheh...  in a world where nothing seems "real" or "concrete," quotations are a must have.  
I still disagree about the voice-changing thing though.  All you have to do is look at how people act during Halloween or costume balls.  It's a night and day difference.  Hell, the purpose of a Masquerade is generally to hide behind an alter ego and do things you normally wouldnt. (masonic orgys with creepy organ music anyone?)

If you give people at least the OPTION to change their voices in-game, it allows them to get more immersed in what they're doing and helps to convey what they'd like to properly.  A stuttering elf is a lot more appealing then a stutturing human in an elven forest IMO.  Those with fears of how they sound on mic suddenly have that fear lifted.  Those who have trouble getting into character suddenly have an easier way to, and feel more comfortable doing it - raising the game experience for everyone.

I've seen total ingrates come out of their shells during roleplay sessions or Halloween.  There's nothing wrong with this and I beleive is a powerful tool to connect players with the game, rather then simply connecting their player characters with the game (as I said in the STO thread).

Last night scribbling into an STO game design I'm working on I thought about the Universal Translator and how it could work in this sense.  Basically, one could alter the translator to transmit whatever they'd like (just like in ST).  If you're a Klingon in-game but just dont have the knack for speaking the language... flip on the translator and have it transmit Klingon as you're speaking English.  Conversely, if you love to speak Klingon you can speak normally and have it translate for those who dont understand.  Hell, you could make yourself sound like a Bynar if you wanted to.

Check out this prog:  www.screamingbee.com
From the few voice-packs I've seen they seem like the best.  Some of their stuff can be add-on integrated into existing VOIP engines in-game as well (Like Wow and EVE).  Fooling around with this program is a helluva lotta fun.  Why someone would say that it couldn't help a game is beyond me.  You'll find yourself fooling with this program for hours and messing with people on TS or Vent... helluva lotta fun.

[DarkSign]

Thanks a lot! You made me buy that :)

But alas im having to run it on my laptop ... and im getting waaaay too much background noise. The mic input and control is for crap, even with an external (albeit cheap) mic.

I'll have to wait till I get my new desktop in a week or so to really play with this thing. Cant wait to wreak havoc!

[Ghambit]

Yah, more than likely the mic on your laptop is omnidirectional, which is bad.  Also, there ARE built-in ways to cut down the background noise both with your pc's sound driver along with the screamingbee driver.  I havent gotten that far yet.

Anyways, I fooled with this some more on my guild's TS server today and really phreaked some people out.   I actually was able to "mimic" certain people in the guild by fooling with some of the preset voices (which you can then save for later use).  I also figured out how to make myself sound like "HAL" from 2001 and proceeded to say "Good morning Dave" over and over.  Making myself a Gnome was easy enough.  They've even got translators that'll change your voice to beeps and clicks, which IMO is where the real tech. opportunities are.

A savvy coder could translate the text-to-speech function to output other languages if they wanted.  I.e. pre-translate the text before outputting the speech.  You could then apply whatever voice pack you wanted to the translation.  Prior to all this you could use a speech-to-text engine so you wouldnt have to type, all the system has to do is learn your voice ala MS Speech.  Untranslatable words would simply come out as your default speech.

Klingon dictionary anyone?

p.s.
I used to have Diamondwares voice changer and it sucked smelly ass compared to www.screamingbee.com

p.p.s.
There is a free "junior" version of the software if you dont feel like paying.

[stray]

I rather like the new Alex voice from OS X Leopard. 

I'm sure that truly good voice synthesis will be here in no time though.

http://www.youtube.com/watch?v=iAXjqV2oU_Q&feature=related

[ajax34i]

I don't know.

Every time someone logs into Vent, I cringe.  That Alex voice sounds nice, but I wonder how it sounds reading some of the "fantasy" names people come up with.  Pick a RP server, do a /who 70, plug the list into Alex, see.

[stray]

I don't know.

Every time someone logs into Vent, I cringe.  That Alex voice sounds nice, but I wonder how it sounds reading some of the "fantasy" names people come up with.  Pick a RP server, do a /who 70, plug the list into Alex, see.

You can tweak the speech rules so that you can get it to say anything correctly...Acronyms, names, etc.. Without the rules, it might get it right 70% of the time. Not that that's much help, but just saying.

[Tarami]

I think you're overlooking the fact that some people just don't like the inane ramblings that happen on pretty much every voice chat. I don't. I stay off them for as long as possible, even for listening. Usually I join the server for looks (if I'm supposed to be there to be a part of the guild or whatever...), then mute it. Most people treat voice chat like an open room, even more open than a text chat, while it's in reality the opposite. People claim that voice chat is efficient, but I don't know. Sure, I'll admit I'm a bit shy, but I'm not sickly so. I don't find it a huge breach of anonymousity to use it either. On the other hand I can't say I find it all that much more social than just chatting using text.

Some people are not comfortable with using voice chat the same way some people aren't comfortable with using phones. It's not really harder than that. Could be noted that I gladly talk face to face, with one or several people in any language I master, and I'm convinced many other "shy" people are. That's not really what it's about, it's the medium. When I chat with text, I leave notes for people to read, I don't shove them into their face. Some people probably find that annoying too, but people are different. I think you'd find that many of those silent people aren't so silent in real life.

[DarkSign]

I don't doubt that some people (not just the shy ones) would shy away from using in-game voices. Who knows...it could catch on, it could utterly phail.
Some will like it and perhaps it could increase roleplaying - which for me would be a plus. I'm not one of those that goes around using "thee" and "thou" and I'd rather kill someone than act like a fruit, but people being in character and immersing themselves in the game is something that might add to the fun factor.

[geldonyetich2]

Personally, I like voice packs.  However, I've noticed that when it comes to Quick-Chat macros in City of Heroes (which have no audio component) and the special emote support in EverQuest 2 (which do) there's a certain matter of teaching the players how to use them.  I'm not sure why it is that players in Tribes 2 have no difficulty picking up the quickchat voice chat functionality but 95% of the players in a MMORPG have no idea what it is.  Even in Planetside, it seems a surprisingly high portion of players only use their voice chat for spamming or not at all.

[DarkSign]

Great examples to distinguish behavior. I'm an avid Tribes player (anyone care for a ski trip?) and you're spot on with that. Perhaps because it serves more of a utilitarian purpose? Or does that say something about the average player of each game?

[damijin]

I wish players found usefulness in quick-chat voice macros in PvP games, since I think it would allow groups of people not on Vent to communicate much better. I use them frequently in CS, and we definitely used them a lot in Planetside -- though it was more or less to spam "You call yourself a soldier?" after running over a teammate with a van.

Battlefield 2 (and on), may have had a good idea with the whole radial menu for voice macros. You often had to access the radial menu to label an enemy as "spotted" which would make them show up on the radar. While you were doing that, you were more likely to learn the other commands, so that if you actually did need to use them, you knew where they were.

Not knowing which keys to press for "I need ammo!" vs "Get down! Sniper!!" is probably the biggest barrier to people using them. The only way I could see mimicking this idea in an MMO would be if there were some usefulness to some of the macros. Like if there were a voice macro for "Assist on my target!" and by doing it, the entire party got a little icon over the head of your target. If something like that were used often enough, people might learn the other macros as well for things like "I need heals!" -- "Somebody wake me up!" -- "I'm poisoned!"

Ideally these voice macros would call attention to the person using them with audio and visuals, highlighting the user. So if someone was asking for heals, any healers in the party would see that persons name flash or something.

[Ghambit]

I yearn for the day when someone speaks on VOIP and the enemy is able to find them because of it.  Currently, Diamond is working on putting that in a game (positional VOIP), like we said prior.

Imagine an emplaced sniper having to whisper or use the customary radio clicks/macros to communicate.  That'd be QUITE awesome.

All this VOIP stuff is win-win... there are no negatives

[KallDrexx]

I wish players found usefulness in quick-chat voice macros in PvP games, since I think it would allow groups of people not on Vent to communicate much better. I use them frequently in CS, and we definitely used them a lot in Planetside -- though it was more or less to spam "You call yourself a soldier?" after running over a teammate with a van.

I'd argue that macros won't change it for most people who would use voip anyways.  If I see someone behind a teammate, or need to immediatly bring attention to something I instinctually go for my click to talk button and say what I need to say.  It just seems easier, and the first thing I think of doing when I see something that needs to be said to my teammates is actually telling them.  In my mind, when I want to tell someone something the first thing I think of is actually speaking to the player to tell him what I want to, not remembering key combinations or mucking around with menus. 

Also you can't the level of detail that a decent PvP team really needs.  Take my previous example where you need to tell a teammate that someone is behind him.  If you just have it say "Someone's behind you!" it's generally useless (except maybe in a 2v2 game) because it doesn't say which teammate needs to be careful, and the teammate who isn't in danger (for that situation at least) might take his attention off what he was doing and can cause issues.  The way to solve that would be to have the macros include the names of players on your team but then you add in the complexity of having to memorize ever changing key combinations.

Another example is position.  When I played CS with my roommate he used the "Enemy sighted" macro a lot.  That does absolutely nothing as 1) I have to know where he is to know if ti's even relevant to me and 2) it doesn't tell me where the enemy he sighted is.  I don't really see an easy and intuitive way around these issues. 

[Fordel]

Tribes 2 allowed context alerts tied to the keybind macros.

A Good example was the general "Attack my Target" bind. If you were hovering over a turret, it would automatically mark the turret for others to see through the HUD/Map and the voice message would say "Attack this turret" instead of just generic target.

Something about the Tribes2 chat keybinds was very intuitive. Everyone used them, from the top ranked clans to the pub CTF Katabatic servers. I've never really used them much outside of Tribes either myself, despite liking them so much in the Tribes games. I can't put my finger on WHY they were so effective compared to other games, but they were.

[DarkSign]

I yearn for the day when someone speaks on VOIP and the enemy is able to find them because of it.  Currently, Diamond is working on putting that in a game (positional VOIP), like we said prior.

Imagine an emplaced sniper having to whisper or use the customary radio clicks/macros to communicate.  That'd be QUITE awesome.

All this VOIP stuff is win-win... there are no negatives

/agreed

Some retard would say that it was being "too realistic" especially for the casual player, but fuck that. The more it works as meaningful gameplay, the more everyone will pick up on it as useful.

[Samwise]

Didn't some game already do that?  Was it Splinter Cell?



Cool idea, but if using the in-game voice communication is too much of a disadvantage people (at least the "hardcore") will just use external programs.

[MikeRozak]

FYI, I'm working on an indie MMO (kind of) with both voice chat (that relies on voice disguise/transformation) and text-to-speech. You can try it out at www.CircumReality.com , warts and all.


As for some specific technologies mentioned so far:

- Using speech recognition to transcribe the speech, then pipe the text (perhaps with transplanted prosody) to another player's PC, and then text to speech to resynthesize it - While this is possible, speech recognition for general purpose speech (including shouting) just isn't accurate enough. It'd be like the game "Telephone operator" or "Chinese Whispers".

- Disguising a male's voice so it sounds female (not just higher pitch) is possible (and vice versa), although far from perfect. You can do this in the voice chat in my game, or with the wave editor I include in www.mXac.com.au/m3d . It WON'T change the accent though. You'd need the above SR technology to include an accent change.

- Personally, I'm a fan of text-to-speech, particularly for NPC dialogs. I use it in my game. It's not as good as real voice acting, but it's a hell of a lot cheaper (important for an indie). The biggest impedement to TTS quality is prosody (pitch, inflection, timing).

- It's possible to make a text-to-speech voice of your own voice. This requires that you record around 1000 sentences. It'll work with 100, or even as few as 10, but the quality is signficantly less. Optimally, you'd record 10,000 sentences (takes forever). The tools for this are in www.mXac.com.au/m3d

- I see in-game voice-chat and TTS more as casual-player features.

- Enemy detect VOIP - I have the ability in the scripting language, but haven't implimented it yet since my game doesn't include that much combat.

- Players can alternatively type (and not use voice chat) and other players will hear what's typed using text-to-speech. (Other players can mute text-to-speech and just read the text too.)



I can go into excruciating detail about the technology if you want.

[DarkSign]

Your project really looks great. Im going to take a deeper look at it for sure.

If you're inclined I'd love to hear more about the ongoing discussion.

[MikeRozak]

Your project really looks great. Im going to take a deeper look at it for sure.

If you're inclined I'd love to hear more about the ongoing discussion.

Which part of the discussion? It seems to be pretty wide-ranging so far.

[DarkSign]

Oh. Sorry. The technology involved in your project.

[MikeRozak]

Oh. Sorry. The technology involved in your project.

Still a fairly broad question, but I'll go into more specifics:

I've written my own voice encoding algorithm that takes PCM and outputs a voiced-and-unvoiced spectrum, as well as F0 (pitch), and phase for the harmonics. In other words, F0 described the pitch from the voice box, and the voiced + unvoiced spectrum describes the harmonics of the glottis being filtered as they pass through the throat, nose, and mouth.

To change someone's gender from male to female, you first need to raise their F0, since females have smaller voice boxes (with higher pitch). However, if you do only this, then your voice sounds like Michael Jackson, or even falsetto. A woman's throat and mouth are smaller too, so the algorithm bends the spectrums up (simulating the smaller size). It's not a great female voice (see comment about individual formants), but it's not recognizable as your own.

The algorithms can non-uniformly bend the spectrums to simulate different-shaped mouths (to an extent), creating a variety of different male and female voices. It's not good enough that you can exactly copy a famous personality's voice though. (To do the job right, the algorithms needs to identifiy individual formants and modify those is isolation... which I haven't bothered with yet.)

Alternatively, if you change the voice-box sound, you can create really odd sounds, like speaking cellos, or bell-like voices.

Players can customize all this.


The same voice encoding is used for VoIP, which I treat differently than most MMOs. Current MMO players want to use VoIP as an efficient way to (a) talk to their pary members, and (b) conduct raids. Therefore, VoIP is configured like a radio headset, with channels and whatnot. Importantly, only one's friends/guildmates can hear what's going on.

Because of the nature of my game, while I expect parties to form , I don't expect guilds. Therefore, when a player speaks, it's broadcast to every other player in the room (unless they whisper). If they "shout", then the audio is broadcast to players in neighboring rooms also (with 3D sound). Because players can hear others talk, they shouldn't get the feeling of being alone in a world of silent strangers.

And if the PCs speak a different language, the speech is backward masked at random points. (A "whisper" causes their voice to be modified so it sounds like it's whispering.)

For players that don't wish to use voice chat, they can select a text-to-speech voice (with voice modifications) to use.

I could get into how text-to-speech works, but that's a fairly lengthy discussion.

Any specfics?