1
\$\begingroup\$

In the game I'm working on I have to play several short consecutive voice-clips to form a complete sentence. Example (each [] bracket is a different voice clip):

[Bob here,] [we're at] [some town] [and are on our way to] [some city].

Stitching together different voice-clips like this makes it sound stilted and disconnected. This is because there are unnatural pauses when switching clips, and the pitch and tone of the speaker changes.

My current efforts include two methods for removing the unnatural pauses:

  1. starting the next clip early if a silence is detected at the end of the preceding clip
  2. skipping the first few milliseconds of the new clip up to the first detected 'sound'.

These work OK at removing the unnatural pausimh, but detecting what 'silence' is is difficult, especially when dealing with multiple voice-actors and microphones.

How could I make stitching together voice-clips sound more natural? Any advice would be appreciated. This has to be done in real-time inside the game (I'm using Unity), and can't be pre-processed or done ahead of time.

\$\endgroup\$
6
  • 1
    \$\begingroup\$ Detecting the boundaries of speech is, I think, a Hard Problem. If you're absolutely sure you're unable to solve the whole problem offline, can you solve part of it? E.g. can you process the clips to conform to the same definitions of 'silence' and 'max volume'? (i.e. remove the mic from the question) \$\endgroup\$ Sep 26 '19 at 20:14
  • \$\begingroup\$ @ChrisMills-Price Yes, the clips could be normalized to have the same levels of volume, which should make detecting 'silence' easier. That's a good idea. \$\endgroup\$
    – DeeCeptor
    Sep 26 '19 at 20:38
  • \$\begingroup\$ With a lot of effort pitch could be interpolated on the fly, but tone is a human emotional construct. I really don't think tone matching on the fly could ever be sone seemlessly. \$\endgroup\$
    – Weckar E.
    Oct 23 '20 at 7:10
  • \$\begingroup\$ I don't think I have ever heard a game pull this off successfully. Not unless the character speaking is a robot or computer, where unnatural pronunciation is actually aesthetically appropriate. \$\endgroup\$
    – Philipp
    Feb 18 at 14:04
  • 1
    \$\begingroup\$ @Rohan The voice stitching in HL1 only works for the soldiers because they shout everything they say. They emphasis every individual word, so it's not that noticeable. There is also a lot of distortion on their voices, obscuring little issues even more. The announcer has even more words than the soldiers and uses even more stitching, but it's a computer voice so the unnatural pronunciation works for it. \$\endgroup\$
    – Philipp
    Jun 18 at 6:51
0
\$\begingroup\$

you can assign multiple audio sources to one object. for exxample maximum 10 phrases in a sentence. so you have:

AudioSource[] Phrases;

void Awake()
{
  phrases[0] = GetComponents<AudioSource>()[0];
}

then you can get length of clip that check what is best place to start new phrase audio source. like this:

  phrases[1].Play(phrases[0].clip.length +or- someuLongValue);// for better syncing
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.