8
\$\begingroup\$

Let's take a situation where you have an RPG with spells and each spell animations have a different number of frames and they have very different requirements for sound effects. Let's assume that each spell has only 1 continuous animation associated with it (as opposed to multiple modular pieces that are used to form a full animation) ala the old 16 Bit Final Fantasy games.

The only way I can think of in order to make sure that the sounds and animations sync up is to:

  • Get the number of frames of an animation.
  • Get the time between each frame of the animation. (if it's 30 fps then it's 1/30th of a second per frame.)
  • Then create a sound file that is exactly the same length as the animation.

So this means that if an animation is 5 seconds long, running at 30 fps, with a total of 150 frames, the sound file will also be 5 seconds long. If the animation should have an "impact" sound on the 30th frame, that means the sound file will include the sound of impact at the 1.0 second mark.

At the end, we start the animation and the sound effect at exactly the same time and hope that the frames and the sound sync up.

This sounds like it could pose problems when frames are skipped or something happens during the animation and the sound gets played slightly too soon or too late, and will cause the sound and the animation to go out of synch. Is this the best approach or are there usually a better way that I'm just not seeing?

The answer doesn't necessarily need to be for Cocos2D specifically if it's conceptual, but if there's a specific solution for cocos2d, I'd love to hear it.

EDIT: I just also realize that with this method, if we happen to go in and adjust the number of frames or timing of the animation later, we MUST also go back and change the sound file. This sounds like a terrible cause for human error (forgetting to update sound files after animation change.) I hope there are better methods out there.

Improve this question
\$\endgroup\$
2
  • \$\begingroup\$ this is why constant time game loops are handy: then you don't need to worry about animation going out of sync \$\endgroup\$
    – ratchet freak
    Nov 30, 2013 at 22:16
  • \$\begingroup\$ @ratchetfreak I believe cocos2d will manage the timing of the animation correctly. If I create an animation and tell cocos2d that I want exactly 1/30th of a second between frames, it will ensure this and skip frames if the performance isn't good enough. This ensures that the animation is completed at the right timing (i.e. in constant time.) Given that, are you saying that the method I outlined above is the right way to go? \$\endgroup\$
    – Jamornh
    Nov 30, 2013 at 22:32

2 Answers 2

Reset to default
6
\$\begingroup\$

Do it via events.

Spell begin is an event. Start playing the sound for that event.

Enemy getting hit by spell is also an event. If the enemy is further away and you Throw a dart, for example, you only play the second sound (dart hitting) once the dart reaches the target (if you consider Throw as a spell).

If you need to tie it to a frame (so for example, play the "explode" sound 30 frames from now regardless of real-time frame rates)), the easiest way to do so is using callbacks. Callbacks are just "blocks of code you schedule to run in the future". Here is an example of my callback creation setter:

- (void) addCallback:(Callback*)callback inHowManyTicks:(unsigned long long)execTicksIntoTheFuture
{
  callbacks.push_back( new TimedCallback( tick + execTicksIntoTheFuture, callback ) ) ;
}

A TimedCallback is just a wrapper around a std::function (or you could use an Objective-C ^{block}. The std::function is executed when its frame is up.

Another way (less global) is to include events in your animation. So, if you need to play specific sounds on specific animation frames a lot, then store a map<int,Sound*> in the Animation class. On each frame of the animation, check if there is a corresponding Sound* to play for that Frame.

You could also store a map<int, std::function>, in the Animation object, which would allow you to callback a function during the animation.

Improve this answer
\$\endgroup\$
7
  • \$\begingroup\$ I think you may have misinterpreted the question slightly. Your method would work for short sounds meant for "punch", "hit", "kick", "shoot" type that lasts no more than a split second where synchronization isn't an issue (you can just play the sound effect "on event" as you suggested.) However with a long animation + sound (i.e. armageddon, where 5-6 meteors fall from the sky and hit the ground at different frames, but are part of 1 sprite animation [like how final fantasy does them] will only have 1 start event, not one per meteorite) this method won't ensure synch right? \$\endgroup\$
    – Jamornh
    Dec 1, 2013 at 8:24
  • 3
    \$\begingroup\$ @Jamornh In your meteor shower example you shoot an event: for each meteor starting to fall, for, each meteor hitting the ground, maybe one for the characters struggle casting the spell. Using this solution you can even change the number of meteors and have no problems with audio. \$\endgroup\$
    – akaltar
    Dec 1, 2013 at 10:30
  • 1
    \$\begingroup\$ @Jamornh You could also queue an event for "play explosion sound 30 frames from now", the easiest way to do this is using a callback function. I'll detail this in my answer. \$\endgroup\$
    – bobobobo
    Dec 1, 2013 at 14:22
  • 1
    \$\begingroup\$ @Jamornh The animation doesn't necessarily have to be modular to be able to shoot multiple events(at predetermined times). In your effect editor you could just say, play boom sound at frame 32. \$\endgroup\$
    – akaltar
    Dec 1, 2013 at 15:10
  • 1
    \$\begingroup\$ Yes, that would work. If you were using JSON, I'd suggest a datastructure like { 'images':'sprite-%02d.png', 'beginRange':1, 'endRange':32, 'sounds':{ 0 : 'startSpell.wav', 30 : 'impact.wav' } }. If your game is very early stage and compile time is still low, you can start by hardcoding the data structures in to see if it works. \$\endgroup\$
    – bobobobo
    Dec 1, 2013 at 20:42
1
\$\begingroup\$

The way i do it is make custom event listeners for my animation class and have them control my sound. so if my animation has started callback.start(); and start my sound in that method. if my animation was paused do callback.pause(); and pause the sound. when the animation has ended you callback.end(); and have the sound end as well.

but for perfect synchronization i would go as far as counting sound frames and sleeping(pausing) my sound if it gets to far a head and do the same for my animation.

I have never had to do that until this day because the first suggestion satisfies my needs just fine for now.

Improve this answer
\$\endgroup\$
2
  • \$\begingroup\$ Could you elaborate on how you would count a sound "frame"? Do you mean counting the number of miliseconds that has passed since the sound started and perform a count per each predefined interval? \$\endgroup\$
    – Jamornh
    Dec 1, 2013 at 8:25
  • \$\begingroup\$ no no , i quite literally mean the frame the sound is on. I dont know how you do it in cocos2D but in java sound there is methods for getting the sounds framecount and current "frame",there is also a way to set the current frame to a certain . im sure if you look into it you could find similar variables within your sound interface, but like someone pointed out it is best to have your updates handle animation time ect. so that you dont have make such syncing hacks. \$\endgroup\$
    – Jonathan Camarena
    Dec 1, 2013 at 10:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .