8
\$\begingroup\$

You may be familiar with ryhtm games as Dancing Line, Geometry Dash, or even Guitar Hero. They have something in common : every single action in the game is synchronized with the music. My purpose is to make a game like Geometry Dash where the player has to do some actions in rythm while the character scrolls automatically in the level. I have a few questions about this type of game :

  • Do I have to keep my FPS constant as it has to be always synchronized with the music ?
  • How can I build my levels with a music in mind to make sure that it will be synchronized ?

I am working on Unity using C#.

\$\endgroup\$
17
\$\begingroup\$

I don't recommend using Time.deltaTime to track progress through playing the song as described in another answer.

It's great for smoothing & interpolating effects to play out in realtime, but when you want to match the song's time you should ask the song.

In Unity, you can measure an Audio Source's progress through playing a track directly using AudioSource.time or AudioSource.timeSamples to get the precise playhead position. (Note that the latter measures in samples, which may vary in rate per second depending on how you've compressed your track)

This way, you don't need to run your own separate timer and hope they stay in sync. You can wire up game triggers of beat matching effects to run off of the song's own timeline.

For layering gameplay sounds onto the music, you can use AudioSource.PlayScheduled to ensure your sounds land exactly on the beat, as even a small mismatch can be noticed

\$\endgroup\$
  • \$\begingroup\$ Any ideas how to sync graphics to the AudioSource? F.e. I want to show an explosion or collision connected to the music. I want to show it exactly when sound played. \$\endgroup\$ – syabro Aug 26 '18 at 18:48
  • \$\begingroup\$ That's what all of the answers here are about. Did you find any trouble putting the suggestions in these answers into practice, @syabro? \$\endgroup\$ – DMGregory Aug 26 '18 at 20:38
2
\$\begingroup\$

As an alternative if you are fine with not working from scratch.

There are ready made assets already (under Editor Extensions/ Audio like https://www.assetstore.unity3d.com/en/#!/content/54639 (paid) or https://www.assetstore.unity3d.com/en/#!/content/39835 (free)) where you can sync music tracks to event. In that example the weapon is shooting on the beat of the music.

If you want to learn how they work, try 1-2 of the free ones and look at the source.

\$\endgroup\$
  • \$\begingroup\$ I saw a presentation from the Koreographer devs a little while back - looked like a great piece of tech! At the time its audio analysis capabilities were limited, but if you knew in advance the track you're working with, you could use the tech to very precisely schedule game events to specific beats/moments in the music. I don't know whether they've added the ability to do more analysis automatically since that demo. \$\endgroup\$ – DMGregory Aug 1 '17 at 16:03
1
\$\begingroup\$

Unity already decouples gameplay from framerate. If you remember to always use Time.deltaTime in your Update-functions, the actual graphics framerate should not affect gameplay speed. So when you start the game and the audio track at the same time, they should stay synchronized.

Regarding designing levels around music, there are two approaches:

  • The manual approach. Meticulously hand-craft levels around specific audio tracks. Having the original sheets music at hand can help, but it's also possible to do it just by ear.
  • The procedural approach. Use audio analysis algorithms to auto-generate levels around audio tracks. The most simple is to just look for volume spikes. That should already allow you to identify drum beats in many songs and spawn obstacles accordingly. A step further is detecting volume shifts separately in different frequency spectrums. This makes beat detection more reliable and allows you to spawn different obstacles for different pitches. But that's still just scratching the surface. Audio analysis is a very wide and interesting field. People wrote a lot of scientific literature about it.

    How deep you need to dive into this field depends on how ambitioned you are. Some games are satisfied with just detecting the beats and randomly-generate all the rest of the level design(Crypt of the Necrodancer). Others try to put as many aspects of level design under the control of the audio as they can (Audiosurf).

Many rhythm games use a hybrid approach. You generate a first draft of the level using a generator and then tweak it by hand to make it more playable.

\$\endgroup\$
  • \$\begingroup\$ Thanks for your answer, I'll look into it ! Is Time.deltaTime really precise and there won't be any time lag created with long musics ? \$\endgroup\$ – Shashimee Aug 1 '17 at 9:15
  • \$\begingroup\$ @Shashimee It should be precise enough. If you want to be really sure to not lose anything to rounding differences, use Time.time instead. \$\endgroup\$ – Philipp Aug 1 '17 at 9:18
  • \$\begingroup\$ I would recommend procedural approach because it opens many more possibilites for game to develop. Players can add their own tracks, you can change mesh and all the other stuff to the beat, like particles. Also, you won't spend a loooooot of time on designing levels. \$\endgroup\$ – Candid Moon _Max_ Aug 1 '17 at 12:51
  • 1
    \$\begingroup\$ @Philipp unfortunately Time.time can be even more susceptible to imprecision if the game has been running a while. Since it measures game time since startup, rather than since the start of the track or the start of the level, after playing a song or two it will have lost as many low bits of precision as a naive sum of deltas, and the error can continue to increase if the game is left running / slept-restored repeatedly. \$\endgroup\$ – DMGregory Aug 1 '17 at 13:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.