So, I've been looking for a way to detect beats on a streamed song on mobile(like listening on Spotify or Youtube etc.) I've found the most used methods are either using already known songs or using spectral flux analysis, which would require knowing all the song in advance.

Since both my audio and my programming knowledge are limited I'm struggling both in implementing this algorithm (I've tried Allan Pichardo's Beat detection algorithm ) and in thinking of another way of detecting onsets in an unknown song.

My goal is to be able to detect if the player moves "in sync" with the beats of a song, with a given margin of error such as in Crypt of the NecroDancer .

I've also given thought to a machine learning approach, where a trained model could tell if a given piece of a song is or not an onset, but don't know if it can be implemented on mobile, and if there is a dataset available to train on.

The final question would be, is the spectral flux analysis still the way to do this or would there be easier, more efficient methods?

  • \$\begingroup\$ Where specifically have you struggled in using Allan Pichardo's method? The more precisely you can describe the issue, the better we can target solutions to help. \$\endgroup\$ – DMGregory Feb 28 '20 at 16:16
  • \$\begingroup\$ @DMGregory In finding a good threshold value that works for more than one musical genre at a time \$\endgroup\$ – Mihai Barbu Feb 28 '20 at 16:24
  • \$\begingroup\$ That sounds like something to ask an expert in Audio, rather than generic game development. Have you reviewed existing Q&A on the topic on the Sound Design StackExchange? \$\endgroup\$ – DMGregory Feb 28 '20 at 16:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.