Serialization/Deserialization Solutions for Loading & Playing Audio Files

Question

I'm creating a music game in unity, and the first issue I encountered was that I had no way to load music into the game from a file. I tried using a WWW to load the mp3 files, but in my research I came across multiple sources that told me that this method would not work, and I was unable to get it to work in my own practice.

So, I found the Naudio library, which allowed me to convert the MP3 files to an array of floats which I then used AudioClip.SetData() to create an audio clip with. To avoid having to do this conversion every time I wished to load a new song, I thought to then serialize the converted audio data to a binary file which I would then deserialize to create a new audio clip when the song is requested, and that audio clip is then stored in a list for if I need it later during runtime.

This worked for a long time, but now I'm almost ready to ship my game and I'm realizing that the file sizes for these binary files are massive - upwards of 100MB in most cases of a 4 minute song. This is obviously unacceptable considering that the file size of a regular MP3 song of that length is less than a tenth of that.

So, I looked into using a DeflateStream when serializing and my research brought me to the DotNetZip library, which someone recompiled for use with Unity here. However, this did little to remedy my issue because 1) The compressed file sizes were smaller, but not small enough (still around 50MB for a 4-minute song), and 2) The time it takes to serialize the file skyrocketed from 3-5 seconds to upwards of 30 seconds.

Perhaps I'm looking at this problem the wrong way. Can anyone offer me some insight as to how I can load and play mp3 files at runtime?

Answer 1

Unity internally uses ogg vorbis for audio - whenever you drop an mp3 into your Unity project, the editor converts it to vorbis and uses that. Importantly, this means Unity can natively decode ogg vorbis audio files at runtime including streaming them via WWW.

So, rather than essentially creating a new binary audio format (both ogg and mp3 do some very complex audio processing to achieve the level of compression they reach), use the ogg decoder to your advantage. Convert your mp3's to ogg and stream those in via WWW instead.

If however you need to stream mp3, you could include NAudio in your project and run the mp3 decode at runtime but instead of decoding the entire file, use a streaming AudioClip instead. Unity AudioClips have a 'stream' mode which raises an event called PCMReaderCallback - this event just asks for the next chunk of floats right when it needs them. Hook in the mp3 decoder here for minimal memory use and almost no startup delays as it'll be decoding on demand instead.