Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

Do games use multiple threads to play music and other operations besides things like server connections and sockets?

And is that why when the game freezes (for any reason) the music can sometimes keep playing, because only the game thread is slowing down?

share|improve this question
2  
Some games will, some games won't. I know I've noticed more often than not that when a game freezes the music just stutters. –  Joseph Mansfield Sep 11 '13 at 16:23
1  
What's the problem you're trying to solve? –  Byte56 Sep 11 '13 at 18:29
    
I was just wondering what kind of things threads can be used for, and this seemed to be a good one to ask about, and because I couldn't figure out another way to play music while the program was running –  Joseph Dour Sep 11 '13 at 20:59
1  
Have you heard the proverb, "If all you have is a hammer, everything looks like a nail"? Typically, when you get a new tool, you don't go looking for problems to solve with it. This often leads to using the wrong tool for the job. Instead, you should learn about your new tool, so that when a problem arises you know which tool would best solve that problem. –  Byte56 Sep 11 '13 at 21:26
    
I've written A/V and networking libraries before, and yes they do often use threads to continually read/write buffers. Sometimes these libraries give you a buffer to read/write data from/to, sometimes you pass in a callback; refer to your libraries and don't worry about how they are implemented. Because if you are, you're not doing games dev :) –  congusbongus Sep 12 '13 at 3:27

3 Answers 3

up vote 10 down vote accepted

As always, it depends. Audio is a very complex topic, and the details can be vastly different from API to API, and from system to system.

Introduction to audio processing

Note: for all the examples below, we will be considering linear PCM, at 44100Hz (samples per second), with 16-bit samples, and two channels (stereo). This is a very common configuration.

Audio processing usually consists of two components: a data producer and a data consumer.

The consumer is normally the audio driver, or audio hardware. In order to play sounds, the data consumer will need a constant stream of bytes. The exact amount of bytes that a consumer needs is defined exactly by:

bytes_per_second = samples_per_second * bytes_per_sample * channels

So, for 44100 samples per second, 2 bytes per sample (16 bits), and 2 channels (stereo), this equates to 176400 bytes per second.

The provider is normally your program. Your task is to provide data for the consumer. Your task consists of creating exactly 176400 bytes per second of audio you want to play.

In a perfect world (and real-time environments), you must be able to provide exactly 44100 samples per second, which means 1 sample every 22675.7 nanoseconds.

These samples would be received by the consumer and immediately sent to the speakers, so it is crucial that you send 1 sample exactly at every 22675.7 nanosecond mark. This is very difficult to get right, and if you fail to provide the required data at the exact time mark, the consumer will not know what to do, and the results will not be what you expect (more on this later).

Buffering

To solve this problem, most audio processing systems work with a FIFO buffer (like a queue) for the provider to store data on one end as it sees fit, and the consumer will consume as it sees fit. The idea is that the producer will in average be a given amount of bytes ahead of the consumer, so the producer has relaxed restrictions on the exact timing at which the bytes have to be produced.

For example, if you make a system in which the producer is 256 samples (1024 bytes) ahead of the consumer, the producer can be up to 256 samples behind in its obligation to produce bytes, and the consumer will still have audio data to play. 256 samples is approximately 6ms worth of sound data, so your program can stall for up to 6ms before the consumer will notice.

Make no mistake, you still have to provide an average of 176400 bytes per second, and if for some reason you fall back, you will have to catch up at some point in the future. The purpose of the buffer is simply to relax the timing at which you have to provide the data, not reduce the amount of data you have to send.

Delay

However, buffering creates a problem. If the producer is 256 samples (6ms) ahead of the consumer, any audio you queue to play will only be heard on the speakers 6ms after you enqueue it. This is called "delay".

In general terms, the greater the amount of bytes the producer is ahead of the consumer, the more slack the producer will have to fall behind, but also the greater the delay between the moment at which the program decides to play a sound, and the moment at which the sound is actually played on the speakers.

Actual systems

In practice, audio systems are much more complex, with large chains of producer-buffer-consumers, filters, splitters and mixers, called the "Audio graph". However, when playing audio to speakers, there will be a final consumer, which sits right behind a DAC, and will in fact provide exactly 1 sample exactly every 22675.7 nanoseconds. The circuit that does this is commonly outside of the CPU, and has its own clock, so no matter what happens on the CPU, this guy will work in perfect sync.

Why is audio programming difficult?

Even in the most simple audio graphs, you still have to buffer audio to be played. The large variety of hardware that exists, with many different types of drivers, on so many systems, means that there are way too many variables to get your sound right at the right time.

Delay is directly proportional to the length of your buffers, as well as to the amount of elements your audio graph has. You usually want to make the buffers as short as you can, but in modern OSs, where your program only has a very small and unpredictable share of CPU to use, creating a program where the audio is very responsive, while not leaving the buffers empty is very difficult.

Onto your question

So, what happens if your program is stuck doing something, and you can't fill the buffer? Well, it depends on how your system is built.

Low-level APIs

If you made your program so you're actually responsible of filling the buffer, like when you're programming with DirectSound, OpenSL, Core Audio or other low-level audio API, and the thread responsible for filling the buffer stalls, then you won't be able to keep on filling the buffer. That's what the buffer is for, so initially nothing will happen. The sounds you've already enqueued will be played as requested.

If your program is still stalling, and the buffer runs empty, then what happens is up to the audio driver/hardware, and is never good. Many audio systems will just fill the output with zeroes, so the audio will go silent. If your program manages to recover shortly after that, this will manifest as an audio "click", also called "audio glitches".

However, other audio systems may fill the output with garbage, so you may hear white noise. Many other systems with circular buffers will just loop through the buffer, so you will hear the last few milliseconds of successfully queued bytes looped over and over again.

High-level APIs

If you're making your program with an API that manages the enqueueing for you, like XAudio2, SoundPool, AVAudioPlayer or other high-level audio API, then the results will unsurprisingly be dependent on the API. Many of these APIs will manage the sound queue on a separate thread with higher priority, so as long as that thread doesn't stall, you won't hear glitches.

share|improve this answer
    
This is correct and thorough, but the major ideas might be a bit tough to grok if you're not already familiar with the subject material. Maybe some more organization, and headings to give future readers some anchoring concepts? –  michael.bartnett Sep 12 '13 at 3:46

Yes. An audio device plays a fixed buffer size of samples then request the next buffer. Also generally music is streamed from disc so there's another thread handling data streaming.

share|improve this answer

Sound back-ends usually use threads to make sure that the sound/music is played continuously and not interrupted or distorted when larger calculations are done. However you won't need to deal with those yourself, as they are implemented by the backend, for example OpenAL.

There is one additional issue that can be solved via threads, namely the streaming of the music from the hard disk, this however, isn't as time critical as the playing of the sound itself and mostly depends on how much you buffer, if you buffer multiple seconds at a time and have a recuring loop such as the game loop a dedicated thread should not be necessary.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.