Maximum audio delay before the player notices?

Question

Given some event in a game, what is the maximum delay to producing audio that the player will properly associate the audio with that event (and not perceive lag)?

Answer 1

The following result are calculated for lip synchronization which is concidered to be "the most noticeable a/v sync error".

---

Wikipedia says

For television applications, audio should lead video by no more than 15 milliseconds and audio should lag video by no more than 45 milliseconds. For film, acceptable lip sync is considered to be no more than 22 milliseconds in either direction.

---

The Media and Acoustics Perception Lab says

The results of the experiment determined that the average audio leading threshold for a/v sync detection was 185.19 ms, with a standard deviation of 42.32 ms

---

The ATSC says

At first glance it seems loose: +90 ms to -185 ms as a “Window of Acceptability”

and

• Undetectable from -100 ms to +25 ms
• Detectable at -125 ms & +45 ms
• Becomes unacceptable at -185 ms & +90 ms

(– Sound delayed,+ Sound advanced)

---

To conclude

The results aren't so far from one another. It seems that the maximum acceptable delay is around 150ms, which is 9 frames at 60 frame per second.

Answer 2

It depends of the event

Feeling that, say, an explosion you see and hear is a single event will have the tolerances described in other answers - no more than ~50ms; some people may be more sensitive (e.g. musicians), so I'd suggest to aim at 30ms or no more than 2 frames at 60fps.

I believe that the perceived distance should affect those tolerances. People expect far sounds to be slightly delayed, since in real life sound lags sight by approximately 1ms per each foot of distance. So an explosion on an zoomed out RTS game 'map' might have a larger tolerance for sound lag than the player firing their own gun in an FPS.

Specialized cases, such as having a proper feel for a music/rythm game may require much tighter tolerances, 15-20ms or even lower - for example, if the player hears both the "input action" such as singing into a mic or banging a plastic instrument, and also a sound generated by your system for the same event, then a 50ms lag will cause the "original" and "played" sounds to mix weirdly.

In addition, keep in mind the lag between start of the audio file and the "event" inside that audio file - in many audio clips, the "event" won't be right on the edge, you may have a sound of a lightning strike where the 'strike' happens 200ms after the beginning, which would be obvious to everyone, and pretty much all sound files, even a drum-hit, will have some delay there.

Don't measure averages - look at worst case

Sight&hearing are deeply connected in human perception, and if one of them stutters relatively to other, then it will be perceivable. It's not okay if most of the time it's very fast but occasionally there's a 0.2 second delay while something is loading - people will notice such situations. This is why audio is often kept running on a separate thread, isolated from the other activities and just getting rapid notifications on what preloaded clips should be played.

Answer 3

Any situation where a player causes the sound (music games, guns in FPS) will need very low delay as the player has sent an impulse to make it happen at that moment, so as with a musician hearing their instrument delayed, will be particularly aware of very small delays. Sound engineers fret about recording delays below 5 mSec ruining the "groove"

The Journal of the American Academy of Audiology states that people (not just musicians), when listening to their own voice delayed, are aware of delays as short as 3mSec, and a delay of longer than 10 mSec was objectionable 90% of the time .

Humans use the time delay between their ears for directional information, and thus must be able to process and extract information from delays below 1mSec

The 185.19 ms quoted above is irrelevant as it is referring to a leading sound error, and anyhow, to what people found acceptable when passively watching a film, not actively engaged in a game.

Answer 4

The accepted answer here mainly discusses perception of audio synchronization in passively watching video. In these cases, the audience can't easily pin down exactly when the audio should play except by attending to telltale signs in the video. This means they have limited anticipation of the sound.

There are two important cases in games where this low-anticipation assumption doesn't hold:

1. When the player themselves caused the sound (as SamB points out), so from the moment they form the intent to press the button they know exactly when they expect to hear the sound.

2. When the sound is supposed to land on on a periodic beat, as in music games or anything with a ticking timer/counter, this rhythm allows the player to anticipate the next sound and notice if plays out of time.

In this talk from GDC 2013, Mathieu Pavageau argues that players can perceive differences in synch precision above about 5ms, much less forgiving than the examples from lip synching would suggest. Check out the sections "Time Perception Examples" and "Example of Ubisoft Games" to hear it for yourself. You can hear the Rayman Origins menu doesn't sound "laggy" per se when synched within 16 ms (video frame), but when synched within 5 ms it sounds noticeably better & tighter.

Pavageau advocates using a low-level audio callback to get this kind of sub-frame precision if you want tight-feeling rhythmic gameplay of this variety.

Answer 5

For games which require a person to react to audio cues, every millisecond by which the sound is delayed will cause the person's response to likewise be delayed. Someone who is simply watching a movie or cut-scene may not notice too much if the audio and video aren't exactly in sync, but it's often important and sometimes critical that audio be in sync with what the player is expected to be doing.

Answer 6

In theory, everything above 50ms can be noticeable when it comes with it's association to pictures, at 25ms you can start hearing a sound and its delay as two separated sounds, so I would say I'll highly recommend you stay under 50ms and if you can even stay at something from 5ms to 15ms it would be really nice.

I hope this will help you!

https://en.wikipedia.org/wiki/Delayed_Auditory_Feedback