# How can you sync video files with animations within Unity?

I'm wondering if anyone might be able to offer some advice regarding an issue I am currently facing in trying to sync video files with animations within Unity.

Ideally my aim is to have a video file play whilst an animated camera (exported from blender) moves in sync with the rendered video file. I've nearly got this working, however there is a very slight delay between the two in which at certain points (quick rotations of the camera) the animation lags behind the video.

I have attempted to solve this issue by creating a script in which for every frame of the video, the animation component is accessed and the animation file frame set to its equivalent time with regards to the video player position. To do so I made use of 'animationState.normalizedTime' (mapping the video frame from its current position to a value of 0 - 1, so as to fit within the requirements of the normalized time function).

I'm also trying to make use of 'videoPlayer.frameReady' in order to set the animation with each new frame, with the idea being to try to only set the animator once a new frame is displayed.

The result I'm currently getting is that the animation is linked to the playback of the video, and will match up regardless of speed of playback, however there are parts in which the animation does not match up to the video. These are particularly noticeable when the camera makes a sudden movement, or when it is rotating. My current thoughts are that perhaps this could have something to do with camera easing settings (from blender to unity) or with the animation component settings. I've tried to remove animation compression and keyframe reduction, however am still having these issues unfortunately.

Does anyone have any ideas as to how this might be solved?

I've included my code below :

public class animationSync1 : MonoBehaviour
{

// Animation component
private Animation animator;

// Video player (add via inspector)
public VideoPlayer vp;

// total frame count of video
private float totalFrame;

// current video frame
private float currentFrame;

// animation frame count
private float animationFrame;

// Start is called before the first frame update
void Start()
{

// Animation component
animator = GetComponent<Animation>();

// get total number of frames from video file
totalFrame = vp.frameCount;

// start video playback
vp.Play();

// begin playback of animation
animator.Play("Camera_entrance|Camera_entranceAction");
// set animation speed to 0 in order to set the animation frame point manually in fixed update
animator["Camera_entrance|Camera_entranceAction"].speed = 0f;

}

// Use of fixed update to call 30 times a second (to try to match up to video and animation playback)
void FixedUpdate()
{

// get current video frame
currentFrame = vp.frame;

// get frame count between 0 - 1 from current video frame
animationFrame = currentFrame / totalFrame;

// set the animation position to that of the current video frame
animator["Camera_entrance|Camera_entranceAction"].normalizedTime = animationFrame;

}
}

• Don't be afraid to show your code. Code is much more explicit and exact about the precise steps you're taking than a prose description of similar length. May 21 '20 at 12:44

I think checking the frameReady event is the way to go. I managed to solve my stuttering issues by updating the camera animation every time the video frame had changed. In my case I was updating every frame anyway, so i didn't bother subscribing to the frameReady event.

I just checked the previous frame against the current one and updated the camera position then redrew my backplate quad that was drawing the video.

I'll test to see if there is better performance from using the frame ready event as well. But currently this seems to work fine for me as the video frame rate is well below my update frame loop.

The following code snippet works for me ;

    void Update()
{
float current_video_time = (float)videoplayerObj.time;
if (current_video_time != previous_video_time)
{
Mesh mesh = planeObj.GetComponent<MeshFilter>().mesh;
//enamble camera animation
cameraAnimObj.Play();
cameraAnimObj["Take 001"].time = myTimeoffset + (float)videoplayerObj.time;
previous_video_time = current_video_time;
//force camera position update
cameraAnimObj.Sample();
//stop camera animation
cameraAnimObj.Stop();
}
}

• Hi Rob, Thank you very much for your reply. I've read through and tested your code and it seems to have fixed the sync issues that I've been having trouble with. I've not tested it using the frameReady function yet but plan to. Thank you again you've really helped me out with this issue!
– Nic
May 27 '20 at 9:12
• Great, glad it helped :)
– Rob
May 27 '20 at 15:34