When designing an instant replay system that will playback the last n seconds of gameplay, I've seen people essentially record all of the game events and game object states and play it back. This seems like a very heavy weight solution. TowerFall achieves its instant replay by saving the last n seconds of the frame buffer and simply plays them back in order.

How would you achieve this in Unity? My initial attempt was to simply keep a Queue of Texture2D objects of the entire screen, ensuring the Queue never got larger then a few seconds worth of Texture2D objects.

However, I quickly discovered this was extremely inefficient as memory ballooned to several gigs and the frame rate dropped significantly. I know TowerFall wasn't developed in Unity, but it seems like it was able to efficiently achieve this, so I'm curious if my approach was just wrong?

Here is the code I used (from memory)

Queue<Texture2D> frames = new Queue<Texture2D>();

// 130 frames is the equivalent of ~5 seconds.
if (frames.Count > 130) {
    var trash = frames.Dequeue();
    trash = null;
}

var frame = new Texture2D (Screen.width, Screen.height, TextureFormat.RGB24, false);
// Read screen contents into the texture
frame.ReadPixels(Rect(0, 0, Screen.width, Screen.height), 0, 0);
frame.Apply();
frames.Enqueue(frame);