# How to blend two cameras when traveling through a portal in Unity3D

Before I get to my question, I know the most obvious solution would be to use the normalized view port rect, however I need more complex shapes than a rectangle, and I've looked in to using the view port rect and it seems to not be my solution.

EDIT: Some people were confused by my question, let me go in to a bit more detail. What's happening is as the player moves in to one portal, I create a clone FPS controller and move it out of the other. This gives me two cameras, and the view you see on the right above. It's just showing one camera, and clipping through the portal. What I want is something similar to this where the cameras blend to create the illusion of a smooth transition. What I want to do is delete everything from the green checker image to the left in the image below, and replace that with the other camera. That way you get the part of camera A's view that is peaking out of the portal, blended with the part of camera B's view that's peaking out of the other portal, to get one complete image. And as you move through the portal the cut changes appropriately.

I've been designing a portal system, I have everything down, including getting the player to smoothly move through the portal. My main problem now is getting that camera blend effect Valve does. I need two cameras to blend together seamlessly, as if you were poking your head through the portal. And it can't just be a rectangle, it has to match however the player is looking through the portal.

My best lead on this right now is to possibly project a depth mask shader behind each portal, then make the camera from the portal you're traveling to be depth only. Then somehow mix the two cameras. My main problem is figuring out exactly how I'd do this, how to make the second camera only render what's outside of the portal, and have the rest default to camera 1 to get one full screen projection.

If you could give me ideas, or explain how I may do this with the depth mask shader that would be a tremendous help. I'll continue to work on this and update as I make break throughs.

• Can you elaborate what is the "camera blend effect Valve does"? I've understood that in portal rendering a single camera is enough. In first person games it's attached to the player and is automatically transformed to the new location when going trough the portal together with the player. For rendering the portal effect a transformed copy of the scene is rendered. This can be achieved with a second camera as well, but there shouldn't be any need for blending between these cameras. – msell Apr 5 '13 at 5:19
• I already have the visual effects down perfectly. What I'm trying to do is figure out how to get the effect of walking through the portal. I already have all the character graphics and such moving smoothly, I just need it to look smooth from a first person perspective. – Timothy Williams Apr 5 '13 at 15:13
• I'm sure many people would like to help you with the problem if they understood what it was. You really need to explain the issue better, perhaps add pictures or even a video visualizing what is wrong. – msell Apr 9 '13 at 5:04
• Could you add a link to an example video of the effect? – Mike Baxter Apr 9 '13 at 6:15
• What if you used only one camera / FPS controller? When the camera moves through the portal, you can transform it to the new location and orientation. If the portal rendering is correct, the transition should be seamless and no blending needed. – msell Apr 10 '13 at 5:44

## Understanding the Problem

From what I can see the problem you are describing is the result of the camera's near plane intersecting the plane defined by the portal. While this intersection occurs you can see behind the wall the portal is on.

This is similar to a problem experienced in other games when the player is just transitioning from above water to underwater. If the camera is just above the water, no post processing effect is applied to obfuscate the players view (make it dark and blurry and blue). So if the bottom of the near plane is just under the water the player can see clearly under water.

If I am correct, you can confirm it by changing the position of this plane when defining the projection matrix. As the distance from the camera origin to the near plane increases, so should the problem.

## The Easy Solution

Making the near plane very close to the camera should almost eradicate this issue. This solution is not a complete one, but will yield a good enough result in the vast majority of cases and it's efficient.

## The Full Solution

If just putting the near plane closer to the camera doesn't satisfy, then you can create a "mask" to blend the images generated by rendering the scene from the player and portal perspective.

Assuming you only allow for portals to be applied to flat surfaces, you can calculate the line of intersection between the camera's near plane and the plane defined by the portal (or the wall on which it lies). This line will split the screen into two parts. Determining which side of the line a screen pixel is on will let you know which render image to use, the portal image or the player camera image.

Keep in mind that if this problem is occurring then the camera view frustum has to be completely within the portal so the intersection line will always cut completely from one screen edge to another.

This link should help with the maths to find the line. The code below should be roughly correct.

The line of intersection is defined using a point on the line and the line direction. Below the intersection direction is calculated using the cross product of the portal normal and the camera view direction (near plane normal). A point on the line is given by casting a ray from a point on the near plane directly towards the portal plane (along the portal normal) and finding the intersection point.

Vector3 intersectionDir = Vector3.cross(portalNorm, viewDir);
Ray ray = new Ray(camPos + viewDir * camNearPlaneDist, portalNorm);
Vector3 intersectionPos = ray.intersects(new Plane(portalVert1, portalVert2, portalVert3);


Make sure the viewDir is a unit vector. portalVert1, 2 and 3 are just 3 of the 4 vertices used for the portal decal or the surface it's on. There are other ways to define the plane on which the portal lies, but I assume this is the most readily available information.

Once you have these two vectors to define the intersection line multiply each by the view and then projection matrices to get them in screen space.

You can then use a post process shader to blend these images. You select which image to use at each pixel by determining which side of the dividing line the current pixel lies on. This is done by taking the pixel position (which is also the position you use to look up the render target texel) and doing;

float d = (pixelX - intersectionPos.X) * intersectionDir.Y - (pixelY - intersectionPos.Y) * intersectionDir.X;


The side is given by whether d is more or less than 0. If it's exactly 0, then you're on the line.

For reference on the maths above see this.

This method can also be used when creating a depth mask / stencil buffer for use before rendering from the portal perspective. You can create a full screen quad and use the line to slice it.

• The near clip plane was a good idea, but not exactly what I'm looking for. That second part is an interesting approach though. Currently I'm using a depth mask shader on everything behind the exiting portal plane, then setting the exiting camera's clear flags to depth only. This makes it so any part of the exiting camera that is peaking out of the portal draws on the screen, and the entering camera draws on the depth mask, so it creates one image by mixing the parts of each camera that is peaking out of it's respective portal. The only problem is I'm getting some minor clipping issues and a s – Timothy Williams Apr 11 '13 at 19:44
• slight jerk as the player travels through. This idea you brought up sounds quite promising. So essentially what I need to do is calculate the line along the screen that the camera near plane and the portal plane are intersecting upon, direction gives me which direction the line is going, and point is the various points of the line. How would I then determine in a shader/script which side of the dividing line a pixel lays on? – Timothy Williams Apr 11 '13 at 19:47
• I updated the answer a bit to give some basic code. Make sure to try changing the view near clipping plane distance to confirm the problem. Making this as small as possible may also suffice. Remember, get it working, get it working right, then get it working right fast. – OriginalDaemon Apr 11 '13 at 23:53
• Related article: en.wikibooks.org/wiki/OpenGL_Programming/Mini-Portal_Smooth – msell Apr 12 '13 at 5:11
• I did test out changing the near clipping plane distance, it made the problem slightly better, but as the article Mell posted states it does not fix the problem when strafing through. I still feel like the line of intersection may work though, so I'm going to try it out. I'll also look a bit more in to that article Mell posted. In the end it will come down to whether a depth mask is faster or this line of intersection is faster – Timothy Williams Apr 12 '13 at 14:05

The suggested answers were very good, but I ended up going for a different technique using a depth mask.

What you do is take THIS script and shader, you put the script on every object with a renderer in your scene and set the render queue to 3020 (I'll post a script to make this easy later).

Then, you create a box of planes (all facing inwards, in the picture you can't see the side of the box closest to you, but when you're inside the box all you should see is gray) behind BOTH of your portals like so: And set them on to a special layer of their own (I chose "DepthMask" for mine), then you add a material with the shader above to them.

You then take the main camera and uncheck your special layer from it's culling mask (I unchecked DepthMask layer) and set it's depth to 0.

Then when you're teleporting through and clone the camera, set the other camera's clear flags to "Depth Only" and it's depth to 1.

You then get a seamless screen blended between the two camera views.