I have a Unity scene with OpenVR and Single Pass Stereo Rendering.

I need to do a fullscreen 3D masking effect in VR, based on reconstructed worldspace position from depth. I'm doing it using Graphics.Blit and a fullscreen quad.

This is what it looks like: enter image description here

The problem is once you have the reconstructed world pos, you can't just say "everything past wpos.x==300 is black", because each eye will offset the mask and it appears like you're looking at it cross-eyed.

This is because the wpos.x==300 method does not take into account the Interpupillary Distance (IPD) between the two vr "cameras" rendering the scene. According to the unity docs there's a unity_StereoEyeIndex, and I used that to offset the eye ray for each eye by an arbitrary amount in camera space.

I can eyeball this offset using a float, but my question is how do you calculate it exactly? How do you make stereo masking shaders in VR?


I haven't yet reverse-engineered the problem fully, and am still waiting for a full answer. But I have fixed it for now by avoiding Graphics.Blit() because it's too mysterious in VR, and instead used a sphere in the game world with a special material.

The shader I use is basically what @DMGregory did in this answer for his "Object in World" method. Note that you have to correct the depth sampling with this:

screenUV = UnityStereoTransformScreenSpaceTex(screenUV);
screenUV.y = 1.0f - screenUV.y;
  • \$\begingroup\$ I wrote an earlier answer about reconstructing worldspace position within skewed frusta in VR - would that address your issue? \$\endgroup\$
    – DMGregory
    Commented Mar 1, 2017 at 14:02
  • \$\begingroup\$ Oh hey DMGregory, I am using your answer actually. But I added my per-eye offset on top of it: clip.x +=(unity_StereoEyeIndex * 2 - 1) * _EyeDistOffsetTemp;. Also, I found that in your answer var p = _camera.projectionMatrix; is wrong right now (maybe a unity update broke it). Because the projection is slightly off and moving your head around you can see, especially in the distance, that the reconstructed world pos rotates slightly more than the regular geometry. Did you test your code recently? \$\endgroup\$ Commented Mar 1, 2017 at 14:49
  • \$\begingroup\$ Adding the per-eye offset might be the problem — the quad version already includes that offset natively via the camera's view & projection matrices. So if you're using that version you might be doubling up. (And if you're not using that version, try it — I trust its math more than I trust the blitting script) ;) \$\endgroup\$
    – DMGregory
    Commented Mar 1, 2017 at 14:51
  • \$\begingroup\$ Hmmmm.... unless Unity is not truthfully populating _WorldSpaceCameraPos for each VR camera. In which case we should be able to dig the true values out of the translation component of the view matrix. Let me know if that looks like the issue, and I can update that answer to calculate this vector rather than trust the built-in one. Unfortunately my VR rig is on loan for an exhibit so I can't verify with real hardware right now. \$\endgroup\$
    – DMGregory
    Commented Mar 1, 2017 at 15:02
  • \$\begingroup\$ I just tested exactly what you wrote in "Blit / Post Effect". And the projection is off, and the eyes aren't offsetted, like I said. I'm using version 1.2.1 of the OpenVR Plugin, Unity 5.5.0f3, Single Pass Stereo Rendering Method. If it works for you and not for me then there must be some black magic involved. \$\endgroup\$ Commented Mar 1, 2017 at 15:05


You must log in to answer this question.

Browse other questions tagged .