Constraining camera bounds within bounds based on Unity UI object dimensions

Question

I have a perspective camera which has bounds which I wish to constrain within larger bounds. All the bounds are calculated in world space.

The code for containing the camera within the constraining bounds is straightforward:

        var camRect = this.cameraComponent.GetCameraWorldRect();
        var correction = Vector3.zero;

        if (camRect.xMin < constrainingRect.xMin)
        {
            correction.x += Mathf.Abs(constrainingRect.xMin - camRect.xMin);
        }

        if (camRect.xMax > constrainingRect.xMax)
        {
            correction.x -= Mathf.Abs(constrainingRect.xMax - camRect.xMax);
        }

        if (camRect.yMin < constrainingRect.yMin)
        {
            correction.y += Mathf.Abs(constrainingRect.yMin - camRect.yMin);
        }

        if (camRect.yMax > constrainingRect.yMax)
        {
            correction.y -= Mathf.Abs(constrainingRect.yMax - camRect.yMax);
        }

        this.cameraComponent.transform.position += correction;

This can be visualized as:

The problem I am facing is that I have a UI object that is covering the right hand side of the screen and I am uncertain how to correctly calculate the width of the UI object so that the constrained camera bounds can take it into account.

The camera that is drawing the UI object is configured as follows:

The camera that is being constrained is configured as follows:

Apart from the layer, the parameters which affect the rendering of the objects for each camera configuration are the same.

The UI object has the following configuration with a Canvas Renderer:

This is how I am currently attempting to calculate to width of the UI object but given that the UI is being rendered using a perspective camera I wasn't too certain that ScreenToWorldPosition was correct.

    private float CalculateMenuWidth()
    {
        // Generate padding to account for the side menu
        var canvas = LevelEditor.Instance.Menu.GetComponentInParent<Canvas>();
        var menuRect = LevelEditor.Instance.Menu.GetComponent<RectTransform>();
        var menuRectMax = menuRect.rect.max * canvas.scaleFactor;
        var menuRectMin = menuRect.rect.min * canvas.scaleFactor;

        if (this.cameraComponent.orthographic)
        {
            return Mathf.Abs(this.cameraComponent.ScreenToWorldPoint(menuRectMax).x - this.cameraComponent.ScreenToWorldPoint(menuRectMin).x);
        }

        // Perspective camera requires distance for ScreenToWorldPoint
        var distance = Mathf.Abs(this.cameraComponent.transform.position.z);

        return Mathf.Abs(this.cameraComponent.ScreenToWorldPoint(new Vector3(menuRectMax.x, 0f, distance)).x - this.cameraComponent.ScreenToWorldPoint(new Vector3(menuRectMin.x, 0f, distance)).x);
    }

However, the size of menu returned by the method above appears to be too large.

How can I correctly calculate the width of the UI object so it appears that the camera constraints do not allow the camera to travel past the UI object?

EDIT: I've added a link to a Unity project which is the minimal working example for reproducing the issue.

https://www.dropbox.com/s/mq36mfqz5wvvsch/CameraConstraintGUIIssue.zip

The camera can be controlled with WASD to translate and the Shift key will speed up the movement.

The camera is constrained inside the checkerboard. An offset is created so that the camera should stop moving to the right once the right edge of the checkerboard is adjacent to the left edge of the menu. Unfortunately, I can't seem to calculate the size of the menu correctly so this occurs.

The code for constraining the camera and calculating the menu size is in EditorCamera.cs.

Edit2:

The constraint from KingDolphin

The constraint I am attempting to calculate (quoted from above, An offset is created so that the camera should stop moving to the right once the right edge of the checkerboard is adjacent to the left edge of the menu.):

Answer 1

EDIT:

To fix the problem shown in Edit2, a slightly more complicated process has to happen.

So the big problem that makes calculating the width mildly complicated is that you have two different cameras: a UI camera, and the actual camera that we want to constrain.

This is totally fine, but the math becomes slightly more annoying.

The reason that it's slightly more complicated is that directly converting the menuRect corners into world coordinates converts it based off of the UI camera which is a far larger camera (in terms of FOV).

So what we have to do is scale the size of the menuRect down to the size of the constrained camera.

To do this, we have to use viewport coordinates.

The viewport is the only thing that the two cameras share 100% of the time.

Which means that the viewport coordinates for the UI camera are the exact same for the constrained camera.

This allows us to convert coordinates with respect to the UI camera into different coordinates with respect to the constrained camera.

First, we have to get the worldSpaceCoords of the menuRect as before:

var worldSpaceCorners = new Vector3[4];
menuRect.GetWorldCorners(worldSpaceCorners);

Then, we have to loop through each corner and convert each corner into viewport coordinates RELATIVE TO THE UI CAMERA.

for (var i = 0; i < 4; i++)
{
    var viewport1 = uiCamera.WorldToViewportPoint(worldSpaceCorners[i]);
}

And finally, we have to convert the viewport coordinate back into a world coordinate RELATIVE TO THE CONSTRAINED CAMERA.

for (var i = 0; i < 4; i++)
{
    var viewport1 = uiCamera.WorldToViewportPoint(worldSpaceCorners[i]);

    worldSpaceCorners[i] = this.cameraComponent.ViewportToWorldPoint(viewport1);
}

So the final result is:

private float CalculateMenuWidth()
{
    var menuRect = GameObject.Find("Panel").GetComponent<RectTransform>();
    var uiCamera = GameObject.Find("UICamera").GetComponent<Camera>();

    var worldSpaceCorners = new Vector3[4];
    menuRect.GetWorldCorners(worldSpaceCorners);

    for (var i = 0; i < 4; i++) {
        // Transform to viewport from UI camera
        var viewport1 = uiCamera.WorldToViewportPoint(worldSpaceCorners[i]);

        // Transform from viewport to space using our camera
        worldSpaceCorners[i] = this.cameraComponent.ViewportToWorldPoint(viewport1);
    }

    return Mathf.Abs(worldSpaceCorners[2].x - worldSpaceCorners[0].x);
}

private void ContainWithin(Rect rect)
{
    // Make sure camera zoom fits inside level
    // ...

    // Create padding around the map so the UI buttons don't obscure it for ray casting, viewing, etc.
    var paddingRight = this.CalculateMenuWidth();
    // Left, bottom, top
    // ...

    this.cameraRect = this.cameraComponent.GetCameraWorldRect();
    var correction = Vector3.zero;

    if (this.cameraRect.xMax > rect.xMax + paddingRight)
    {
        correction.x -= this.cameraRect.xMax - (rect.xMax + paddingRight);
    }

    // Left, bottom, top
    // ...

    this.cameraComponent.transform.position += correction;
}

OLD: (scroll up for updated answer/explanation)

Okay, I think you've overcomplicated calculating the width of the UI object, especially since you want it in world coordinates.

All it takes is simply getting the world coordinates of the UI object using

var menuRect = GameObject.Find("Panel").GetComponent<RectTransform>();

var worldSpaceCorners = new Vector3[4];
menuRect.GetWorldCorners(worldSpaceCorners);

And then very simply subtracting the x-coordinates of the opposite corners

return Mathf.Abs(worldSpaceCorners[2].x - worldSpaceCorners[0].x);

This is how to correctly calculate the width, but I noticed another problem (at least with the example project you gave).

When calculating the 'correction' vector; the code should be like this:

var correction = Vector3.zero;

if (this.cameraRect.xMin < rect.xMin + paddingLeft)
{
    correction.x -= this.cameraRect.xMin - (rect.xMin + paddingLeft);
}

if (this.cameraRect.xMax > rect.xMax - paddingRight)
{
    correction.x -= this.cameraRect.xMax - (rect.xMax - paddingRight);
}

if (this.cameraRect.yMin < rect.yMin + paddingBottom)
{
    correction.y -= this.cameraRect.yMin - (rect.yMin + paddingBottom);
}

if (this.cameraRect.yMax > rect.yMax - paddingTop)
{
    correction.y -= this.cameraRect.yMax - (rect.yMax - paddingTop;
}

For why – I'll focus on just the xMax if-statement. Hopefully, this (terrible) drawing illustrates how it works:

Basically, the if statement needs to see if the right side of the rect (cameraRect.xMax) is farther right (greater than) the farthest position that it can be.

The farthest position it can be without the UI is rect.xMax. So, to calculate the farthest position with the UI is rect.xMax - width of UI (paddingRight).

Now to move the cameraRect to the position just before its maximum position, you need to calculate the dx to move it by.

And the dx is simply the difference between its current position (cameraRect.xMax) and the farthest right position (rect.xMax - paddingRight).

So, the correction.x (dx) is cameraRect.xMax - (rect.xMax - paddingRight)

This same idea can be applied to the left, top, and bottom.

Hopefully, this solves your issue and explains it adequately.