0
\$\begingroup\$

I have a Touch-Input device which is linked to my Unity Program and a VR Device. The Touch-Input device which is bolted somewhere BUT the position can be changed.

I can get the "2D Coordinates" X / Y from the touch input device (from 0 to 1)

For each of those 2D coordinates I also have a 3D-VR Coordinate , grabbed by a LEAP or gloves. (Those coordinates are not always the corner ones.)

Here the problem on a wonderful paint art:

enter image description here

So what I have is the 2d and 3d positions of my fingertip (on the touch and inside the 3d world

And what I need is the a 3d Object (for example Cube) with correct rotation and position.

Bonus points for Scale!

\$\endgroup\$
  • 1
    \$\begingroup\$ I'm not sure I understand this currently, do you mean the user has a glove in one hand, and touching the Touchinput device with their other hand? And you want the cube to be created on the 3D position of the glove? If you could describe this into more details I might be able to provide an answer, but right now I'm having trouble picturing what you are trying to achieve. \$\endgroup\$ – TomTsagk Feb 8 at 16:15
  • \$\begingroup\$ I updated my question! \$\endgroup\$ – OC_RaizW Feb 8 at 21:29
  • \$\begingroup\$ The user is using a glove- or a similar device to track his hand. and with the SAME hand he touches a touchscreen. And now I want to know where the touchscreen is . \$\endgroup\$ – OC_RaizW Feb 8 at 21:30
0
\$\begingroup\$
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;

public class Calibrate : MonoBehaviour
{
    public Vector2 Current_2D_TouchscreenPosition;
    public GameObject PointingFingerTip_DummyPosition;

    private Vector3 CalibrationPos_3D_1;
    private Vector3 CalibrationPos_3D_2;
    private Vector3 CalibrationPos_3D_3;

    private Vector2 CalibrationPos_2D_1;
    private Vector2 CalibrationPos_2D_2;
    private Vector2 CalibrationPos_2D_3;

    private int CalibrationIndex;
    private bool CalibrationCompleted;

    private Matrix TouchToWorldMatrix;

    public void SetCalibration(Vector3 World3DPosition, Vector2 Touch2DPosition )
    {
        //TODO: Check if values are valid!
        if (CalibrationIndex == 0)
        {
            CalibrationPos_3D_1 = World3DPosition;
            CalibrationPos_2D_1 = Current_2D_TouchscreenPosition;
            CalibrationIndex++;
        }
        else if (CalibrationIndex == 1)
        {
            CalibrationPos_3D_2 = World3DPosition;
            CalibrationPos_2D_2 = Current_2D_TouchscreenPosition;
            CalibrationIndex++;
        }
        else if (CalibrationIndex == 2)
        {
            CalibrationPos_3D_3 = World3DPosition;
            CalibrationPos_2D_3 = Current_2D_TouchscreenPosition;

            //Creating The Matrix-
            TouchToWorldMatrix = CreateTouchToWorldMatrix();
            CalibrationIndex = 0;
            CalibrationCompleted = true;
        }

    }

    Vector3 GetWorldPosition(Vector2 TouchPosition)
    {
        if (!CalibrationCompleted)
        {
            return new Vector3(0,0,0);
        }
        return MultiplyVectorWithMatrix(TouchPosition, TouchToWorldMatrix);
    }

    private Matrix CreateTouchToWorldMatrix()
    {
        //Create an empty Matrix with 4 Rows and 3 Columns
        Matrix M_LHS = new Matrix(4, 3);
        //Create another empty Matrix with 3 Rows and 3 Columns
        Matrix M_RHS = new Matrix(3, 3);

        //Fill Left Hand-Side Matrix (
        //  Row, Column
        M_LHS[0, 0] = CalibrationPos_3D_1.x;
        M_LHS[1, 0] = CalibrationPos_3D_1.y;
        M_LHS[2, 0] = CalibrationPos_3D_1.z;
        M_LHS[3, 0] = 1;

        M_LHS[0, 1] = CalibrationPos_3D_2.x;
        M_LHS[1, 1] = CalibrationPos_3D_2.y;
        M_LHS[2, 1] = CalibrationPos_3D_2.z;
        M_LHS[3, 1] = 1;

        M_LHS[0, 2] = CalibrationPos_3D_3.x;
        M_LHS[1, 2] = CalibrationPos_3D_3.y;
        M_LHS[2, 2] = CalibrationPos_3D_3.z;
        M_LHS[3, 2] = 1;


        //Fill Right Hand Side Matrix
        //  Row, Column
        M_RHS[0, 0] = CalibrationPos_2D_1.x;
        M_RHS[1, 0] = CalibrationPos_2D_1.y;
        M_RHS[2, 0] = 1;

        M_RHS[0, 1] = CalibrationPos_2D_2.x;
        M_RHS[1, 1] = CalibrationPos_2D_2.y;
        M_RHS[2, 1] = 1;

        M_RHS[0, 2] = CalibrationPos_2D_3.x;
        M_RHS[1, 2] = CalibrationPos_2D_3.y;
        M_RHS[2, 2] = 1;

        //Create a Touchscreen to Worldspace Matrix
        return M_LHS * M_RHS.Invert();
    }




    public Vector3 MultiplyVectorWithMatrix(Vector2 vector, Matrix m)
    {
        float X_Multed = (vector.x * m[0, 0]) + (m[0, 1] * vector.y) + (m[0, 2]) ;
        float Y_Multed = (vector.x * m[1, 0]) + (m[1, 1] * vector.y) + (m[1, 2]) ;
        float Z_Multed = (vector.x * m[2, 0]) + (m[2, 1] * vector.y) + (m[2, 2]) ;

        return new Vector3(X_Multed, Y_Multed, Z_Multed);

    }

}

Allright I managed to solve this issue.

I used this answer: https://math.stackexchange.com/questions/3108619/how-to-find-a-2d-coordinate-fields-corners-in-a-3d-coordinate-field-if-i-have-3/3109504?noredirect=1#comment6409762_3109504

And it works.

You can basically copy-paste this script and use it. It works with creating a matrix ...

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.