You've almost got it. Given that you want the absolute position, I would switch from using the touch delta position to the touch position (absolute). I then normalized it and used your xPosLimit variable to determine the final xPosition.

Treat it as pseudocode tho, I wrote it from memory.

void Update()
{  
    if (Input.touchCount > 0 && Input.GetTouch(0).phase == TouchPhase.Moved)
    {
        // Control sliding using touch input.
        Vector2 touchPos = Input.GetTouch(0).position;

        //normalize the X coord (make it between 0 and 1), Y coord is irrelevant
        float normalizedX = touchPos.x / Screen.width;

        //interpolate between -xPosLimit and +xPosLimit by the normalizedX coord
        //no need to clamp it
        float xPos = Mathf.lerp(-xPosLimit, xPosLimit, normalizedX);

        //set the new position, leave Y and Z alone
        transform.position = new Vector3(xPos, transform.position.y, transform.position.z);
    }
}

You've almost got it. Given that you want the absolute position, I would switch from using the touch delta position to the touch position (absolute). I then normalized it and used your xPosLimit variable to determine the final xPosition.

Treat it as pseudocode tho, I wrote it from memory.

void Update()
{  
    if (Input.touchCount > 0 && Input.GetTouch(0).phase == TouchPhase.Moved)
    {
        // Control sliding using touch input.
        Vector2 touchPos = Input.GetTouch(0).position;

        //normalize the X coord (make it between 0 and 1), Y coord is irrelevant
        float normalizedX = touchPos.x / Screen.width;

        //interpolate between -xPosLimit and +xPosLimit by the normalizedX coord
        //no need to clamp it
        float xPos = Mathf.lerp(-xPosLimit, xPosLimit, normalizedX);

        //set the new position, leave Y and Z alone
        transform.position = new Vector3(xPos, transform.position.y, transform.position.z);
    }
}

Update: See comment below, the issue was solved with the above code AND by disabling interpolation on the rigidbody.