Maybe it is a point of view opinion but it seems you should use another approach :
- If you have no plans on making your camera track another object then you should make it a child of your object (i do think it is not the good answer but i think it's still better than moving two objects separately when you want them to be synched)
- If you need your camera to be versatile you should make it completely separated from input for it's position and input relative only for it's rotation.
Good tools for that is :
- Making your cam position relative to a transform
cam.position = object.position + localpos - Calculate
localposusingcam.RotateAround(object.position,Vector3.Up,input.horizontal)or something similar
this should already make your code easyer to maintain and it could allow you to switch between targets painlessly