I am writing a planet renderer and have come across a problem that is really limiting the usability of the program. When I am on the planet (anywhere but the north pole) I cannot rotate the camera to the left or to the right as I would if I was on a flat plane. Instead I can only move as if I were aligned to a regular flat plane. This gives me awkward viewing as seen here:
This limits me to only really looking straight ahead of the camera and straight behind the camera.
I drew this to try to illustrate the problem:
Will setting the up-vector of the camera to be the the point on the sphere where I am located normalized alleviate this problem? I am at a dead-end here and any ideas would be greatly appreciated. Thanks.