gluLookAt strange behavior implementing camera

Question

I'm having a super tough time trying to implement what I thought would be a very straightforward problem. I'm trying to demonstrate a free camera that can move forward/back, strafe left/right, fly up/down, and look up/down/left/right. The mouse will control the look and the keyboard (WSADRF) will control the movement.

The movement appears to work just fine, and the look behavior appears to be correct from my camera starting point, but as I deviate away from the camera start point, the mouse movement goes into a crazy non-linear type thing where stuff is rotating all over and going nuts. Here's my basic camera setup:

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(70.0, static_cast<GLdouble>(w) / h, 1.0, 10000.0);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glm::vec3 lookat = camera_.pos - camera_.Back();
glm::vec3 up = camera_.Up();
gluLookAt(
  camera_.pos.x, camera_.pos.y, camera_.pos.z,
  lookat.x, lookat.y, lookat.z,
  up.x, up.y, up.z
  );

My camera starts off at (0, 0, 25) and is looking at (0,0,0) where my model is. The look behaviour is fine there. As I start to approach a point like (5,0,0), the look behaviour gets completely boned. Also at that point, it feels like the camera "up" vector gets screwed up. Trying to fly up/down makes the camera kind of move in/out. Here's the code I'm using to update the camera:

static float p = 0.f;
static float y = 0.f;
static float r = 0.f;
p -= 0.5f * delta_y * dtime;
y -= 0.5f * delta_x * dtime;
camera_.ori = glm::quat(glm::vec3(p, y, r));

This problem is super frustrating, because it feels like I'm so close to having everything working, but something is evading me! For reference, here's how I calculate the Up, Right, and Back vectors from the camera orientation, which I got from this site:

glm::vec3 Actor::Right() const
{
  return glm::normalize(glm::vec3(1-2*(ori.y*ori.y+ori.z*ori.z), 2*(ori.x*ori.y+ori.w*ori.z), 2*(ori.x*ori.z-ori.w*ori.y)));
}

glm::vec3 Actor::Up() const
{
  return glm::normalize(glm::vec3(2*(ori.x*ori.y-ori.w*ori.z), 1-2*(ori.x*ori.x+ori.z*ori.z), 2*(ori.y*ori.z+ori.w*ori.x)));
}

glm::vec3 Actor::Back() const
{
  return glm::normalize(glm::vec3(2*(ori.x*ori.z+ori.w*ori.y), 2*(ori.y*ori.x-ori.w*ori.x), 1-2*(ori.x*ori.x+ori.y*ori.y)));
}

Answer 1

Thanks to @DaleyPaley I was able to figure this out. The problem lay in my code to figure out the camera vectors Right, Up, and Back. I was just using some code that I found online, and once I started showing the actual camera placement and vectors from the perspective of a hardcoded camera, I could tell that the vectors being produced by Right, Up, and Back were definitely not orthogonal to each other. They were getting really wonky and flipping over each other.

I switched to this code, and everything resolved itself immediately:

glm::vec3 Actor::Right() const { return glm::mat3_cast(ori)[0]; }
glm::vec3 Actor::Up()    const { return glm::mat3_cast(ori)[1]; }
glm::vec3 Actor::Back()  const { return glm::mat3_cast(ori)[2]; }