I'm not so sure if I understood your problem right, but I'll try to answer anyways.
gluLookAt(), as its name implies, creates a view matrix for a camera, and takes three vectors as parameters.
The first one is the camera's position. This is where your camera will be located. If you want your camera to rotate around itself, you most likely don't want this parameter to change.
The second one is the location of the point of interest, or where your camera will be pointed at.
The third one is a vector that represents where 'up' is.
So, as you mentioned, you want your camera to rotate around itself. There are actually three meanings of "rotation" in three-dimensional space:
Suppose this lovely plane donated to us by Wikipedia is your camera:
Doing yaw (yawing?) is pretty simple. You keep your 'up' vector to (0, 1, 0), and change the point of view to rotate around the XZ plane, by setting it to (cos(theta), 0, sin(theta)).
For simplicity, I'm supposing your camera is located at (0, 0, 0), but if not, you can simply add the location of your camera to your point of interest.
Rolling is also quite simple. You keep the point of interest static, and change the 'up' vector to rotate around the XY plane, by setting it to (cos(theta), sin(theta), 0).
Once again, for simplicity, I'm supposing you're looking at (0, 0, 1) or (0, 0, -1).
Finally, pitching is slightly more complicated, since you will be simultaneously changing your point of interest and 'up' vectors, but it's not so complicated either.
Let's start by changing the point of interest to rotate around the YZ plane, by setting it to (0, sin(theta), cos(theta)). This will take care of rotating the camera.
Now, to stabilize the camera, we need to set the 'up' vector to be orthogonal to both the point of interest vector, and the X axis. You can also build it with trigonometric functions, or you can use the facilities of the cross product, which does exactly that. So you set the 'up' vector to cross(point_of_interest, (1, 0, 0)). If your camera ends up upside down, reverse the parameters or do the cross product with (-1, 0, 0) instead.
This covers the three main rotations. Doing more complicated rotations involves using quaternions. Quaternions is a slightly more complicated topic I can't cover in a simple answer, but I'm sure there are many tutorials around online, so go ahead and google it.
Have a nice day.