Jittery Rotational Movement with Arcball implementation

Question

I have an assignment where I need to implement arcball using Opengl (~2.0). I have it more or less implemented but I have a some problems/issues/bugs and I'm not sure whats either causing them or how to solve.

Arcball rotation: Specically minor variance in mouse movement away from the axis I'm rotating on seems to "confuse" my program, causing artifacts (such as rotating briefly a completely different direction before resuming its current path).

Code:

void mouse_motion(int mx, int my)
{

    float angleRadians;
    //handles mouse motion events
    if ( arcball_on && (mx != last_mx || my != last_my) ) {  
        // if left button is pressed
        lArcAngle = arcAngle;
        cout << " Last Angle: " << lArcAngle << endl;
        cur_mx = mx;
        cur_my = my;

        glm::vec3 va = get_arcball_vector(last_mx, last_my);
        glm::vec3 vb = get_arcball_vector(cur_mx, cur_my);
        angleRadians = acos(min(1.0f, glm::dot(va, vb)));
        arcAngle = ( ( angleRadians * 180 ) / 3.14159265 );
        cout << " Angle in degrees: " << arcAngle << endl;
        //cout << " ArcAngle after Mult: " << arcAngle << endl;
        arcAngle += lArcAngle;
        cout << " ArcAngle after accumulation: " << arcAngle << endl;
        axis_in_world_coord = glm::cross(va, vb);

        last_mx = cur_mx;
        last_my = cur_my;
        //glutPostRedisplay();

    }
// other code for other functionality...
}

void display()
{   
    //Background color
    glClearColor(1, 1, 1, 0);
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); 
    //glClear(GL_COLOR_BUFFER_BIT);
    glEnable(GL_DEPTH_TEST);

    // Matrix setup   
    glMatrixMode(GL_PROJECTION);
    glViewport(0, 0, width, height);
    glLoadIdentity();
    gluPerspective(40, (float)width / (float)height, 0.1, 1000);

    // Matrix setup
    glMatrixMode(GL_MODELVIEW);
    glLoadIdentity();
    glTranslatef(0, 0, -3);

    int myFlagCtr = getFlagCtr();
    //cout << "CRX: " << crx << " CRY: " << cry << " CRZ: " << crz << endl;

    glPushMatrix();
        //glRotatef(arcAngle, axis_in_world_coord.x, axis_in_world_coord.y, axis_in_world_coord.z);
        glTranslatef(vec_blx, vec_bly, zScaling);
        glRotatef(arcAngle, axis_in_world_coord.x, axis_in_world_coord.y, axis_in_world_coord.z);

    if (myFlagCtr == 1) {
        // Set Wireframe
        glPolygonMode(GL_FRONT_AND_BACK, GL_LINE);
        // color red
        glColor3f(1.0f, 0.0f, 0.0f);
    }
    else if (myFlagCtr == 2) {
        glPolygonMode(GL_FRONT_AND_BACK, GL_LINE);
        // color red
        glColor3f(1.0f, 0.0f, 0.0f);
        glEnable(GL_POLYGON_OFFSET_LINE);
        glPolygonOffset(-1.0, -1.0);
        //draw object
        blDrawMouseMesh();
        glPolygonMode(GL_FRONT_AND_BACK, GL_FILL);
        // color red
        glColor3f(0.0f, 1.0f, 0.0f);
    }
    else if (myFlagCtr == 0 || myFlagCtr == -1) {
        // Bring back to Default
        glPolygonMode(GL_FRONT_AND_BACK, GL_FILL);
        glColor3f(0.53f, 0.12f, 0.47f);
    }
        blDrawMouseMesh();
    glPopMatrix();
    glutSwapBuffers();
}

As you can see between the two mouse vectors I get the angle between them, and the orthonogol vector which is the axis I am rotating around. But this only seems to work in a rather "dirty" and not particularly smooth way.

Edit: I now think the problem is the accumulation of degrees, when I switch axis's it remembers the angle from before, so it will "rotate" by the same amount previously plus the new angle, I'm not sure how to solve given the loading of the identity matrix causing non-accumulated values to reset.

Answer 1

You are using axis-angles to represent rotations, which is a bad idea, as you have discovered. You need to combine the rotations (i.e. compute the concatenation of one rotation by another), which is ugly to do with axis angle, and most certainly not what the code you posted does.

You should use quaternions, in particular use shortest arc rotation:

https://stackoverflow.com/questions/1171849/finding-quaternion-representing-the-rotation-from-one-vector-to-another

To find the quaternion that takes va to vb, and then multiply your view rotation quaternion by this delta one. So you code becomes:

glm::vec3 va = get_arcball_vector(last_mx, last_my);
glm::vec3 vb = get_arcball_vector(cur_mx, cur_my);
glm::gtc::quat shortest = shortest_arc_quaternion(va,vb);
quaternionRotation = shortest * quaternionRotation;

and later

glm::mat4x4 matrix = glm::gtc::mat4_cast(quaternionRotation);
glMultMatrixf(&(matrix[0][0]));

Quaternions can be turned into a matrix, which will replace glRotatef() by multiplying this matrix on top of the stack.