@Andrew Tomazos is correct, but conversion of the three vectors to a matrix first requires the storage of 16x additional floats, and is a bit slow.
A better solution is found by writing out the basis vectors a,b,c and plugging these into the matrix-to-quaternion conversion, where the elements are explicit.
m[0][0] = a.x, m[1][0] = b.x, m[2][0] = c.x
m[0][1] = a.y, m[1][1] = b.y, m[2][1] = c.y
m[0][2] = a.z, m[1][2] = b.z, m[2][2] = c.z
Now, we rewrite the matrix-to-quaternion function using the subsituted orthonormal vectors a,b,c. The result is the following function:
// Quaternion from orthogonal basis
Quaternion& Quaternion::fromBasis(Vector3DF a, Vector3DF b, Vector3DF c)
{
float T = a.x + b.y + c.z;
float s;
if (T > 0) {
float s = sqrt(T + 1) * 2.f;
X = (c.y - b.z) / s;
Y = (a.z - c.x) / s;
Z = (b.x - a.y) / s;
W = 0.25f * s;
} else if ( a.x > b.y && a.x > c.z) {
s = sqrt(1 + a.x - b.y - c.z) * 2;
X = 0.25f * s;
Y = (b.x + a.y) / s;
Z = (a.z + c.x) / s;
W = (c.y - b.z) / s;
} else if (b.y > c.z) {
s = sqrt(1 + b.y - a.x - c.z) * 2;
X = (b.x + a.y) / s;
Y = 0.25f * s;
Z = (c.y + b.z) / s;
W = (b.z - c.y) / s;
} else {
s = sqrt(1 + c.z - a.x - b.y) * 2;
X = (a.z + c.x) / s;
Y = (c.y + b.z) / s;
Z = 0.25f * s;
W = (b.x - a.y) / s;
}
return *this;
}
This function gives a quaternion (X,Y,Z,W) directly from the orthonormal basis vectors a,b,c without intermediate storage of a matrix. You may still need to normalize the quaternion. If you have a direction and up-vector, say for a camera, you can construct the basis as: a=dir, b=up, c=cross(dir,up)