# Transformed Vertices have wrong Z Axis

For quite some time I've been working on my open 3D game engine, to learn how this stuff works. You can see the full codebase relating to my problem here on GitHub.

I'm using an OpenGL 3.3 Core Profile Context on Mac OS X 10.10.3 and a GeForce GTX 760, if this matters.

Everything seems to work fine when submitting my vertices and MVP matrix to the Pipeline. But at two different points I try to transform vertices on my CPU, which leads to problems.

To simplify rendering the debug bounding boxes in a single frame, I transform the vertices by their MVP matrix and later submit them all to OpenGL with an identity matrix as transformation matrix:

for (auto& c : corner) {
glm::vec4 t = VP * glm::vec4(c, 1.0f);
vertices.emplace_back(glm::vec3(t) / t.w);
}
// ...


Most of the time, this seems to work fine. But when some vertices of a bounding box lie behind the camera, they will be "mirrored" and extend into empty space, as you can see in this screenshot:

Something similar happens with my portal based rendering approach, as described in Portals and Mirrors: Simple, Fast Evaluation of Potentially Visible Sets. Basically, I'm looking for a visible portal, and then for more portals that are visible through the already found portal. The relevant code:

for (int c = 0; c < 4; c++) {
glm::vec3 port = portal.getVertex(c);
glm::vec4 result = VP * glm::vec4(port, 1.0f);
glm::vec3 vert = glm::vec3(result) / result.w;
// ...
}


This basically works as expected, as you can see in the following screenshot:

However, as before, there is a problem with vertices behind the camera. I would expect the transformed vertices Z axis to be in the range [0;1] and vertices behind the camera should be outside of that range. What I'm really seeing, however, is that the Z part is always 1, as you can see in this screenshot (the first line is the world vector, the second is the vector multiplied with the matrix, the third is the multiplied vector divided by w):

To give you the full picture, this is how I create my projection matrix:

projection = glm::perspective(45.0f, size.x / size.y, 0.1f, 75000.0f);


And my view matrix is created using the quaternion of my camera:

glm::mat4 translate = glm::translate(glm::mat4(1.0f), pos);
glm::mat4 rotate = glm::toMat4(quaternion);
view = glm::inverse(translate * rotate);


According to every description I've read of the OpenGL transformation pipeline, I've done everything to have my world coordinates now in clipping space. But, looking at the Z axis, this seems to be false. I'm sure I'm missing something obvious with my math here.