How does cube mapping work?

Question

Based on my reading of cube mapping tutorials so far, my understanding is that you need a direction vector, and from the direction vector we can determine the point of intersection with one of the six planes.

Is the direction vector the player's view direction? Besides that, I do not understand how having only one ray will give us the texture to fill the whole screen.

Furthermore, what if we are looking at one of the edge of the cube? Now we have to sample texture from two faces?

Can someone please explain what am I missing here?

Answer 1

So far, my understanding is that you need a direction vector, and from the direction vector we can determine the point of intersection with one of the six planes.

Basically, yes.

---

Is the direction vector is the players view direction?

Any direction. In your shader code you can query the cubemap with any direction you want. It does not have to be where the player is looking.

For instance, aside from using cubemaps for skyboxes, one common use is reflections. For abstract, you compute the direction for the reflection and query there.

---

I do not understand how having only one ray will give us the texture to fill the whole screen?

If you want to fill the screen with a cubemap, you would do a raycasting setup.

The basic setup is as follows:

1. Set a squad to fill the screen.
2. In your fragment shader, compute the direction for the ray and query there.

Alright, let me qualify that…

The fragment shader will run for every pixel (well, every fragment) of the screen, in parallel. Now, in a perspective projection, each pixel will be the projection of a line that reaches the camera. You want the direction of that line. That is what you query the cubemap with.

How do you get that direction? Imagine you are projecting the world into the near plane of the camera. If you know the size of the output, the position of the current pixel, and the distance to the nearplane, you can compute the position of the point in the near plane. Well, the direction from the camera to that point.

The viewing frustrum - Intergraph Computer Systems.

---

For anybody doing this in OpenGL, know, I kind of made it more complex that it is.

If you are going to fill the screen with a squad, and you need to find points in the near plane… Make the squad match the near plane, with the camera at the origin, and you solve this neatly in the vertex shader:

#version 330 core
layout (location = 0) in vec2 position;

uniform mat4 inverseProjection;
uniform mat4 inverseView;

out lowp vec3 cameraRayDirection;

void main()
{
    gl_Position = vec4(position.xy, 0.0, 1.0);
    cameraRayDirection = ((inverseView * inverseProjection) * gl_Position).xyz;
}

In actuality, you only need two coordinates, don't you? I made position vec2 in this code, yes, it works.

In the fragment shader, you get them interpolated, and ready to use.

---

Furthermore, what if we are looking at one of the edge of the cube? Now we have to sample texture from two faces?

You don't have to worry about it. OpenGL understands cubemaps. You can bind your textures with GL_TEXTURE_CUBE_MAP, set the six faces, and your shader code does not have to worry about it. Please refer to a tutorial on uploading textures with OpenGL.

In your shader program, you define your texture sampler as samplerCube, for example:

uniform samplerCube SkyBox;

And then query using an unit vector:

return texture(SkyBox, direction);

---

Now, if you wanted to emulate this in CPU, not using any shader code… Then the cubemap probably isn't the best format.

You can convert the texture to a cylindrical form that you may query with the angles of the direction you want. See Converting to/from cubemaps.

Baring that, using the cubemap would require to find the intersection of the unit vector with the cube.

Answer 2

Adding to the accepted answer regarding how the texel lookup works:

Furthermore, what if we are looking at one of the edge of the cube? Now we have to sample texture from two faces?

No, cubemaps sample only 1 of the 6 textures in 2D.

The textures can internally have a 1 pixel border that reproduces the pixels for the neighbouring texture on the cube, giving the impression that it's sampling from multiple faces but it is only sampling a single 2D texture.

The pseudo-code steps of what happens on the GPU are:

n = Normalise(xyz);
i = ScalarIndexOfLargestMagnitudeIn(n);
n = n / n[i]; // intersect with cube surface
uv = DeleteScalarAtIndex(n, i);
face_index = i*2 + IsNegative(n[i]);
pixel = Texture2DRead(cube_maps[face_index], uv * axis_flip_rule[face_index] + vec2(0.5, 0.5));

Where ScalarIndexOfLargestMagnitudeIn returns the index of the largest scalar in the vector (0=x, 1=y, 2=z)

DeleteScalarAtIndex returns a 2D vector from a 3D vector with scalar number i removed. (i=0: return (y,z), i=1: return (x,z), i=2: return (x,y))

IsNegative returns 0 for positive value, 1 for negative value

There's also axis_flip_rule to flip the uv axis according to the face index that depend on which API you're using (ie: DirectX and OpenGL are different).

That can be implemented by flipping the 2D textures in memory in advance if you're doing it in software and left out of the texture lookup code.