So far, my understanding is that you need a direction vector, and from the direction vector we can determine the point of intersection with one of the six planes.
Basically, yes.
Is the direction vector is the players view direction?
Any direction. In your shader code you can query the cubemap with any direction you want. It does not have to be where the player is looking.
For instance, aside from using cubemaps for skyboxes, one common use is reflections. For abstract, you compute the direction for the reflection and query there.
I do not understand how having only one ray will give us the texture to fill the whole screen?
If you want to fill the screen with a cubemap, you would do a raycasting setup.
The basic setup is as follows:
- Set a squad to fill the screen.
- In your fragment shader, compute the direction for the ray and query there.
Alright, let me qualify that…
The fragment shader will run for every pixel (well, every fragment) of the screen, in parallel. Now, in a perspective projection, each pixel will be the projection of a line that reaches the camera. You want the direction of that line. That is what you query the cubemap with.
How do you get that direction? Imagine you are projecting the world into the near plane of the camera. If you know the size of the output, the position of the current pixel, and the distance to the nearplane, you can compute the position of the point in the near plane. Well, the direction from the camera to that point.

The viewing frustrum - Intergraph Computer Systems.
For anybody doing this in OpenGL, know, I kind of made it more complex that it is.
If you are going to fill the screen with a squad, and you need to find points in the near plane… Make the squad match the near plane, with the camera at the origin, and you solve this neatly in the vertex shader:
#version 330 core
layout (location = 0) in vec2 position;
uniform mat4 inverseProjection;
uniform mat4 inverseView;
out lowp vec3 cameraRayDirection;
void main()
{
gl_Position = vec4(position.xy, 0.0, 1.0);
cameraRayDirection = ((inverseView * inverseProjection) * gl_Position).xyz;
}
In actuality, you only need two coordinates, don't you? I made position vec2 in this code, yes, it works.
In the fragment shader, you get them interpolated, and ready to use.
Furthermore, what if we are looking at one of the edge of the cube? Now we have to sample texture from two faces?
You don't have to worry about it. OpenGL understands cubemaps. You can bind your textures with GL_TEXTURE_CUBE_MAP
, set the six faces, and your shader code does not have to worry about it. Please refer to a tutorial on uploading textures with OpenGL.
In your shader program, you define your texture sampler as samplerCube, for example:
uniform samplerCube SkyBox;
And then query using an unit vector:
return texture(SkyBox, direction);
Now, if you wanted to emulate this in CPU, not using any shader code… Then the cubemap probably isn't the best format.
You can convert the texture to a cylindrical form that you may query with the angles of the direction you want. See Converting to/from cubemaps.
Baring that, using the cubemap would require to find the intersection of the unit vector with the cube.