Skip to main content
Game Development

Return to Answer

added 235 characters in body
Source Link
Mooing Duck
Mooing Duck
  • 255
  • 3
  • 10

The oldest "3d" games straight up did not take lighting into accountstraight up did not take lighting into account, as they were rendered on the CPU, which lacked the processing power to do much per-pixel . The developers just added the shadows to the objects textures directly. Computers generally ran at a resolution around 640 × 480, and if you had a target framework 24fps, and wanted to set each pixel separately, then they had to run in a loop, which was 22,118,400 operations per second. Computers at the time could only do about 8,000,000 operations per second, so even if just setting all pixels to hardcoded white one at a time wasn't viable at that resolution and framerate.

Obviously, most rays won't actually hit a light source, so many implementations add some variant of "Photon mapping", where they first emit rays from the various light sources into the scene, which reflect around like normal ray tracing, and keep track of how much of which colors of lights are in each general area, in order to "light" a scene. After doing this for ~half a framesome small number of bounces, they then do normal ray tracing. This is much more complex to design, and takes a lot more memory to store the lighting information, but results in theoryrays detecting light dramatically more often, can calculateand greatly reducing noise, and thus quality. Due to the same imagecomplexity, a lot of implementations instead just calculate a "shadow map" with 50% less ray processing. Or alternativelyonly a tiny number of bounces, be twiceonce every few frames, which works fine as bright and halflong as noisy forlights don't move fast while close to the same amount of ray processingcamera.

The oldest "3d" games straight up did not take lighting into account, as they were rendered on the CPU, which lacked the processing power to do much per-pixel . The developers just added the shadows to the objects textures directly. Computers generally ran at a resolution around 640 × 480, and if you had a target framework 24fps, and wanted to set each pixel separately, then they had to run in a loop, which was 22,118,400 operations per second. Computers at the time could only do about 8,000,000 operations per second, so even if just setting all pixels to hardcoded white one at a time wasn't viable at that resolution and framerate.

Obviously, most rays won't actually hit a light source, so many implementations add some variant of "Photon mapping", where they first emit rays from the various light sources into the scene, which reflect around like normal ray tracing, and keep track of how much of which colors of lights are in each general area, in order to "light" a scene. After doing this for ~half a frame, they then do normal ray tracing. This is much more complex to design, and takes a lot more memory to store the lighting information, but in theory, can calculate the same image with 50% less ray processing. Or alternatively, be twice as bright and half as noisy for the same amount of ray processing.

The oldest "3d" games straight up did not take lighting into account, as they were rendered on the CPU, which lacked the processing power to do much per-pixel . The developers just added the shadows to the objects textures directly. Computers generally ran at a resolution around 640 × 480, and if you had a target framework 24fps, and wanted to set each pixel separately, then they had to run in a loop, which was 22,118,400 operations per second. Computers at the time could only do about 8,000,000 operations per second, so even if just setting all pixels to hardcoded white one at a time wasn't viable at that resolution and framerate.

Obviously, most rays won't actually hit a light source, so many implementations add some variant of "Photon mapping", where they first emit rays from the various light sources into the scene, which reflect around like normal ray tracing, and keep track of how much of which colors of lights are in each general area, in order to "light" a scene. After doing this some small number of bounces, they then do normal ray tracing. This is more complex to design, and takes a lot more memory to store the lighting information, but results in rays detecting light dramatically more often, and greatly reducing noise, and thus quality. Due to the complexity, a lot of implementations instead just calculate a "shadow map" with only a tiny number of bounces, once every few frames, which works fine as long as lights don't move fast while close to the camera.

Source Link
Mooing Duck
Mooing Duck
  • 255
  • 3
  • 10

I'm not a game dev, but I did take a class on this in college, and can give a brief tldr.

1990-1995 - CPU Rendering

The oldest "3d" games straight up did not take lighting into account, as they were rendered on the CPU, which lacked the processing power to do much per-pixel . The developers just added the shadows to the objects textures directly. Computers generally ran at a resolution around 640 × 480, and if you had a target framework 24fps, and wanted to set each pixel separately, then they had to run in a loop, which was 22,118,400 operations per second. Computers at the time could only do about 8,000,000 operations per second, so even if just setting all pixels to hardcoded white one at a time wasn't viable at that resolution and framerate.

Games used insane tricks to calculate 3d-like graphics at very low framerates and very low resolutions. One of the most fastest of these was made by John Carmack, "Wolfenstein 3d" ran at 320x200 at roughly 20fps. Wolfenstein3d (1992) pulled this off by only having 3d walls, straight up and down. If you look closely, it doesn't actually even have floors or ceilings. Everything else was 2d sprites. In 1993, John Carmack made an even better "Doom" engine in 1993, which was able to add sloped floors and ceilings, but walls were still perfectly vertical. Each wall/ceiling/floor could have hardcoded light+color set as part of the level.

1995 - 2007 - Fixed Pipeline GPUs

In 1995/1996, the Nintendo 64, PlayStation, and Sega Saturn released, with dedicated "GPUs", and shortly therafter, the first consumer personal computer dedicated 3D graphics cards became available. These generally work in a relatively straightforward graphics pipeline. First, The CPU sends commands to the GPU telling it what to add to the next frame. These usually include:

  • where the camera is (as a transform matrix, handling camera location, size, rotations, and other weird scaling things)
  • Ambient light data (color + intensity).
  • objects:
    • The address of a 3d model
      • vertex (corner) coordinates and "normal vectors".
      • faces (trianges) connecting these vertexes.
    • The address of the textures to use for the model.
    • A transform matrix (This handles location, size, rotations, and other weird scaling things).
    • Directional light source data (direction + color + intensity).
    • other data.

Then the pipeline runs:

  1. Projection: For each "vertex" (corner) in each model, the GPU multiplies the 3d coordinates by the model transform matrix, to figure out where this vertex actually is "in the space", and the camera transform matrix, to figure out where this vertex actually is relative to the camera. (GPUs actually merge this into a single step).
  2. Lighting: For each "vertex" (corner) in each model, The GPU looks up the "normal" vector (aka "what direction is this point on the object facing"), and calculates the lighting for that vertex:
    • "ambient light": the light that bounces everywhere, setting a minimum lighting in a space. This is usually just passed in, not calculated.
    • "directional light": The light from the light source to use. Usually the sun, but sometimes toward a nearby light source. The more the point faces the light source, the more of this light it "absorbs". If it faces perpendicular or away, then it does not absorb any directional light.
    • "camera light": Weirdly, things look better if we calculate as if the camera itself were projecting a light. The more the point faces the camera, the more of this color it "absorbs". If it faces perpendicular or away, then it does not absorb any camera light.
  3. Clipping: For each "face" (triangle) in each model, the GPU discards any triangles outside of the camera, or facing away from the camera.
  4. Window-Viewport transformation: For each "vertex" (corner) in each model, a final matrix transformation is calculated, converting the relative camera coordinates to 3d pixel coordinates.
  5. Rasterization: For each "face" (triangle) in each model:
    • the GPU looks up the pixel coordinates and lighting of its vertexes, and the texture for the face.
    • For each pixel in that face:
      • The GPU checks if a color has already been chosen for this pixel that is closer to the camera. if so, it skips this pixel.
      • The GPU looks up the texture color for that point in the triangle.
      • The GPU interpolates the lighting for that point based on how close it is to each of the three corners.
      • The GPU stores the pixel color, including how far it is from the camera.

2000-2007 Shaders

Around 2000, GPUs started releasing where the Rasterization step wasn't directly wired, but instead ran the Rasterization step as a program (aka "Pixel Shader") on GPU processors. This also allowed applications to replace the program, to do customized shading of the pixel. This could be used to calculate textures at runtime, reducing code size and memory usage, and making the textures dynamic. Several games used this to make realistic pupils that always watched the camera/player.

Almost immediately after, GPUs also did the same for the Projection step, allowing applications to replace the Projection with a custom "Vertex shader" . This enabled programs to make models "morph", allowing super smooth 3d animation, rather than simply rendering.

Around 2007, GPUs also added "Graphics Shaders" (aka "Tesselation Shaders"), which allowed the CPU to send simple instructions, and then the GPU would calculate or modify the model on the fly, allowing it to make Models more or less detailed depending on how far they were from the camera. Technically Tesselation Shaders are different, but I honestly don't understand the difference.

2008-2018 - Unified Shaders

By 2008, GPUs had transformed into just thousands of tiny processors each capable of running a sequence of shader programs, fully customizable. Each processor is ~10x slower than the CPU, but even low-end integrated laptop GPUs have ~96 cores. High end integrated graphics have ~384 cores, and the GeForce RTX 4090 has 16,384 cores. At this time, people began using these to calculate things besides graphics, including, bitcoin mining, and also Artificial Intelligence.

2018-Present - Realtime Ray Tracing + AI

And now we finally get back to something involving 3d lighting. GPUs added specialized processors for calculating Ray Tracing, which works almost 100% differently.

  • The GPU imagines a screen with a low resolution, often about a quarter of the real resolution.
  • The GPU imagine tens or hundreds of "rays" shooting out of each Pixel. The closer to perfectly straight they are, the more "important" they are.
  • The GPU calculates which triangle each would hit, and where on that triangle.
  • The GPU calculates the texture color for that point, and the "shininess". It also calculates the normal vector.
  • The GPU "reflects" the ray over the normal vector, tints it a color pased on the texture color and shininess, separates into tens or hundreds of rays, and repeats, with an adjusted "importance" depending on the shininess. Some implementations may deliberately try to aim some of the reflected rays at light sources, which can greatly improve performance and quality, but greatly interferes with proper reflections, which kind of defeats the point of Ray Tracing, unless you do some tricky statistical math to compensate.
  • This continues until a ray hits a light source, or it runs out of time.
  • The GPU takes the weighted average color for each ray depending on their importance.
  • This results in a very dark and fuzzy image, so the final step is to use AI to brighten the image, sharpen the edges, and upscale it to the final resolution.

Ray Tracing can produce perfect curves, since it's not actually limited to triangles, and also produce incredibly realistic reflections and lighting, which are the focus of your question. But they're incredibly processor intensive.

Another note is "volume tracing", where rays are allowed to go through semi-transparent triangles, and may emit additional rays as they move "through" things, giving realistic smoke, clouds, or glass.

Obviously, most rays won't actually hit a light source, so many implementations add some variant of "Photon mapping", where they first emit rays from the various light sources into the scene, which reflect around like normal ray tracing, and keep track of how much of which colors of lights are in each general area, in order to "light" a scene. After doing this for ~half a frame, they then do normal ray tracing. This is much more complex to design, and takes a lot more memory to store the lighting information, but in theory, can calculate the same image with 50% less ray processing. Or alternatively, be twice as bright and half as noisy for the same amount of ray processing.