I was wondering how 3d engines like Unity or OpenGL added textures to objects.
How would Unity add this texture to the side the arrow is pointing to?
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It only takes a minute to sign up.Sign up to join this community
This deserves a very broad answer because texturing is historically THE one big job of graphics renderers.
Put shortly, this happens during rasterization. Rasterization is a process in the rendering pipeline, where the graphic card generates pixels (fragments) from geometry and then passes it on to the next stage of the pipeline, the pixel (fragment) shader, where the texture is read from memory, mixed with some color operation and put into the output for the next stage of the pipeline to treat (output merging).
So more in depth, it goes like that: the artist is first responsible in this system, he designs the texture coordinates in the modeling software, called DCCT (digital content creation tool), a widely used example is Maya (Autodesk).
In the case of the cube, you can imagine the UV being (0,0)(0,1)(1,1)(1,0) for one quad.
An exporter will take the format internal to maya and write to file (.fbx for example), then the game engine will read that format, maybe using fbx library, and load in the graphics card memory, using vertex buffer locking,
memcpy, and unlock. The texture coordinates are part of the vertex buffer. Then same for the texture in a separate piece of code, and finally at least one shader has to be uploaded in the graphics card too. this shader will probably just declare that it reads the texture by declaring a "sampler" in the uniform list.
Later when the engine runs, for one frame, it will call "draw primitive" routine which triggers the graphic card to run the pipeline.
This pipeline starts by fetching vertex data from the bound vertex stream (the one we uploaded before), then it goes into primitive assembly stage, which makes batches to create triangles. Then it undergoes transformation from 3d to 2d in the vertex shader. And this is when the rasterizer comes in. The rasterizer is a fully 2d process, which knows the resolution of the render target, and it takes the projected coordinate (given by vertex shader stage before) and puts them on their place on screen. this gives 3 points hopefully (there are details such as degenerated forms and clipping we can omit for now). The rasterizer will emit fragments by scanning the whole void between those 3 points, the hardware scheduler takes each emitted fragment by batch of 2x2 and distributes it to a free shader core. The shader core, will execute the pixel shader with some function parameter given by the rasterizer. These are the
varying variables. They are called varying because they vary between the 3 vertices, usually linarly in 3d space, which means "perspective corrected" in 2d space. The varyings comprise the UV coordinates, therefore the pixel shader knows what texel to fetch in the texture to put at this pixel location. (the pixel pos is the
SV_screenPos stuff determined at the time of fragment emission; one shader core executes code for a fixed screen position).
So what happens in a texturing shader is that the shader code will call
tex2d intrinsic to require the texture unit to give a texel at the UV read from the vertex stream and interpolated by the rasterizer. The texture unit is responsible for executing filtering too, the UV are in floating point format so the actual point in the texture is usually not exactly the center of a texel, that is why it gets filtered by interpolating between neighbor pixels. There are multiple kinds of filtering, magnification, minification, and inter-mipmap.
Finally the result of the
tex2d is the color of the texel, so the pixel shader just has to copy it to output, it can do more if it wants, like apply lighting.
At this point this is done basically, the output of the pixel shader is written to the render target by the ROPs, if some other geometry is not drawn over this later (other objects coming in front), this is what you will see at this pixel in the end, when swapping buffers (
additional paragraph about texture storage, a texture is a byte array, each pixel is stored in a 3x8 bits zone, or 4x8 bits for texture containing transparency. Same truth for an image file on disk. When the texture is loaded from file to memory, it is just a long flat series of bytes to which you can access one pixel using
p=width*y+x formula. Which means the layout is "line by line, scanning horizontally to the right". In the graphic card however, the internal storage is a Z-curve (morton code), which is mandated by the need of vertical cache locality.
The procedure is called UV Mapping or Coordinates. Here's a good tutorial for OpenGL: