I know what raytracing is, but I'm a bit curious how games are able to reflect off-screen geometry using raytracing in real-time.
I tried looking up the technical details but most of the articles are "what it is"
The basic algorithm for raycasting is actually surprisingly simple. In pseudocode it is:
for each pixel on the screen
calculate an angle which begins at the camera and crosses through that pixel on the near-plane of the camera
trace a ray from the camera position with that angle through the 3d geometry of your scene until you hit an object
if the object is not reflective
color the pixel in the color of the object
calculate the reflection angle at the impact point
continue tracing the ray from the impact point with the new angle
The reflected ray might move out of the view frustum and still report back a color value which then gets applied to the screen pixel it originally originated from.
Decades ago I followed a tutorial about how to implement a basic CPU-based raytracing renderer and did it myself. I found that really illuminating. A real, modern raytracing engine would of course use the GPU for performance reasons, but doing it on the CPU as an academic exercise usually makes it easier to understand the basic principles if you are not familiar with GPU programming. All you need is some basic trigonomety and collision detection, which are both things most game developers should already be familiar with. I don't think it was more than 100 lines of code. It shouldn't be hard to find a similar tutorial in your favorite programming language.
The complicated stuff about raytracing starts when you want to implement light sources. Especially when you want to simulate them in real-time and not just have a pre-calculated lightmap. But that's outside of the scope of the question.
You can calculate the intersection of a ray with a triangle (you could use a lot of shapes, but since ray tracing usually needs a helping hand from rasterization, it's best to use those), that gives you a point on it. Then you can get the normal by taking the cross product of two vectors made out of any two sides of this triangle (and by additionally querying a normal map if needed). From there it's just a simple formula (this is a built-in in glsl, but I don't know about the shading language used for ray tracing)
This by itself is really fast, most modern computers can deal even with 5-6 bounces per ray. The part that's slow is shading the object correctly.
It's not possible to render a full, shaded scene with acceptable graphics in real time with consumer graphics cards using only ray tracing. Even stuff like that Quake 2 RTX port needs to do a lot of tricks to get a good framerate (it only does the ray tracing to part of the pixels every frame and guesses the others based on the surroundings. This can lead to some strange effects, but it works).
One neat detail to add here is that we can use hybrid strategies to get some of what raytracers are good at (reflection accuracy) and some of what rasterization is good at (raw speed / coverage in one pass)
We can first draw our scene with regular rasterization to determine which visible pixels have reflective materials, and the detailed surface normal at those points from the normal maps via the G-buffer.
Then we can fire reflection rays from those point reflectors, and intersect them against a set of proxy geometry we've used to approximate our scene content (eg. a bunch of oriented boxes or ellipsoids)
This is a depth-only raycast: we don't try to compute the exact surface colour or illumination at the struck point, all we need to know is how far down the ray did we hit something. This and the fact that we're checking only against very simple bounding shapes helps this part go very fast.
Now with an estimated hit point in hand, we can look up into a conventional rasterized reflection probe. But now since we know not just the direction of the reflected ray but also its depth, we can use this to perspective-correct our lookup, and get reflections that parallax more correctly as the viewpoint moves.
It's not a perfect mirror reflection because of the approximations we took, but it can handle semi-shiny/bumpy surfaces or shallow puddles very convincingly, without the "infinitely distant background" artifacts we normally get from reflection probes.
So this is the kind of flavour of thing you'll often see with realtime raytracing at present: using raycasts tactically, in tandem with rasterization techniques, to add the kinds of detail rays are great at capturing, but without the full complexity of rendering all shading through recursive ray bounces as we might do in offline renders.