Typically this is done via DLL injection.
You insert a piece of custom software that poses as the graphics driver while you're running the game. Each frame, when the game tries to ask the graphics driver:
please set the texture sampler to use the tree texture and then draw this batch of triangles
...your software receives that message, records it, then passes it on to the real graphics driver to do the work.
In a second pass, you can take those recorded rendering instructions and play them back, to reproduce each drawn frame. Or you can modify them to alter the frame: for instance, you can recognize the ID of the tree texture, and and substitutes a modified instruction when passing the message along to the real graphics driver for playback:
please set the mask shader to draw pure-green and then draw this batch of triangles
This is the general idea behind how the work behind papers like "Playing for Data: Ground Truth from Computer Games" was done. You can read the paper for more details, and examine the code they used for their particular injection.
As Romen says, techniques like this can also be used to exploit games in ways that their creators might object to, so I won't go further into a step-by-step walkthrough in a public forum like this, but you can find more information on your own by following those keywords.