Game Development Stack Exchange is a question and answer site for professional and independent game developers. It only takes a minute to sign up.
Sign up to join this community
I'm trying to do an experiment. I'd like to do some object masking to the assets inside of a game, so during run time, when you are playing, they're texture maps are replaced by the masks.
The masks will literally be just color maps. They are supposed to be labels for a machine learning program.
Preferably if one could replace the texture maps with color masks during runtime that would be better.
Typically this is done via DLL injection.
You insert a piece of custom software that poses as the graphics driver while you're running the game. Each frame, when the game tries to ask the graphics driver:
please set the texture sampler to use the tree texture and then draw this batch of triangles
...your software receives that message, records it, then passes it on to the real graphics driver to do the work.
In a second pass, you can take those recorded rendering instructions and play them back, to reproduce each drawn frame. Or you can modify them to alter the frame: for instance, you can recognize the ID of the tree texture, and and substitutes a modified instruction when passing the message along to the real graphics driver for playback:
please set the mask shader to draw pure-green and then draw this batch of triangles
This is the general idea behind how the work behind papers like "Playing for Data: Ground Truth from Computer Games" was done. You can read the paper for more details, and examine the code they used for their particular injection.
As Romen says, techniques like this can also be used to exploit games in ways that their creators might object to, so I won't go further into a step-by-step walkthrough in a public forum like this, but you can find more information on your own by following those keywords.
