2
\$\begingroup\$

I'm trying to do an experiment. I'd like to do some object masking to the assets inside of a game, so during run time, when you are playing, they're texture maps are replaced by the masks.

The masks will literally be just color maps. They are supposed to be labels for a machine learning program.

Preferably if one could replace the texture maps with color masks during runtime that would be better.

\$\endgroup\$
2
  • 1
    \$\begingroup\$ I don't think this question is within the scope of this stack exchange community because it can easily lead into a discussion about breaking the ToS for a game. What game do you want to replace textures for? There is no generic technique because it depends on the texture format, graphics API, game engine, etc. \$\endgroup\$
    – Romen
    Jan 10 '20 at 22:45
  • 1
    \$\begingroup\$ Well I hadn't thought about which game to use. I'll be considerate of the terms. It's mainly for private use anyway. \$\endgroup\$
    – lazylama
    Jan 11 '20 at 23:51
4
\$\begingroup\$

Typically this is done via DLL injection.

You insert a piece of custom software that poses as the graphics driver while you're running the game. Each frame, when the game tries to ask the graphics driver:

please set the texture sampler to use the tree texture and then draw this batch of triangles

...your software receives that message, records it, then passes it on to the real graphics driver to do the work.

In a second pass, you can take those recorded rendering instructions and play them back, to reproduce each drawn frame. Or you can modify them to alter the frame: for instance, you can recognize the ID of the tree texture, and and substitutes a modified instruction when passing the message along to the real graphics driver for playback:

please set the mask shader to draw pure-green and then draw this batch of triangles

This is the general idea behind how the work behind papers like "Playing for Data: Ground Truth from Computer Games" was done. You can read the paper for more details, and examine the code they used for their particular injection.

As Romen says, techniques like this can also be used to exploit games in ways that their creators might object to, so I won't go further into a step-by-step walkthrough in a public forum like this, but you can find more information on your own by following those keywords.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.