13
\$\begingroup\$

So since 256 color mode is depreciated and no longer supported under Direct3D mode, I got the idea to use a pixel shader instead to simulate the NES palette of all possible colors so that fading objects and whatnot don't have smooth fade outs with alpha channels. (I know objects couldn't really fade out on the NES, but I have all objects that do fade in and out on a solid black background, which would be possible with palette swapping. Also, the screen fades in and out when you pause which I know also was possible with palette swapping as it was done in a few Mega Man games.) Problem is, I know next to nothing about HLSL shaders.

How do I do it?

\$\endgroup\$
3
  • \$\begingroup\$ The comment below describes the approach using the pixel shader, but simply a color limitation like this could be achieved using an appropriate style guide for your art team. \$\endgroup\$
    – Evan
    Feb 17, 2013 at 3:48
  • \$\begingroup\$ do you just want to reduce palette or also want to perform dithering (possible using shaders too). otherwise zezba9000 answer seems the best one to me \$\endgroup\$
    – tigrou
    Feb 17, 2013 at 12:08
  • \$\begingroup\$ I'm wanting to just reduce the possible colors to the NES palette. I needed it mainly to catch alpha channel effects and other ink effects because in 256 color mode it catches them but Direct3D no longer supports 256 color mode so in it the effects are smoothed out in true colors. \$\endgroup\$ Feb 18, 2013 at 17:43

3 Answers 3

4
\$\begingroup\$

In the pixel shader you could pass in a 256x256 Texture2D with the pallet colors all lined up horizontally in a row. Then your NES textures would be converted to direct3D Texture2Ds with ever pixel converted to a 0-255 index value. There is a texture format that only uses the red value in D3D9. So the texture would only take up 8bits per pixel, but the data that comes into the shader would be from 0-1.

// Pixel shader might look like this:

float4 mainPS() : COLOR0
{
    float4 colorIndex = tex2D(MainTexture, uv);
    float4 palletColor = tex2D(PalletTexture, float2(colorIndex.x, 0);
    return palletColor;
}

EDIT: A more correct way would be to add in all the blending pallet version you need aligned vertically in the texture and referencing them with your colorIndex's 'alpha' value:

float4 mainPS() : COLOR0
{
    float4 colorIndex = tex2D(MainTexture, uv);
    float4 palletColor = tex2D(PalletTexture, float2(colorIndex.x, colorIndex.a);
    return palletColor;
}

A third way would be to just fake the NES low fade quality by toon shading the alpha color:

float4 mainPS() : COLOR0
{
    float4 colorIndex = tex2D(MainTexture, uv);
    float4 palletColor = tex2D(PalletTexture, float2(colorIndex.x, 0);
    palletColor.a = floor(palletColor.a * fadeQuality) / fadeQuality;
    //NOTE: If fadeQuality where to equal say '3' there would be only 3 levels of fade.
    return palletColor;
}
\$\endgroup\$
5
  • 1
    \$\begingroup\$ You mean 256x1, not 256x256, I'm guessing? Also, be sure to disable bilinear filtering on both textures, else you'll get blending between palette "entries". \$\endgroup\$ Feb 17, 2013 at 15:27
  • \$\begingroup\$ Also, you can't do any kind of lighting or any math on the texture values with this scheme, since anything you do is likely to just send you to a completely different part of the palette. \$\endgroup\$ Feb 17, 2013 at 15:33
  • \$\begingroup\$ @Nathan Reed You can to do lighting. You just calculate light on the "palletColor" before you return its color value. Also you could make a 256x1 texture, but the hardware may just under the hood use 256x256 anyway and 256x256 is the fastest size to use on most hardware. Unless thats changed idk. \$\endgroup\$
    – zezba9000
    Feb 18, 2013 at 16:56
  • \$\begingroup\$ If you do lighting after palettization then it probably won't even be one of the NES colors anymore. If that's what you want, that's fine - but that doesn't sound like what the question was asking for. As for the 256 thing, that's possible if you have a GPU over 10 years old...but anything more recent than that will certainly support rectangular power-of-2 textures, like 256x1. \$\endgroup\$ Feb 18, 2013 at 17:40
  • \$\begingroup\$ If he just doesn't want smooth fadeouts he could do: "palletColor.a = (float)floor(palletColor.a * fadeQuality) / fadeQuality;" He could even do the same thing as the 3D texture method but with a 2D texture by changing line 4 to: "float4 palletColor = tex2D(PalletTexture, float2(colorIndex.x, colorIndex.a);" The alpha channel just indexes different pallet layers on a single 2D texture. \$\endgroup\$
    – zezba9000
    Feb 18, 2013 at 19:17
3
\$\begingroup\$

If you don't really care about texture memory usage (and the idea of blowing an insane amount of texture memory to achieve a retro look has a kind of perverse appeal) you could build a 256x256x256 3d texture mapping all RGB combinations to your selected palette. Then in your shader it just becomes one line of code at the end:

return tex3d (paletteMap, color.rgb);

It may not even be necessary to go all the way to 256x256x256 - something like 64x64x64 may be sufficient - and you could even change palette mappings on the fly using this method (but at significant cost owing to a large dynamic texture update).

\$\endgroup\$
4
  • \$\begingroup\$ This may be the best method if you want to do lighting, alpha-blending, or any other type of math on your textures, and then snap the end result to the nearest NES color. You could precompute your volume texture using a reference image like this one; with it set to nearest-neighbor filtering, you might well get away with something as small as 16x16x16, which wouldn't be much memory at all. \$\endgroup\$ Feb 17, 2013 at 15:49
  • 1
    \$\begingroup\$ This is a cool idea, but will be much slower and not as compatible with older hardware. 3D Textures will sample much slower then 2D ones and 3D Textures will also require much more bandwidth which will slow it down even more. Newer cards this wont matter though, but still. \$\endgroup\$
    – zezba9000
    Feb 18, 2013 at 17:02
  • 1
    \$\begingroup\$ Depends how old you want to go. I believe 3d texture support goes back to the original GeForce 1 - is 14 years old enough? \$\endgroup\$ Feb 18, 2013 at 18:48
  • \$\begingroup\$ Lol Well yes maybe they do, never used those cards, guess I was thinking more inline with Phone GPU's. There are plenty of targets today that don't have 3D texture support. But because he is using D3D and not OpenGL guess even WP8 supports this. Still a 3D texture would take up more bandwidth then a 2D one. \$\endgroup\$
    – zezba9000
    Feb 19, 2013 at 3:29
1
\$\begingroup\$

(both of my solution works only if you dont care about changing palletes on the fly using shaders)

You can use any type of texture and do just a simple computation on a shader. Trick is that you have more color information than you need, so what you will just get rid of information that you dont want to.

8bit color is in formar RRRGGGBB. Which gives you 8 shades of red and green and 4 shades of blue.

This solution will work for any RGB(A) color format textures.

float4 mainPS() : COLOR0
{
    const float oneOver7 = 1.0 / 8.0;
    const float oneOver3 = 1.0 / 3.0;

    float4 color = tex2D(YourTexture, uvCoord);
    float R = floor(color.r * 7.99) * oneOver7;
    float G = floor(color.g * 7.99) * oneOver7;
    float B = floor(color.b * 3.99) * oneOver3;

    return float4(R, G, B, 1);
}

note: i wrote that from the top of my head, but im really sure it will compile and work for you


Another possibility, would be to use D3DFMT_R3G3B2 texture format that is actually same as 8bit graphics. When you put data to this texture, you can use simple bit operation per byte.

tex[index] = (R & 8) << 5 + ((G & 8) << 2) + (B & 4);
\$\endgroup\$
2
  • \$\begingroup\$ Thats a good example. Only I think he needed to be able to swap out the color pallet. In that case he would have to use something like my example. \$\endgroup\$
    – zezba9000
    Feb 17, 2013 at 8:23
  • \$\begingroup\$ This is not the NES colour palette at all. The NES didn’t use 8-bit RGB, it used a fixed palette of about 50 to 60 colours in the YPbPr space. \$\endgroup\$ Feb 17, 2013 at 10:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .