Using GetRawTextureData<Color32>() in Unity with non RGBA32 textures?

Question

I am trying to manipulate a Texture2D using GetRawTextureData<Color32>() as this does not require extra memory to be allocated like GetPixels32() does.

However, I can't make sense of the data it is providing or how to use it.

GetRawTextureData<Color32>() is directly compatible with RGBA32 Textures as per: "RGBA32 texture format data layout exactly matches Color32 struct" at this page.

But if a Texture2D is loaded from a JPG byte Array and it has an RBG24 format, the output it provides is nonsense:

        var texelsReal = texture.GetPixels32();
        var texelsColor32 = texture.GetRawTextureData<Color32>();
        var texelsColor = texture.GetRawTextureData<Color>();

        for (int i = 0; i < 200; i++) {
            Debug.Log("TEXEL REAL: " + texelsReal[i] + " || TEXEL COLOR32: " + texelsColor32[i] + " TEXEL COLOR: " + texelsColor[i] + " || texture format " + texture.format);
        }

This will output something like:

TEXEL REAL: RGBA(120, 146, 233, 255) || TEXEL COLOR32: RGBA(225, 146, 190, 174) || TEXEL COLOR: RGBA(0.003, 0.000, 0.000, 183.106) ||Texture Format RGB24

Only the GetPixels32() version is actually legitimate and the others are total nonsense.

I need GetRawTextureData<T> to contain real data, as I want to then reallocate pixels by shifting them around using it as per this thread.

But when I try that currently, I get garbled nonsense because I'm moving around nonsense from one place to another and it then seems to apply that nonsense at the end and ruin the photo.

I tried converting the texture like this:

Texture2D copyTex = new Texture2D(photoThumbT2D.width, photoThumbT2D.height, TextureFormat.RGBA32, false);
bool succeeded = Graphics.ConvertTexture(photoThumbT2D, copyTex);
Debug.Log(" CONVERT SUCCESS " + succeeded + " FORMAT " + copyTex.format + " ORIG " + photoThumbT2D.format);
copyTex.Apply(true); //breaks it for unknown reasons...
photoThumbT2D = copyTex;

But this just creates a grey texture. I was instructed in the other thread to use a struct like:

public struct RGB24 { public byte r, g, b }

Then presumably:

var texels = texture.GetRawTextureData<RBG24>();

But this gives me an error saying no implicit conversion is possible. I don't know how to solve that if I can.

Any solutions? Thanks.

Answer 1

Just make your method generic on the texel data type:

static void RotateImage<T>(Texture2D tex, float angleDegrees) where T:struct {
    int width = tex.width;
    int height = tex.height;
    float halfHeight = height * 0.5f;
    float halfWidth = width * 0.5f;

    var texels = tex.GetRawTextureData<T>();
    var copy = System.Buffers.ArrayPool<T>.Shared.Rent(texels.Length);
    Unity.Collections.NativeArray<T>.Copy(texels, copy, texels.Length);

    int oldX;
    int oldY;

    float phi = Mathf.Deg2Rad * angleDegrees;
    float cosPhi = Mathf.Cos(phi);
    float sinPhi = Mathf.Sin(phi);

    int address = 0;
    for (int newY = 0; newY < height; newY++)
    {
        for (int newX = 0; newX < width; newX++)
        {
            float cX = newX - halfWidth;
            float cY = newY - halfHeight;
            oldX = Mathf.RoundToInt(cosPhi * cX + sinPhi * cY + halfWidth);
            oldY = Mathf.RoundToInt(-sinPhi * cX + cosPhi * cY + halfHeight);
            bool InsideImageBounds = (oldX > -1) & (oldX < width) & (oldY > -1) & (oldY < height);


            texels[address++] = InsideImageBounds ? copy[oldY * width + oldX] : default;
        }
    }
    tex.Apply(true);

    System.Buffers.ArrayPool<T>.Shared.Return(copy);
}

Then choose what underlying type to use to match the number of bytes in your texture format (a few examples shown, but not exhaustive):

public static void RotateImage(Texture2D tex, float angleDegrees)
{
    switch (tex.format) {
        // 1 byte per texel.
        case TextureFormat.Alpha8:
            RotateImage<byte>(tex, angleDegrees);
            break;

        // 2 bytes per texel.
        case TextureFormat.ARGB4444:
        case TextureFormat.R16:
            RotateImage<ushort>(tex, angleDegrees);
            break;

        // 3 bytes per texel.
        case TextureFormat.RGB24:
            RotateImage<(byte, byte, byte)>(tex, angleDegrees);
            break;

        // 4 bytes per texel.
        case TextureFormat.ARGB32:
        case TextureFormat.RGBA32:
            RotateImage<Color32>(tex, angleDegrees);
            break;

        // 4 floats = 16 bytes per texel.
        case TextureFormat.RGBAFloat:
            RotateImage<Color>(tex, angleDegrees);
            break;
        
        // TODO: Other formats, as needed.
        default: 
            Debug.Log($"Unsupported texture format: {tex.format}");
            break;
    }
}

I've been quick and dirty here, lumping together formats that use "similar enough" data layouts, in terms of the byte stride between texels. This is good enough if you just want to copy the whole texel value from one place to another, but can land you in trouble if you want to inspect individual component values. For instance, if your image is in ARGB format, casting it to Color32 will put its alpha value in the .r component, red in the .g, green in the .b, and blue in the .a.

If that matters to you, you can define your own structs with the needed layouts:

public struct Color32ARGB {
    public byte a;
    public byte r;
    public byte g;
    public byte b;
}

public struct Color24RGB {
    public byte r;
    public byte g;
    public byte b;
}

...etc., and use those.

---

Or, you can just use bytes for everything:

public static int BytesPerTexel(TextureFormat format)
{
        switch (tex.format) {
        // 1 byte per texel.
        case TextureFormat.Alpha8:
            return 1;

        // 2 bytes per texel.
        case TextureFormat.ARGB4444:
        case TextureFormat.R16:
            return 2;

        // 3 bytes per texel.
        case TextureFormat.RGB24:
            return 3;

        // 4 bytes per texel.
        case TextureFormat.ARGB32:
        case TextureFormat.RGBA32:
            return 4;

        // 4 floats = 16 bytes per texel.
        case TextureFormat.RGBAFloat:
            return 16;

        // TODO: Other formats, as needed.
        default: 
            Debug.Log($"Unsupported texture format: {tex.format}");
            return -1;
    }
}

static void RotateImage(Texture2D tex, float angleDegrees) {

    int bytesPerTexel = BytesPerTexel(tex.format);
    int width = tex.width;
    int height = tex.height;
    float halfHeight = height * 0.5f;
    float halfWidth = width * 0.5f;

    var texels = tex.GetRawTextureData<byte>();
    var copy = System.Buffers.ArrayPool<byte>.Shared.Rent(texels.Length);
    Unity.Collections.NativeArray<byte>.Copy(texels, copy, texels.Length);

    int oldX;
    int oldY;

    float phi = Mathf.Deg2Rad * angleDegrees;
    float cosPhi = Mathf.Cos(phi);
    float sinPhi = Mathf.Sin(phi);

    int address = 0;
    for (int newY = 0; newY < height; newY++)
    {
        for (int newX = 0; newX < width; newX++)
        {
            float cX = newX - halfWidth;
            float cY = newY - halfHeight;
            oldX = Mathf.RoundToInt(cosPhi * cX + sinPhi * cY + halfWidth);
            oldY = Mathf.RoundToInt(-sinPhi * cX + cosPhi * cY + halfHeight);
            bool InsideImageBounds = (oldX > -1) & (oldX < width) & (oldY > -1) & (oldY < height);


            if (InsideImageBounds) {
                int from = oldY * width + oldX;
                for (int i = 0; i < bytesPerTexel; i++)
                    texels[address++] = copy[from++];
            } else {
                for (int i = 0; i < bytesPerTexel; i++)
                    texels[address++] = default;
            }
        }
    }
    tex.Apply(true);

    System.Buffers.ArrayPool<byte>.Shared.Return(copy);
}