Rotating a Unity Texture2D 90/180 degrees without using GetPixels32 or SetPixels32? (To avoid heap fragmentation)

Question

I have a script which is loading a photo from path into a texture (photoThumbT2D), then rotating it:

                byte[] byteArray = File.ReadAllBytes(path);
            
                UnityEngine.Object.Destroy(photoThumbT2D);
                photoThumbT2D = new Texture2D(0, 0);
                photoThumbT2D.LoadImage(byteArray);
                RotateImage(photoThumbT2D, -90);

Where the RotateImage function is adapted from here as:

public static void RotateImage(Texture2D originTexture, float angle) {
        int width = originTexture.width;
        int height = originTexture.height;
        float halfHeight = height * 0.5f;
        float halfWidth = width * 0.5f;

        Color32[] originPixels = originTexture.GetPixels32();
        Color32[] rotatedPixels = originTexture.GetPixels32();

        int oldX;
        int oldY;

        float phi = Mathf.Deg2Rad * angle;
        float cosPhi = Mathf.Cos(phi);
        float sinPhi = Mathf.Sin(phi);
    
        for (int newY = 0; newY < height; newY++) {
            for (int newX = 0; newX < width; newX++) {
                rotatedPixels[newY * width + newX] = new Color32(0, 0, 0, 0);
                int newXNormToCenter = newX - width / 2;
                int newYNormToCenter = newY - height / 2;
                oldX = (int)(cosPhi * newXNormToCenter + sinPhi * newYNormToCenter + halfWidth);
                oldY = (int)(-sinPhi * newXNormToCenter + cosPhi * newYNormToCenter + halfHeight);
                bool InsideImageBounds = (oldX > -1) && (oldX < width) && (oldY > -1) && (oldY < height);

                if (InsideImageBounds) {
                    rotatedPixels[newY * width + newX] = originPixels[oldY * width + oldX];
                }
            }
        }

        originTexture.Reinitialize(width, height);
        originTexture.SetPixels32(rotatedPixels);
        originTexture.Apply();

    }

However, this application of RotateImage is leading to what I believe is a heap fragmentation problem where the more times it runs the more memory usage piles up in the profiler. This is discussed here.

In my case, it manifests in Android only (no problem in Editor) as a steady increase in GC Used Memory (Tracked Memory, In Use) until the app crashes:

In the Memory Profiler I get a new chunk of Managed Heap created for every time it runs until everything crashes (here there about 13 identical blue chunks created):

Apparently the only solution is to use NativeArray<Color32> nativeArray = texture2D.GetRawTextureData<Color32>(); and work with that, but since the NativeArray is a direct representation of the Texture data, I'm not sure if it's possible mathematically to rotate it without making a new copy to work from. And if you make a new array copy of the actual color data, you will get the same heap problems as with GetPixels32 and SetPixels32.

Any thoughts or solutions?

---

Addendum

Unfortunately, DMGregory, I learned from your method but I am still having trouble with your solution.

MEMORY ACCUMULATION

I have tested, and even just the code:

        photoThumbT2D = Resources.Load<Texture2D>("bigphoto");
        photoThumbnailVE.style.backgroundImage = photoThumbT2D;


        buttonClickedEvent += delegate {
            Debug.Log("RUN CLICK");

            var texels = photoThumbT2D.GetRawTextureData<Color32>();
            var texelsCopy = System.Buffers.ArrayPool<Color32>.Shared.Rent(texels.Length);
            //Unity.Collections.NativeArray<Color32>.Copy(texels, texelsCopy, texels.Length);
            System.Buffers.ArrayPool<Color32>.Shared.Return(texelsCopy);
};

Continues to trigger memory accumulation in Unity for Android in the same manner as the OP code. Thus in even this manner the Rent and Return functions are leading to memory accumulation.

Can you think of any reason this would not be happening? I have Incremental GC collection turned on in Android. I can't make any sense of it. Might it be because it's being run as a delegate or the parent object running it was created by a coroutine (which should be long gone)?

I have tested further and this can be solved instead by using a static List to store the duplicate data in but I would love to know why any other method is causing this problem.

Answer 1

Two tricks we can use to get the RotateImage method down to zero allocations (in steady state):

• We'll use GetRawTextureData<T> to get a view into the texture's existing CPU-side memory buffer, to edit it directly without creating a new array to call SetPixels with.

• We'll use System.Buffers.ArrayPool to borrow and return a reusable array to store the old data. This can get recycled for future calls, rather than being claimed as garbage and recreated anew.

public static void RotateImage(Texture2D tex, float angleDegrees)
{
    int width = tex.width;
    int height = tex.height;
    float halfHeight = height * 0.5f;
    float halfWidth = width * 0.5f;

    var texels = tex.GetRawTextureData<Color32>();        
    var copy = System.Buffers.ArrayPool<Color32>.Shared.Rent(texels.Length);
    Unity.Collections.NativeArray<Color32>.Copy(texels, copy, texels.Length);

    float phi = Mathf.Deg2Rad * angleDegrees;
    float cosPhi = Mathf.Cos(phi);
    float sinPhi = Mathf.Sin(phi);

    int address = 0;
    for (int newY = 0; newY < height; newY++)
    {
        for (int newX = 0; newX < width; newX++)
        {
            float cX = newX - halfWidth;
            float cY = newY - halfHeight;
            int oldX = Mathf.RoundToInt(cosPhi * cX + sinPhi * cY + halfWidth);
            int oldY = Mathf.RoundToInt(-sinPhi * cX + cosPhi * cY + halfHeight);
            bool InsideImageBounds = (oldX > -1) & (oldX < width)
                                   & (oldY > -1) & (oldY < height);
            
            texels[address++] = InsideImageBounds ? copy[oldY * width + oldX] : default;
        }
    }

    // No need to reinitialize or SetPixels - data is already in-place.
    tex.Apply(true);

    System.Buffers.ArrayPool<Color32>.Shared.Return(copy);
}

You'll still see an allocation the first time you call this, or when you call it with a larger image than you've used before. That's from the ArrayPool allocating a new array for the requested size. But repeat calls will be able to re-use the array from the pool and so you won't see continually increasing memory use or fragmentation.

If ArrayPool isn't working as desired on mobile (maybe it's emulated as just new to avoid reserving large blocks of RAM on modest phones), you can get a similar effect by reusing your own persistent static List<Color32> tempCopy;, growing it as needed.