3
\$\begingroup\$

I have a script which is loading a photo from path into a texture (photoThumbT2D), then rotating it:

                byte[] byteArray = File.ReadAllBytes(path);
            
                UnityEngine.Object.Destroy(photoThumbT2D);
                photoThumbT2D = new Texture2D(0, 0);
                photoThumbT2D.LoadImage(byteArray);
                RotateImage(photoThumbT2D, -90);

Where the RotateImage function is adapted from here as:

public static void RotateImage(Texture2D originTexture, float angle) {
        int width = originTexture.width;
        int height = originTexture.height;
        float halfHeight = height * 0.5f;
        float halfWidth = width * 0.5f;

        Color32[] originPixels = originTexture.GetPixels32();
        Color32[] rotatedPixels = originTexture.GetPixels32();

        int oldX;
        int oldY;

        float phi = Mathf.Deg2Rad * angle;
        float cosPhi = Mathf.Cos(phi);
        float sinPhi = Mathf.Sin(phi);
    
        for (int newY = 0; newY < height; newY++) {
            for (int newX = 0; newX < width; newX++) {
                rotatedPixels[newY * width + newX] = new Color32(0, 0, 0, 0);
                int newXNormToCenter = newX - width / 2;
                int newYNormToCenter = newY - height / 2;
                oldX = (int)(cosPhi * newXNormToCenter + sinPhi * newYNormToCenter + halfWidth);
                oldY = (int)(-sinPhi * newXNormToCenter + cosPhi * newYNormToCenter + halfHeight);
                bool InsideImageBounds = (oldX > -1) && (oldX < width) && (oldY > -1) && (oldY < height);

                if (InsideImageBounds) {
                    rotatedPixels[newY * width + newX] = originPixels[oldY * width + oldX];
                }
            }
        }

        originTexture.Reinitialize(width, height);
        originTexture.SetPixels32(rotatedPixels);
        originTexture.Apply();

    }

However, this application of RotateImage is leading to what I believe is a heap fragmentation problem where the more times it runs the more memory usage piles up in the profiler. This is discussed here.

In my case, it manifests in Android only (no problem in Editor) as a steady increase in GC Used Memory (Tracked Memory, In Use) until the app crashes:

enter image description here

In the Memory Profiler I get a new chunk of Managed Heap created for every time it runs until everything crashes (here there about 13 identical blue chunks created):

enter image description here

Apparently the only solution is to use NativeArray<Color32> nativeArray = texture2D.GetRawTextureData<Color32>(); and work with that, but since the NativeArray is a direct representation of the Texture data, I'm not sure if it's possible mathematically to rotate it without making a new copy to work from. And if you make a new array copy of the actual color data, you will get the same heap problems as with GetPixels32 and SetPixels32.

Any thoughts or solutions?


Addendum

Unfortunately, DMGregory, I learned from your method but I am still having trouble with your solution.

MEMORY ACCUMULATION

I have tested, and even just the code:

        photoThumbT2D = Resources.Load<Texture2D>("bigphoto");
        photoThumbnailVE.style.backgroundImage = photoThumbT2D;


        buttonClickedEvent += delegate {
            Debug.Log("RUN CLICK");

            var texels = photoThumbT2D.GetRawTextureData<Color32>();
            var texelsCopy = System.Buffers.ArrayPool<Color32>.Shared.Rent(texels.Length);
            //Unity.Collections.NativeArray<Color32>.Copy(texels, texelsCopy, texels.Length);
            System.Buffers.ArrayPool<Color32>.Shared.Return(texelsCopy);
};

Continues to trigger memory accumulation in Unity for Android in the same manner as the OP code. Thus in even this manner the Rent and Return functions are leading to memory accumulation.

Can you think of any reason this would not be happening? I have Incremental GC collection turned on in Android. I can't make any sense of it. Might it be because it's being run as a delegate or the parent object running it was created by a coroutine (which should be long gone)?

I have tested further and this can be solved instead by using a static List to store the duplicate data in but I would love to know why any other method is causing this problem.

\$\endgroup\$
4
  • \$\begingroup\$ Your method appears to destroy the old texture and create a new one. Why not use the method Texture2D.Reinitialize instead? It allows you to do the same thing with the same texture. This also has the advantage that any references to the texture remain valid. \$\endgroup\$
    – Philipp
    Dec 2, 2022 at 9:09
  • \$\begingroup\$ Thanks Philipp, I didn't know about Reinitialize. I tried that and I will remember that for the future. Unfortunately it looks like the problem is something else - maybe the Android garbage collector is broken. I updated my post to reflect it. Any further thoughts are appreciated. \$\endgroup\$
    – mike
    Dec 3, 2022 at 2:31
  • 1
    \$\begingroup\$ Keep in mind it's often cheaper just to rotate the object or UV coordinates referencing the texture, rather than actually moving the pixels around in memory. So that may also be worth considering, if your application would allow for it. \$\endgroup\$
    – DMGregory
    Dec 3, 2022 at 19:21
  • \$\begingroup\$ The second question (working around texture format differences) should be a new post - it's not related to the original topic of avoiding memory allocation. You can solve it easily enough by just reading raw bytes, or a custom struct matching your texture's data layout. \$\endgroup\$
    – DMGregory
    Dec 3, 2022 at 22:17

1 Answer 1

3
\$\begingroup\$

Two tricks we can use to get the RotateImage method down to zero allocations (in steady state):

  • We'll use GetRawTextureData<T> to get a view into the texture's existing CPU-side memory buffer, to edit it directly without creating a new array to call SetPixels with.

  • We'll use System.Buffers.ArrayPool to borrow and return a reusable array to store the old data. This can get recycled for future calls, rather than being claimed as garbage and recreated anew.

public static void RotateImage(Texture2D tex, float angleDegrees)
{
    int width = tex.width;
    int height = tex.height;
    float halfHeight = height * 0.5f;
    float halfWidth = width * 0.5f;

    var texels = tex.GetRawTextureData<Color32>();        
    var copy = System.Buffers.ArrayPool<Color32>.Shared.Rent(texels.Length);
    Unity.Collections.NativeArray<Color32>.Copy(texels, copy, texels.Length);

    float phi = Mathf.Deg2Rad * angleDegrees;
    float cosPhi = Mathf.Cos(phi);
    float sinPhi = Mathf.Sin(phi);

    int address = 0;
    for (int newY = 0; newY < height; newY++)
    {
        for (int newX = 0; newX < width; newX++)
        {
            float cX = newX - halfWidth;
            float cY = newY - halfHeight;
            int oldX = Mathf.RoundToInt(cosPhi * cX + sinPhi * cY + halfWidth);
            int oldY = Mathf.RoundToInt(-sinPhi * cX + cosPhi * cY + halfHeight);
            bool InsideImageBounds = (oldX > -1) & (oldX < width)
                                   & (oldY > -1) & (oldY < height);
            
            texels[address++] = InsideImageBounds ? copy[oldY * width + oldX] : default;
        }
    }

    // No need to reinitialize or SetPixels - data is already in-place.
    tex.Apply(true);

    System.Buffers.ArrayPool<Color32>.Shared.Return(copy);
}

You'll still see an allocation the first time you call this, or when you call it with a larger image than you've used before. That's from the ArrayPool allocating a new array for the requested size. But repeat calls will be able to re-use the array from the pool and so you won't see continually increasing memory use or fragmentation.

If ArrayPool isn't working as desired on mobile (maybe it's emulated as just new to avoid reserving large blocks of RAM on modest phones), you can get a similar effect by reusing your own persistent static List<Color32> tempCopy;, growing it as needed.

\$\endgroup\$
2
  • \$\begingroup\$ Thanks DMGregory. That's fantastic. I didn't know about System.Buffers.ArrayPool. I will use that for other pooling as well then. One follow up question: If the first time I use this, it is for a 20 MB allocation, and then the second time I run it it is for a 50 MB allocation, is the new allocation to the heap done for 30 MB, or does it create a whole new 50 MB allocation? If it creates a whole new one, then best practice would likely be to call it with the maximum anticipated size on build so it won't waste space later. Thanks again. \$\endgroup\$
    – mike
    Dec 3, 2022 at 19:02
  • 1
    \$\begingroup\$ That seems like a question you can answer by testing. In my tests so far, calling Rent(texels.Length) gives me back an array larger than texels.Length (the API says the returned array will be at least this long), so under the hood ArrayPool is already rounding your size up with the aim of minimizing extra allocations when you later request a larger size. But it's likely not adding an extra 30 MB of padding "just in case". So if that's important to you, you can call Rent(expectedMaxSize) and immediately Return() it to hopefully re-use just that single allocation. \$\endgroup\$
    – DMGregory
    Dec 3, 2022 at 19:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .