*Edit: I finally figured out after some profiling that the problematic code was not actually the ones part of ECS, but this line over here:
objectManagerController.GenerateMetaballEdgePointVerteciesList(metaballEdgePointVertexCoordinates); //writes magnets, this is the culprit for the lagg!!!
in OnRenderImage() of MainCameraController:
public void OnRenderImage(RenderTexture src, RenderTexture dest) {
//Debug.Log("Metaball Shader, OnRenderImage for main camera");
//called after operationCamera's OnRenderImage is called game object count times?
//How the fuck was I able to make sure that src contains the full texture upon function call?
//Since all rendered colored slimes are moved back to default layer after each of them are separately
//rendered. Thus by the time MainCameraController.OnRenderImage is called all slimes are back in default layer
//Not the case where main camera is not seeing anything
//Resolution
Vector2Int resolution = new Vector2Int( preprocessCameraRender.width, preprocessCameraRender.height );
//Thread control (hoping this is accurate enough)
threadGroupX = Mathf.CeilToInt(preprocessCameraRender.width / (float)threadsPerGroup);
threadGroupY = Mathf.CeilToInt(preprocessCameraRender.height / (float)threadsPerGroup);
//Make a writable copy of src and dest
//RenderTexture srcWritable = new RenderTexture(src.width, src.height, 32, RenderTextureFormat.ARGB32, RenderTextureReadWrite.Default){ useMipMap = false };
RenderTexture srcWritable = new RenderTexture(preprocessCameraRender.width, preprocessCameraRender.height, 32, colorFormat, 1)
{
useMipMap = false,
autoGenerateMips = false,
enableRandomWrite = true // Set the UAV usage flag
};
srcWritable.Create();
//At this point both srcWritable and destWritable empty
RenderTexture destWritable = new RenderTexture(srcWritable);
destWritable.Create();
RenderTexture metaballEdgeTexture = new RenderTexture(srcWritable);
metaballEdgeTexture.Create();
//Copying src to srcWritable
RenderTexture.active = src;
Graphics.Blit(src, srcWritable);
//metaball edge texture
// RenderTexture metaballEdgeTexture = CreateTexture("MetaballEdgeTexture", resolution, colorFormat);
//Fill compute shader variables
metaballComputeShader.SetTexture(kernelHandleFirst, "Src", srcWritable);
metaballComputeShader.SetTexture(kernelHandleFirst, "Dest", destWritable);
//Inner Edge Texture(empty at this point)
metaballComputeShader.SetTexture(kernelHandleFirst, "MetaballEdgeTex", metaballEdgeTexture);
metaballComputeShader.SetFloat("MaxWidth", preprocessCameraRender.width);
metaballComputeShader.SetFloat("MaxHeight", preprocessCameraRender.height);
metaballComputeShader.SetFloat("GameObjectListCount", colorSlimes.Count);
metaballComputeShader.SetVector( "_Resolution", (Vector2) resolution );
//Texture Arrays
//First dispatch prep
//Read (Raw) Color Texture Array
metaballComputeShader.SetTexture(kernelHandleFirst, "ColorTextureArray", colorTextureArray);
//Read (Larger) Color Texture's Array
metaballComputeShader.SetTexture(kernelHandleFirst, "ColorTextureArrayBeta", colorTextureArrayBeta);
//SDF Texture Array
metaballComputeShader.SetTexture(kernelHandleFirst, "SDFTextureArray", sdfTextureArray);
//Delta X Texture Array
metaballComputeShader.SetTexture(kernelHandleFirst, "DeltaXTextureArray", deltaXTextureArray);
//Delta Y Texture Array
metaballComputeShader.SetTexture(kernelHandleFirst, "DeltaYTextureArray", deltaYTextureArray);
//Weight Texture Array(fill weight values in it)
metaballComputeShader.SetTexture(kernelHandleFirst, "WeightTextureArray", weightTextureArray);
//Blur Texture Array
// metaballComputeShader.SetTexture(kernelHandleFirst, "BlurTextureArray", blurTextureArray);
//First metaball compute shader dispatch
metaballComputeShader.Dispatch(kernelHandleFirst, threadGroupX, threadGroupY, 1);
//Debug.Log("First metaball effect complete!");
// Debug.Log("destWritable size: " + destWritable.width + " " + destWritable.height);
// Debug.Log("metaballEdgeTexture size: " + metaballEdgeTexture.width + " " + metaballEdgeTexture.height);
//Copying destWritable(which accounts for overlap blending) to srcWritable
// RenderTexture.active = dest;
// Graphics.Blit(dest, src);
// RenderTexture.active = null;
//Apply sobel filter to metaballEdgeTexture
RenderTexture sobelledMetaballEdgeTexture = Sobel(sobelComputeShader, metaballEdgeTexture, resolution, threadGroupX, threadGroupY);
//Debug.Log("sobelledMetaballEdgeTexture size: " + sobelledMetaballEdgeTexture.width + " " + sobelledMetaballEdgeTexture.height);
RenderTexture sobelledMetaballEdgeTextureFinal = new RenderTexture(metaballEdgeTexture);
sobelledMetaballEdgeTextureFinal.Create();
//Final dispatch prep
//Receive previous srcWritable(which is now simply a copy of destWritable) as Src
metaballComputeShader.SetTexture(kernelHandleFinal, "Src", srcWritable);
//First write same value as Src to Dest and then rewrite during dispatch
metaballComputeShader.SetTexture(kernelHandleFinal, "Dest", destWritable);
//InnerEdgeTex
metaballComputeShader.SetTexture(kernelHandleFinal, "MetaballEdgeTex", sobelledMetaballEdgeTexture);
//Final processed sobelled metaball edge tex
metaballComputeShader.SetTexture(kernelHandleFinal, "MetaballEdgeTexFinal", sobelledMetaballEdgeTextureFinal);
//Buffer.SetCounterValue(0); only used for Append/Consume buffers
//Color Texture Array
metaballComputeShader.SetTexture(kernelHandleFinal, "ColorTextureArray", colorTextureArray);
//SDF Texture Array
metaballComputeShader.SetTexture(kernelHandleFinal, "SDFTextureArray", sdfTextureArray);
//Delta X Texture Array
metaballComputeShader.SetTexture(kernelHandleFinal, "DeltaXTextureArray", deltaXTextureArray);
//Delta Y Texture Array
metaballComputeShader.SetTexture(kernelHandleFinal, "DeltaYTextureArray", deltaYTextureArray);
//Weight Texture Array(now read from it)
metaballComputeShader.SetTexture(kernelHandleFinal, "WeightTextureArray", weightTextureArray);
ComputeBuffer magnetVertexCoordinateBuffer = new ComputeBuffer(src.width * src.height, sizeof(uint) * 2, ComputeBufferType.Append);
// Create a buffer to hold the counter.
ComputeBuffer countBuffer = new ComputeBuffer(1, sizeof(int), ComputeBufferType.Raw);
int[] countArray = new int[1]; //to read back counter data later
//Set buffer to hold values and buffer to hold counter
metaballComputeShader.SetBuffer(kernelHandleFinal, "MagnetVertexCoordinates", magnetVertexCoordinateBuffer);
magnetVertexCoordinateBuffer.SetCounterValue(0);
//Final metaball compute shader dispatch
metaballComputeShader.Dispatch(kernelHandleFinal, threadGroupX, threadGroupY, 1);
//Debug.Log("sobelledEdgeTextureFinal size: " + sobelledMetaballEdgeTextureFinal.width + " " + sobelledMetaballEdgeTextureFinal.height);
// Copy the append buffer count to the count buffer.
ComputeBuffer.CopyCount(magnetVertexCoordinateBuffer, countBuffer, 0);
// Read the count to a c# int[]
countBuffer.GetData(countArray);
int numEdgePoints = countArray[0];
// Now you can read back just that many edge points.
uint2[] edgePoints = new uint2[numEdgePoints];
magnetVertexCoordinateBuffer.GetData(edgePoints, 0, 0, numEdgePoints);
Debug.Log("edgePoints (post EdgePointsFilled) Count: " + edgePoints.Length);
List<Vector3> metaballEdgePointVertexCoordinates = edgePoints.Select(ep => new Vector3(ep.x, ep.y, 0)).ToList();
//Call object manager controller's GenerateMetaballEdgePointVerteciesList(magnetic)
objectManagerController.GenerateMetaballEdgePointVerteciesList(metaballEdgePointVertexCoordinates); //writes magnets, this is the culprit for the lagg!!!
//Write metaball effect to final dest (non-metaballed, but hoping the magnets will work)
Graphics.SetRenderTarget(dest);
Graphics.Blit(destWritable, dest);
RenderTexture.active = null;
srcWritable.Release();
destWritable.Release();
metaballEdgeTexture.Release();
sobelledMetaballEdgeTexture.Release();
sobelledMetaballEdgeTextureFinal.Release();
countBuffer.Release();
magnetVertexCoordinateBuffer.Release();
//Debug.Log("Metaball effect complete!");
}
This is the code for GenerateMetaballEdgePointVerticesList in objectManagerController:
public void GenerateMetaballEdgePointVerteciesList(List<Vector3> metaballEdgePixels) // Generates and initialises metaball edge vertices passed from MainCameraController
{
//if passed list is empty, reset all vertices (it means game objects are not close enough to metaball)
if (metaballEdgePixels.Count == 0) {
//Reset vertices for each game object
foreach (GameObject colorSlimesDisplay in colorSlimesDisplayGlobal) {
colorSlimesDisplay.GetComponent<SpriteMeshModifier>().Reset();
}
//Reset metaballEdgePointVertices
foreach (var item in GetComponentsInChildren<MetaballEdgePointVertex>())
{
DestroyImmediate(item.gameObject);
}
metaballEdgePointVertices.Clear();
//Reset metaballEdgePointVertex GOs
metaballEdgePointVertexGOs.Clear();
}
else {
//Create magnetic metaball edge points
foreach (var item in GetComponentsInChildren<MetaballEdgePointVertex>())
{
DestroyImmediate(item.gameObject);
}
metaballEdgePointVertices.Clear();
//Reset metaballEdgePointVertex GOs
metaballEdgePointVertexGOs.Clear();
int cntr = 0;
// float circleColliderRadius = 10.0f;
Debug.Log("Logging metaball vertex locations");
//typeof(metaballEdgePointVertex) == typeof(Vector3)
foreach (Vector3 metaballEdgePointVertex in metaballEdgePixels)
{
//Convert passed compute shader RenderTexture.id.xy to world coord
Vector2 normalizedCoords = new Vector2(
metaballEdgePointVertex.x / preprocessCameraRenderTexture.width,
metaballEdgePointVertex.y / preprocessCameraRenderTexture.height
);
Vector2 screenCoords = new Vector2(
normalizedCoords.x * Screen.width,
normalizedCoords.y * Screen.height
);
Vector3 metaballEdgePointVertexWorldCoords = _displayCameraGlobal.ScreenToWorldPoint(new Vector3(screenCoords.x, screenCoords.y, 0));
metaballEdgePointVertexWorldCoords.z = 0;
Debug.Log(metaballEdgePointVertexWorldCoords);
var go = Instantiate(MetaballEdgePointVertex, transform); //transform of the game object that will become parent of GOs == ObjectManager.transform
go.name = go.name + " " + cntr;
// Add CircleCollider2D
// CircleCollider2D circleCollider = go.AddComponent<CircleCollider2D>();
// // Configure CircleCollider2D properties
// circleCollider.radius = circleColliderRadius;
// circleCollider.isTrigger = true;
metaballEdgePointVertexGOs.Add(go);
var ver = go.GetComponent<MetaballEdgePointVertex>();
ver.SetInitialPos(metaballEdgePointVertexWorldCoords, cntr, go); //this is reference to SpriteMeshModifier object that owns vertices
metaballEdgePointVertices.Add(ver);
cntr++;
}
}
}
Here is a visual of that shows the magnets that each vertex is trying to go to:
Here are 2 techniques I can think of:
Create a public RenderTexture in the Unity editor using create -> RenderTexture simply Blit in MainCameraController's OnRenderImage when the magnet texture is created. The logic for extracting opaque pixels from the public RenderTexture to a List and then generation of metaballEdgePointVerticesList in objectManagerController.cs(since this is the de-facto MonoBehavior global game object manager script)
In MainCameraController's OnRenderImage, update a newly created public List metaballEdgePointPositionsList(objectManagerController.cs) instead of instantiating a new List metaballEdgePointPositionsList in OnRenderImage every time.
metaballEdgePointPositionsList = edgePoints.Select(ep => new Vector3(ep.x, ep.y, 0)).ToList();
Then the ECS SystemBase class will fetch from its reference of objectManagerController its metaballEdgePointPositionsList and generate the actual global List metaballEdgePointVertexList in the first compute job using the elements of metaballEdgePointPositionsList.
My question is, which one is the better method of optimization? Also, in order to optimize the generation of vertices, what is the general method of updating MonoBehavior global list in a SystemBase object job? I am having trouble figuring out how exactly I should translate GenerateMetaballEdgePointVerteciesList to a ComputeJob, since the job GenerateMetaballEdgePointVerteciesList is to Add Vertex objects to a MonoBehavior global objectManager, and ComputeJobs require to operate on elements of a NativeArray and cannot modify MonoBehavior objects directly.
May I have some more sophisticated advice in this area in general?
*Here is a demonstration of the lag that arises from updating the mesh vertice: https://drive.google.com/file/d/1wUGs4MV1KoIgkgBqv9zD2g87zs9AlRzN/view?usp=sharing
