I found an answer on this site relating to this question already, but it doesn't seem applicable in the context of my project.

Basically I'd like to create a method which fits this signature:

float3x3 AxisBillboard(float3 source, float3 target, float3 axis)

That is to say, when given a source point (i.e an object's position in world space), a target (camera position in world space), and an axis (the object's up vector, which is not necessarily the global y axis), it produces a 3x3 rotation matrix by which I can multiply the vertices of my point so that it's properly rotated.

I've found many solutions and tutorials online which work nicely assuming I only want to rotate around the y axis.

For example, here's a solution which billboards around the global y axis:

float3 dir = normalize(target - source);
    
float angleY = atan2(dir.x, dir.z);
c = cos(angleY);
s = sin(angleY);
    
float3x3 rotYMatrix;
rotYMatrix[0].xyz = float3(c, 0, s);
rotYMatrix[1].xyz = float3(0, 1, 0);
rotYMatrix[2].xyz = float3(-s, 0, c);

For context, I'm working on a grass shader, and each individual blade of grass should be billboarded to face the camera while remaining aligned with the normal of the terrain.

I found an answer on this site relating to this question already, but it doesn't seem applicable in the context of my project.

Basically I'd like to create a method which fits this signature:

float3x3 AxisBillboard(float3 source, float3 target, float3 axis)

That is to say, when given a source point (i.e an object's position in world space), a target (camera position in world space), and an axis (the object's up vector, which is not necessarily the global y axis), it produces a 3x3 rotation matrix by which I can multiply the vertices of my point so that it's properly rotated.

I've found many solutions and tutorials online which work nicely assuming I only want to rotate around the y axis.

For example, here's a solution which billboards around the global y axis:

float3 dir = normalize(target - source);
    
float angleY = atan2(dir.x, dir.z);
c = cos(angleY);
s = sin(angleY);
    
float3x3 rotYMatrix;
rotYMatrix[0].xyz = float3(c, 0, s);
rotYMatrix[1].xyz = float3(0, 1, 0);
rotYMatrix[2].xyz = float3(-s, 0, c);

For context, I'm working on a grass shader, and each individual blade of grass should be billboarded to face the camera while remaining aligned with the normal of the terrain.

edit:

Thanks to @DMGregory I'm using his matrix method in the following way:

float3 viewDir = normalize(vertexPosition); // camera is at (0, 0, 0)
float3x3 billboardMatrix = transpose(AxisBillboard(worldUp, viewDir));
float3 worldPosition = vertexPosition + mul(billboardMatrix, localPosition);

This works correctly when grass is rotated to align with world space. In other rotations, grass is still billboarding 'correctly' around its local y axis but also has another rotation applied. Any ideas?