Turns out it was an issue with the way I was setting up the compute buffer. 

Specifically, it was `ComputeBufferMode.Dynamic` that was incorrect.

https://forum.unity.com/threads/clarification-on-computebuffermode-immutable.1150250/

Switching to `ComputeBufferMode.Immutable` [which isn't actually _immutable_] solved the problem.