In GPGPU, the preferred workgroup size usually is more dependent on the resources used by your compute shader, kernel, etc. and less on the GPU specs. A compute shader will have a maximum number of threads that could run in a multiprocessor based on the resources it uses (since the resources are shared among threads). This concept is one of the cores of compute shader optimization and is the so called Occupancy. In summary, the Occupancy is the ratio of the number of threads that can run in parallel in your compute shader and the maximum number that the GPU supports.
The differences in GPUs related with GPGPU are mainly the number of available multiprocessors and bandwidth. The scheduler of the GPGPU framework (CUDA, DirectCompute, etc.) has the job to assign the workgroups to multiprocessors in a way that compute shaders not depend too much of the GPU specs to scale well (that's the main reason that a boost in the GPU automatically boosts the code performance).
The role of the programmer is to write compute shaders whose workgroups use minimal resource. This is achieved by:
- Minimizing register usage;
- Minimizing shared memory usage;
- Minimizing conditional branches.
For NVidia GPUs you can use the CUDA Occupancy Calculator to check the occupancy based on the number of registers and shared memory used. The compiler outputs this data. Using this tool you can check any workgroup size and see which one gives the best occupancy. You can also use this in DirectCompute since in NVidia cards it is just a "wrapper" for CUDA functionality. Of course there is a lot of experimentation before getting the optimal number.
See this presentation for more details on GPGPU optimizations.