What are the pros / cons of the three?
closed as not constructive by Nicol Bolas, Maik Semder, Josh Petrie♦, Sean Middleditch, Trevor Powell Jan 5 '13 at 17:39
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. If this question can be reworded to fit the rules in the help center, please edit the question.
coderanger is right about HLSL targeting DirectX, GLSL targeting OpenGL and CG being available with both interfaces.
However there are other things to consider (learned on the OGRE forum) :
So, if you're not using the lastest shader features, CG seem a good choice. GLSL seem a beter one if you're going full OpenGL. HLSL if you're going exclusively on Microsoft platforms.
Now first developping in HLSL for windows to use DirectX and then convert to GLSL for linux and mac could be the better solution to be sure of performance and have the larger set of shader features available. It might however be a lot of work (didn't do it myself so I cannot tell). OGRE graphics engine (and other engines) allow to use any API (DirectX or OpenGL or others) so it helps, but there is still shader code to convert if you go this way.
That's all the information I gathered while choosing my shader language (I've not made my decision yet).
Update: Valve did a conversion of one of their game to OpenGL and found no way to make the DirectX version faster than the OGL one. So keep in mind that the state of driver implementation, API quality, etc., all that change too much each year for you to totally rely on raw performance as an argument to choose one or the other. With this in mind, choose OpenGL/GLSL to simplify your life when working (or having plans or hopes to work) with other platforms than Windows, use DirectX/HLSL if you really want to use only Microsoft platforms and focus and maybe have some good API faster than OpenGL (this is reversing now, so don't count on it); use CG if you want to provide both possibilities to the user, but if you have the work force (and tools) to do it, using both GLSL and HLSL might be a viable solution too.
Update (2012): It is important to note that CG has been discontinued, and is no longer supported or actively worked on by Nvidia. Nvidia recommends that all users switch to a combination of GLSL and HLSL, or a newer library such as nvFX (on github). This is because it was too difficult to maintain feature-compatibility between GLSL and HLSL.
I can only talk about CG vs HLSL because those are the 2 I have used so far.
Cg is not the same as HLSL.
In Cg, NVIDIA did an excellent job in creating a very clean shader syntax. Its very similar to HLSL.
But, tie-together with D3D9/D3D11 (init code, shader compilation code) is much cleaner on HLSL than Cg. -1 Cg. Cg has a nasty bit of start up code that you don't even need to have for HLSL over D3D.
In Cg, you have to "cgGetNamedParameter" for every
In HLSL, this ends up being much cleaner - just one line, and you don't have to maintain that
In the above,
Before I started using HLSL, I used to look at this code and actually get jealous.
Although NVIDIA did their best to make a common interface for Cg between OpenGL/D3D, practically speaking its not that way, and you have
Further I have recently had a bad experience with Cg/ATI cards, which I'm pretty sure isn't my bad. (Someone else try it out?). I think it may be true that NVIDIA doesn't completely test ATI cards, as Klaim claims. Or that ATI doesn't test on Cg. One way or the other, there's a mismatch there and some sort of conflict of interests. -3 Cg.
All in all I preferred Cg. Its shader code syntax and naming is clean, sweet, and tidy. Its too bad its got these other problems.
My very basic understanding is that HLSL is only for DirectX and GLSL is only for OpenGL. Cg is basically the same language as HLSL, but can be used with either DirectX or OpenGL (though via different runtime code).
Another crucial difference between HLSL and GLSL (I don't know CG so I can't speak for it) is that with HLSL Microsoft provide the shader compiler as part of the D3D runtime whereas with GLSL your hardware vendor provides it as part of their driver.
This has advantages and disadvantages on both sides.
With the GLSL method the vendor can tune the compiler to the capabilities of their hardware. They know their own hardware best, they know what to do and what not to do. On the other hand the disadvantage is that - in a world where there are multiple hardware vendors - you have a situationwhere there can be inconsistencies between shader compilers, and the vendor also has complete free reign to screw up.
With the HLSL method Microsoft control the compiler. Everyone is on a consistent tech base, and if a shader compiles successfully in one place then it can be reasonably supposed to compile everywhere. The same shader will produce the same compiled output irrespective of vendor (all other things being equal of course). On the other hand the HLSL compiler has to be a "works consistently on everything" thing, so it's not able to pull any special vendor-specific tricks to get the last few drops of juice out of the tank.
If this comes across as though I have a preference for the HLSL view of the world it's because I do. I've been bitten badly before by wildly inconsistent behaviour on different platforms, and in one case even ended up having to roll a load of GLSL back to ARB ASM just to get a baseline that worked. NVIDIA's GLSL compiler can be viewed as particularly notorious here - it will even accept HLSL syntax and keywords, meaning that if you're not careful you can end up producing shaders that will only work on NVIDIA and nothing else. It is a jungle out there.
You might want to look into the RTSS (Run Time Shader System) that comes with Ogre as well. It is fairly new, but you basically write shaders in code rather than external files. I have not implemented it yet, but definitely plan to use this when the time comes.
Here is a huge series of tutorials on the Ogre wiki as well for writing shaders. http://www.ogre3d.org/tikiwiki/JaJDoo+Shader+Guide&structure=Sandbox+Structure
As for your original question, like Praetor said, its not really a matter of pros/cons, it is a matter of what rendering system you want are going to use. Since using DX/OpenGL with Ogre is just a matter of loading the plugin, you will most likely want to use .cg format.
I have been using both, I started with glsl and moved to hlsl only because the project demands it. Given the choice, it's glsl all the way. glsl feels very slick and hlsl feels like it came out off an engineer's hobby bench.