Glest, especially MegaGlest and the Glest Advanced Engine (forums)
Its a classic RTS engine that is fairly straightforward to 'mod' and add 'factions' using 3D models and XML files to define the unit attributes. It can be used as a basis for 'total conversions' and the code-base is accessible enough to add specific features. It may be that you don't actually need to touch the code, however - its designed to be modded!
MegaGlest 3.4 has just reached beta - its an awesome engine that's approachable to modding.