I'm working on a hobby project (I'm a web/backend developer by day) and I want to create a 3D tile (terrain) engine. I'm using XNA, but I can use MonoGame, OpenGL, or straight DirectX, so the answer does not have to be XNA specific. I'm more looking for some high level advice on how to approach this problem.
I know about creating height maps and such, there are thousands of references out there on the net for that, this is a bit more specific.
I'm more concerned with the approach to get a 3D hexagon tile grid out of my terrain (since the terrain, and all 3D objects, are basically triangles).
The first approach I thought about is to basically draw the triangles on the screen in the following order (blue numbers) to give me the triangles for terrain (black triangles) and then make hexes out of the triangles (red hex).
This approach seems complicated to me since I'm basically having to draw four different types of triangles.
The next approach I thought of was to use the existing triangles like I did for a square grid and get my hexes from six triangles as follows
This seems like the easier approach to me since there are only two types of triangles (I would have to play with the heights and widths to get a "perfect" hexagon, but the idea is the same.
So I'm looking for:
1) Any suggestions on which approach I should take, and why.
2) How would I translate mouse position to a hexagon grid position (especially when moving the camera around), for example in the second image if the mouse pointer were the green circle, how would I determine to highlight that hexagon and then translating that into grid coordinates (assuming it is 0,0)?
3) Any references, articles, books, etc - to get me going in the right direction.
Note: I've done hex grid and mouse-grid coordinate conversion before in 2D. Looking for some pointers on how to do the same in 3D. The result I would like to achieve is something similar to this video.