You still need the grid in order to optimize collision checks, you'll be doing wrong by removing it.

You could try to do it with directx directly, is not that difficult to render squares on the screen and the performance will increase a lot if you do things right. Anyway I'd recommend to have mipmapped textures in different files so you load only what you need. A smarter resource manager will keep unused resources open just in case until it detects it's running out of memory and then release those in order to load what's needed. You don't need to have everything in a single texture unless you're doing something wrong.

If you have everything in a single texture it means you are not managing your texture resources at all. You won't be able to work properly that way; Games which use high resolution textures have proper resource managers to deal with thatand I know because I worked on commercial 3d games which had to fit in less than 350mb due to limitations of the platform. You can't expect anything to work fast by having payloads of 256 mb, but you can try if you like :).

Yeah, you don't have to load everything, you have to load only what you use. Whenever you want to change a pixel on a texture loaded in VRAM, the system has to move the ENTIRE TEXTURE to RAM, just for you to modify a single pixel, the move it back to VRAM. If you have everything in a single texture, this involves moving 256 MB to RAM then back to VRAM again, which locks the whole computer. Having it separated in different files and textures is the correct way of doing it.

But the shape of the sphere wont easily lend to any space partitioning technique, and you want to use spheres. I think that what you are looking for are occlusion algorithms? Those are better for voxels I think.

I think youre confusing the visible shape of the tile with the space partitioning technique. They are two different things.

Thank you, I just solved it but your answer still adds! The hmd stuff is something I didn't think about before, definitively interesting. I'll be working with these kind of transforms for a while so everything helps!

@NathanReed I definitively solved it because you said I was on the right path, then I tried again with renewed hopes until I got it working. Thank you.

@NathanReed Hi Nathan, thanks for your help. I'm using the scissor test for clipping as I already had exposed by the abstraction interfaces, so I thought I would be missing only a transformation to do the job. I think I'm close to get it: now I added a division to scale the normalized target space to viewport space, which scaled the image the way I want (as I get the same size as by using d3d viewports), but when I try to add a translation I get only a distorted FoV. It seems I'm missing some operation but I'm not sure what it is.