This is not a complete answer, but hopefully it will help.
Is your map/tile set static or dynamic? If it's static, I would strongly encourage you to do offline processing and precompute as much as you can that way. You can do it in any way you want (raycasting or something smarter), but you don't really care about making very fast because it will all happen offline while you "bake" your levels. You can either precompute exactly which tiles are visible from each spot, or create some kind of larger regions and all the tiles potentially visible from each region.
Then, at runtime, the query for the visible tiles should be very straightforward and very fast.
Of course, if you're having dynamically created levels, then this doesn't apply at all :-)