Deciding which criterion to start with
I assume you don't want the player to be hidden behind walls. How about enemies? Items? Controls (including ones situated on back-facing walls)? I'll have to assume all, for the purposes of responding.
Whatever is directly between your ortho camera and any gameplay-critical entities, must be made transparent OR culled completely from the view.
For walls near to your player, simply make surface voxels transparent (never consider non-surface voxels for rendering). You need to determine the silhouette of the entire clump that would occlude, and render it only as a silhouette at a single alpha (say 10%). If you were to simply treat each voxel as being at a given alpha, their transparencies might combine to be so opaque so that you would hardly see the critical entities beyond them.
You furthermore mentioned the insides of buildings, which could equally be seen as other rooms (if eg. your game was in a dungeon environment). The best way to deal with this is to keep a list of bounding boxes describing other rooms, and use these for additional zone-based culling. This could be the same as, or separate from, your level chunks. I would recommend it be separate as chunks should ideally be based on a uniform grid in order to keep processing times uniform regardless of where in the world you are. Using non-uniform bounding boxes can mess with this.
Culling what is definitely occluded by opaque objects
Consider a standard isometric engine. Occlusion is often not explicitly handled, because the degree of occlusion is minimal enough in most cases that we can afford to ignore it. In your case, however, it is necessary to know what not to draw. This is a perfect application of the reverse painter's algorithm. Determine the lines of voxel "bases" closest to the lower edge of your screen. In each such "tile" along this line, there exists a column of voxels. Draw each voxel in this column, from the base point near the bottom of the screen, upwards. Proceed to the next row above it, and repeat, till you've drawn all rows in the scene. As for knowing what NOT to draw, this image represents the tile base positions on your screen (the bottoms of all voxels at sea level):
* * * * * * *
* * * * * *
* * * * * * *
* * * * * * <- Second last row
* * * * * * * <- Baseline row
Do you see how the second last row's voxel bases are offset from the baseline's? Every second row's tile centre lies exactly between the next and previous line's x-centres? Same thing, with graphics:
Because of this, the first (leftmost) tile column in the second-from-last line is only occluded if the first AND second tile columns in the baseline BOTH exceed the height of the column that lies behind/between them. If they are equal, then you won't see the sides of the voxel behind, but you will see it's top -- so it will still need to be drawn. If less than both, it is completely occluded.
This solution will not entirely eliminate overdraw (since you still need to draw a whole column even if only one of the two in front of it is occluding it), but it certainly will make a major difference. For complete elimination of overdraw at a per-pixel level, see below.
Dealing with overhangs / floating voxels shouldn't be too hard to figure out given the above.