I'm not an artist and don't claim to be one; this answer is mainly based on hearsay and "articles I read on the internet."
Rigging and skinning works through math. There's no magic or physics involved, and an improperly-designed skin is going to cause you just as much trouble as an improperly-designed rig or an improper weighting. Based on your pictures I think your problem is that you mesh geometry isn't designed to handle deformation at all; you model the polygons around the joints exactly the same way as you model the polygons everywhere else, and at a very high resolution.
The problem is that "everywhere else" doesn't really deform under animation, but the joints do. If the topology of your mesh doesn't account for that, you get creases, jagged edges, and assorted unpleasant weirdness:
To be able to sculpt good-looking muscle activity and corrective shapes, it’s important that your topology supports the correct creasing in required areas. Try not to use an extreme polygon density for the base model for the first skinning pass. Skinning a low-resolution character is much easier and takes a fraction of the time that is required to properly skin a subdivided character with more than 20,000 vertices.
Note, for example, the way this hand is built with geometry designed to support the deformation zones:
You probably need to modify or rebuild your hand model so that it contains that kind of supportive geometry. Remember that skinning only moves vertices, so you have to think about where the vertices will be before, after and during the application of a deformation. If they start to deform so they're inside the skin, you'll get weird visual discontinuities in the render. If they deform outside the skin, weird jagged edges, and so on.