I can't say for sure what kind of approach was used by WC3 developers, but it looks pretty much like Hierarchical Annotated A*. Unit radius defined in the WC3Editor was used as-is for 3d model scaling, but actual unit size for pathfinding was discrete, maybe something like unitSize = (int)(unitRadius / 10). It wasn't vector-based, that's for sure.
Let's say there are a lot of path nodes making high-resolution node grid. Simple unit like a ghoul has a size of 2, so in order to place it somewhere in a grid we need 4 free path nodes near each other. A death knight hero is slightly larger with a size of 3, taking 9 path nodes total. Now we place 2 ziggurats together leaving a 2 node-wide space in-between, and send a ghoul and a death knight on the other side. Ghoul will be able to pass between two ziggurats, while death knight will have to move around. How can it be determined?
In order to see whether a node can accommodate a unit of specific size, let's assign a special clearance value to each node defining a maximum allowed unit size. Basically, it means that several bounding checks were made for a node, and the largest possible bounds were remembered as node's clearance. So when we want to place a death knight on some node, it becomes as simple as comparing node's clearance with death knight’s size. Of course, things are going to become much more complex when there are several units hopping around competing for nodes, but that’s another story.
For further detail you may want to check out this article: