What is the appropriate way to implement wobble/balance mechanics in Unity 3D?

We are trying to implement a minigame where the player character sits on a bouncing ball, and needs to a) hit forward to move forward and b) hit left/right to adjust against the the ball's natural wobble (in other words, if the ball leans left, you must hit right).

We have tried implementing this by setting the bounciness and weight on the ball so that it bounces naturally when the player is not on it, then fixing the player model to the ball (in other words, treating them as a single unit), then having the CharacterController control the unit while it continues bouncing.

However, no matter how we tune the various factors (bounciness, weight, input multiplier, gravity), the ball remains quite sensitive and eventually spirals out of control. We would particularly like to allow the player some opportunity to regain control.

I have played a number of games that require some kind of reflex-based wobble/balance mechanic. Is there a general approach here that we are missing?

This question is for a small academic project, so we're wondering if there's a specific component we need, or part of the Physics engine we've missed, or technique we should look up - but please don't post code. There are no project requirements here except using Unity 3D (e.g. the assignment is not "implement a bouncing ball minigame").

-
Not sure if this will work as a plain physics system. What you might need to do is combine player and ball into an inverted pendulum system which, if necessary, breaks under certain conditions. If there is no time where player and ball will be separate, you might as well simulate them together. en.wikipedia.org/wiki/Inverted_pendulum – jzx Jul 29 '14 at 19:59
Maybe this will help illustrate my point: in the Tony Hawk games, a manual isn't simulated as player+board+ground. It's just player+ground, with the player in a special state that tracks "balance," possibly as an imaginary tangent vector or something. – jzx Jul 29 '14 at 20:03
@jzx we are treating them as a single object (I've clarified the question to make that clearer) - I think that link might be exactly what we're looking for! please post the inverted pendulum + imaginary balance approach as an answer and if it works for us I'll accept it! thanks for your insights – d3vid Jul 30 '14 at 7:08