The first simple trick I would recommend is making the character the rigidbody instead of the elevator. Using Unity's collisions you only need one member of the collision to be an actual rigidbody.
The other thing I notice in my experience with Unity and rigidbodies is that they can get quite shaky when pushing each other forcefully. From what it sounds like you're doing, the elevator is just colliding with the controller and shoving it up which is not the most graceful thing. If you have not already I would try having the elevator exert a force onto the controller so that their velocity increases upwards to move them rather than just shoving their position straight up which the base collision may be doing.
Check out this on the unity forums/answers, you may find some extra help there.