I'm new to this, but it would seem to me that if things aren't working, you should try something else. So maybe try a fewer colliders?

EDIT:  You also might try using a fixed frame rate, like described here: http://obviam.net/index.php/the-android-game-loop/  Maybe you can fix the framerate so it stays high

EDIT2: Maybe you have a memory leak? This article talks about that possibility on iOS: http://www.cocos2d-iphone.org/forum/topic/6641