You should probably look into using an existing 3D physics library. Implementing even basic rolling and bouncing is not an exercise for the faint of heart or those lacking in physics simulation experience. Bullet (bulletphysics.org) supports Android.
The basic concepts of physics simulation in games are pretty intricate, and I couldn't possibly do them justice here. If you're being required to do everything from scartch, you're going to have the following problems to solve (of which Bullet will help with immensely):
- Collision detection with the floor and walls. This is relatively simple math for planes and spheres, but may require more complex work to collide a complex triangle mesh like a basket with a sphere.
- Collision response with the walls, floor, and basket rim. This is going to at the minimum require calculating points of contact and normals, and then calculating friction and elasticity (bounciness) to alter the ball's trajectory.
- Rotational forces for the ball to keep it animates nicely. Simply tracking the linear velocity of the ball is not enough. When the ball bounces, you need to know the center of mass of the ball (easy enough) as well as the point of contact and a few other bits of info to calculate the change in rotational velocity.
- Collision detection with the inside of the basket to detect if the goal has been scored or not.
I'd recommend the books Real-Time Collision Detection to help with parts 1, 2, and 4. For parts 3 and 2, one of the best sources of learning I know of would be to read the manuals, GDC slides, and articles by Erin Catto, who is the primary author of Box2D. Jumping straight into 3D physics without any experience writing a 2D physics engine first, however, is likely a path to madness. Trust me on that one.
Actually rendering everything is a fair bit easier. You can draw static geometry likes the walls and floor easily enough. The ball is just a circular mesh that you need to translate to the proper position and rotate to the proper angles. The basket is likely a simple mesh of triangles to render.
Definitely look into Bullet for handling the physics parts of that if you can. Writing those on your own is non-trivial. Physics simulations are amongst the hardest parts of games to write (even in simple games). That's why physics middleware libraries like Havok or PhysX or Bullet are so widely used both in small indie games and in big commercial game engines that have large teams of badass developers.