It depends on multiple factors including...
- How much geometry you are raycasting against (more = slower)
- The resolution you are rendering your color picking at (higher = slower)
- How you've implemented either solution (CPU raycasting? vs GPU color picking? or something else?).
- Resource constraints -> if your application is already CPU bound, and has plenty of GPU power to spare, doing a super efficient CPU ray cast is still going to be slower than doing an inefficient operation on the GPU.
There is a way to get an answer for your particular situation though - profile it and find out!
It's also important to note, either solution is a reasonable solution in many cases, so going one way or the other is likely not going to be a huge deal. If you aren't having performance problems, don't spend too much thought trying to choose between reasonable answers IMO!