Lag the clients behind the server by x ms with the drawing of things and have the server do the movement of the ball and send the position every x ms to the clients. The clients should store a queue of the positions it's getting from the server which will be choppy if you just set those, but because you have a queue and you're lagging behind the server you can interpolate between the current position to the next position on the queue. This will give you nice smooth and accurate ball position controlled from the server.
You also want to store a rolling list on the server side of these positions that were sent so you have a small amount of history. You'll want this because the clients are lagging behind so when you get paddle movement commands at a point in time you then "rewind" the ball by looking at the history ball position list for the time you get the clients paddle movement minus the ping (the time it took you to get that movement). Then you can validate if a collision happened or not and the server ultimately controls the ball.
At the point of a collision because the ball is lagged behind on the client, on the server the ball may have went passed where the paddle is thought to be on the server. So when a collision from the rewind happens, you'll want to re-position the ball on the server at the collision point and run the physics from there which will reverse it's position. You might want to send that collision point to the users as well even if it's outside your interval of sending ball position because you'll want that specific point for them.
But this way the server controls all physics of the ball an the clients are all just lagged behind but they are none the wiser of that.
The side note of this is that how you tell if the ball actually did get passed a paddle. You'll just have to make that check happen farther away from the paddle based on how fast the ball can move and your update interval. In other words the server needs to consider the lag behind in ms the clients are doing when telling if a ball really got by a paddle.
Side note, this is how the Source engine works. When you shoot on your side that command is sent to the server and it knows how long it took to get there (ping). Let's say the server gets it at tick 500 and it took 50ms to get the command. It knows the clients are lagging behind by 150ms (this is a hardcoded value that the server decides on at the start) so it knows to look at (now - 150 - 50) in the history list of game states on the server. It'll find the closest snapshot and interpolate from there to get positions/rotations of everything in the game at that point which should have been what YOU saw at that point, and then runs the raycast on the shoot to tell if there was a hit or not. Same concept here really. The only difference is you having to snap the ball to the collision point if the server deems one took place.