Server Side Speed Hack Detection Techniques

I have a client side speed hack detection that initiates a timer on both the client and server on connection. When the difference becomes too much the client gets detected as using speed hack.

But I have one problem, they found a way to run a speed hack program and close it but keeping the speed hack activated on the game client (odd) which made the counter on the client run normally even with speed hack enabled.

So I added detection to the server by using the movement packets and checking the time difference between 2 different movement packets come in. The design was okay, it did detect the speed hack by checking that the difference is not less than 50ms but when someone was lagging (queued movement packets) it also detected it as speed hack.

So I improved this by considering a factor of how fast it would process each packet that arrived in bulk which means microseconds. Now I adjusted the detection to less than 50ms but greater than 1ms. This was better it didn't detect lagging too much but it still did.

Now my problem is how can I implement this in a way that I can still check how fast the packets are coming in but not detect the people that are experiencing lag falsely. Or should I just keep adjusting the parameters for detection until I get to the sweet spot?