Definitely rendering at 2000 fps? Nah, I guess you're measuring elapsed frame time like usual, which means you're measuring time it took for the driver to generate the GPU commands and not the time it took the GPU to actually render the frame. After a while CPU waits for the GPU to catch up and it shows as spike in frame time.
As for possible reasons... Well, it could be everything starting from bad hardware or drivers and ending with bad code. Try to do the simplest graphics call possible:
// This should use QueryPerformanceCounter/QueryPerformanceFrequency
And then see if it still spikes. If it doesn't, add a code until it spikes again, and if it does, then it's a technical issue. For additional confirmation enable VSync and see if spiking rate changes. Launch a HWMonitor and see if temperatures and voltages are OK, check System Event Log, use GPUView with xperf, maybe try to somehow debug it with gDEBugger, change drivers, change hardware... Possibilities are endless :)
EDIT: Maik Semder actually gave a better advice :) Your problem may be completely unrelated to the video at all, so check what you're doing every frame, look at the GC, wrap different OpenGL calls with QueryPerformanceCounter.