Determine End-to-End Latency on Laptop or PC

Question

I'm working on a project to determine the end-to-end latency of a cloud gaming system (i.e., XCloud) on a MacBook.

I believe I have half of the process completed so far:

1. A Javascript web app that logs every single time a user presses a button on their gamepad using the Mozilla GamePad API. The timestamp is calculated as milliseconds in Unix epoch time from the client's end.
2. The server-side of the web app which places each button input and its timestamp into a database so that they can be used at a later time for latency analysis.

Using this, I am able to accurately determine each time I press a button on my gamepad along with the button itself. Now, I would like to figure out the exact time the corresponding frame(s) are displayed onto my screen.

As far as I understand, it seems like the generally accepted approach to determine on-screen changes from gamepad inputs is to set up a high-speed camera and capture the actual frames changing on the display. However, I'd like to use a method that can be automated more easily and takes advantage of the actual hardware on my laptop.

Is there a different approach to this? One that might take advantage of my frame buffer or can determine pixel change on my screen? To anyone who is knowledgeable of this or has done something similar before, can you point me in a better direction? I've done quite a bit of research to solve this problem, but I can't find anything online that somewhat fits my needs.

Thank you to anyone who is reading this!

Answer 1

Just comparing time stamps on different devices is not enough to measure network latency, because device clocks are not synchronized within the precision that you care about. These are not precisely-calibrated atomic clocks, they're simple oscillators like the one in your microwave that gradually goes out of sync with the one on your stove or coffee maker. The reason network-connected devices track time better is that they're routinely nudged toward agreement with time servers using Network Time Protocol. As the article states, "NTP can usually maintain time to within tens of milliseconds over the public Internet" - tens of milliseconds is several frames of latency over- or under-estimated if you use this method.

To eliminate synchronization error, you'd want to measure a full round-trip from device A, replied to by device B, and received and processed on device A again. Then you can compare two timestamps measured against A's clock, so there's no synchronization error to deal with. You can divide the total round-trip time by two to estimate one-way latency.

As for the other sources of latency, the reason that high-speed cameras are the preferred option is that they give us a ground truth measurement of the complete end-to-end process, from the mechanical actuation of the button or stick to the emission of visible photons from the screen.

Any system that reads its start and end points from somewhere else in the chain potentially misses some sources of latency:

• Starting your clock when your game loop detects input? What if that input had been buffered in a queue in the device driver for some time before that moment?

• Using screen captures of the system frame buffer? What if this is queued in a longer swap chain, and those bits aren't actually being sent to the display yet? Or what about the delays introduced by the display device's processing and hardware response rates?

So, while you can estimate latency with other methods, it won't give you a true, accurate value that includes all possible sources of latency.

Answer 2

One way to do this without using a high-speed camera is to use a screen capture software that can capture the screen at a high frame rate and record the timestamp for each frame.

To measure the latency between button presses and on-screen changes, you can then use the timestamps from your gamepad input logs and the screen capture software to calculate the time difference between the button press and the corresponding on-screen change.

One example of a screen capture software that can capture the screen at a high frame rate and record timestamps for each frame is OBS Studio. This software allows you to set the frame rate at which the screen is captured and provides a timestamp for each frame in the captured video.

To use OBS Studio to measure the latency between button presses and on-screen changes, you can follow these steps:

• Install OBS Studio on your MacBook and set it up to capture the screen at a high frame rate (e.g., 60 frames per second).

• Start the screen capture in OBS Studio and open the cloud gaming application on your MacBook.

• Use your gamepad to interact with the cloud gaming application and log the timestamps for each button press using the Mozilla GamePad API.

• When you have collected a sufficient number of button press timestamps and screen capture frames, use the timestamps from the gamepad input logs and the screen capture video to calculate the time difference between each button press and the corresponding on-screen change.