Full-screen window hangs when debug break or crash (Windows/GLFW/Cocos2D)

Question

I run into an issue if my game application becomes unresponsive when running in fullscreen on Windows. This mostly applies when I want to use breakpoints in Visual Studio, however if the application crashes, such as from a memory error, the window doesn't close.

I don't think this is strictly a WGL/GLFW/Cocos2D problem, since I have seen this problem in other Windows games, including ones that use Direct3D.

I searched around, but I can't find a definitive answer as to what causes this problem or how to fix it. From what I have read, this issue is only supposed to be caused by using HWND_TOPMOST (or maybe HWND_TOP); however, even changing all uses of that in GLFW to HWND_NOTOPMOST doesn't seem to fix the problem.

I imagine that registering a custom signal handler using std::signal might work, but I am not sure which parts of the Windows API, if any, I should call in an exit handler, that would solve this problem.

Answer 1

You did not mention if you are running Visual Studio and your game application on the same computer or not. If you have not already done so you should utilize the remote debugger functionality of Visual Studio when developing an application that changes the screen resolution.

It allows you to run Visual Studio on a development computer and then deploy and run your game executable on a remote computer. In Visual Studio 2019 (current for 2021), the Visual Studio remote debugging components are available for download on Microsoft.com: download link

This will save you a lot of head aches and allow you to debug more effectively.

Answer 2

The "problem" is that when running the application within Visual Studio, even in Release mode, it automatically attaches its debugger. This appears to be strictly related to Visual Studio trying unsuccessfully to become the active window underneath another unresponsive window. And of course the "deadlock" occurs because it is the MSVC debugger itself keeping the process open/suspended, and preventing the window from closing.