We had a openworld shooter (x360,PS3,PC) that used a quick smoketest on the build server - it loaded the game, stepped through the front end, ran [the avatar] forwards, dumped a screenshot, and exited. If cctray detected the clean exit the build was a success.
We ran it for about the last year of the project, and with a team size of ~100 devs.
It was effective at catching showstopping bugs but it was easy to create a build that passed the smoketest but failed most "real" levels, or didn't work in multiplayer, or nobbled the AI, so it wasn't perfect. It was definitely worth doing.
I've heard since I left they've started running a larger range of smoketests, farmed out to multiple PCs. Apparently maintaining the smoketests is an issue, and there's a small team dedicated to just keeping the build servers and software maintained, so I can't say if that's been a success or not.