I've recently been interviewing design interns and been pitching what I consider fairly easy practical design questions to measure their skill level. I've been shocked by poor quality of answers. Now this may be the quality of the applicants.
What questions do you ask an entry level games designer to probe for their skill level?
A classic is to ask them about a game they feel strongly about. Ask them why they think certain design choices were made. IE, why can't you jump in Gears of War? You're not looking for right answers, you just want to prove that a given intern can actually think critically about games. If they can't get pat their blinders and identify a flaw in a great game or a positive in a bad one, they may not be able to understand the tradeoffs that go into actual game design.
I set poorly specified design tasks to see if they ask questions. Skills can be taught. Starting with the impression that you know the answer or that there is a right answer is harder to overcome.
Also, here's an off-the-wall request that taught me the most about how the minds of the designers I've interviewed worked: "Tell me three things that I don't know."
Tell me a game you've enjoyed and tell me what you can do to make it better.
I believe this is a good question for the following reasons:
I like a variant of Ben's question: pick a game they really love, then ask what they would change about it. If the answer is "nothing, it's perfect in every way"... well, that's how you tell the difference between a gamer and a designer :)
For level designers, you could always ask them to, you know, design a level. Or assuming you have them submit level design samples with their application, have them walk through their level explaining the choices they made. Critique their level as you go, to see if they can defend their choices while remaining polite when they know they're right (and to see if they can take constructive criticism when they're actually wrong).
