# Skill vs. Luck, the ratio and its measurement

Gamer buddies, is there a term to describe the level of variance in a game, in comparison to luck. The card game war would have 0 skill and 1.0 luck because the player cannot affect the game. I can't think of something that has 1.0 skill. At first I thought Spelling Bee, but the words chosen for each contestant is randomly chosen suggesting some luck involved... What ratios do different games have, and how can those ratios be accurately measured? What metrics could be used to accurately measure such a ratio? I'd also like to hear of any 1.0 skill games if anyone can think of one.

To reiterate the question clearly: does there exist such a measurement and if so what is it? Furthermore is there a term for the target of this measurement, so we can have a discussion using a noun.

EDIT: the term luck is used to describe the level of effect that chance, i.e. random events, have in affecting who the winner is. I appreciate everyone's responses.

• Seems like a pretty broad question. Maybe you should ask in chat. It's a very discussion oriented question. – MichaelHouse Aug 31 '13 at 5:13
• This question may get better results if you present an actual problem that you're trying to solve, with things you've tried and what you hope to find, instead of fishing for hard metrics in a very squishy field like design =) – Patrick Hughes Aug 31 '13 at 5:51
• Chess is a game that's a 1.0 for skill and a 0.0 for luck. Skill there is measured with the Elo rating system. However, once you have any random elements in a game, the ratio between skill and luck becomes very blurred. Even the Elo system is just an attempt to quantify (a) skill. I've not heard of any such systems that try the same for luck. Without being able to quantify both using a universal metric, there's no such thing as a ratio between the two. – MichaelHouse Aug 31 '13 at 6:26
• See this duplicate question asked for board games: boardgames.stackexchange.com/questions/9697/… – MichaelHouse Aug 31 '13 at 6:27
• "is there a term to describe the level of variance in a game, in comparison to luck" The question presupposes an obligate condition that "luck" and "skill" exist in a linear combination, such that having some of one naturally detracts from the other. There is no reason to make such an assumption. Furthermore, it would be impossible to "compute" such things, as it requires that game design is somehow quantifiable. Which it is most assuredly not. – Nicol Bolas Aug 31 '13 at 12:38

## 5 Answers

This answer assume familiarity with normal distributions and standard deviations.

A simple but usually reasonable assumption is that we can describe the outcome of a game as a random event where player1 wins if player1's skill plus a normal distributed random variable is greater than player2's skill. The standard deviation of that normal distribution can be compared to the difference between the two player's skills, and for a larger group of players we can compare the standard deviation of the normal distribution to the standard deviation of the skill levels of that player group.

Thus if we for instance have a group of players where the standard deviation of those players's skills is double the standard deviation of the luck of the game we could with some reason say that the game for this group is 1/3 luck and 2/3 skill, but this is only valid for that specific group of players, there is no universal way to measure luck versus skill in a game.

## Edit: Some examples to illustrate the difficulties of the question

All games for two players.

Flip and choose
First a coin is flipped to determine who goes first, then each player in turn choose a number from 1 to 10. Whoever choose the biggest number wins, in case of a draw the player who started wins.

Gomoku with coin flip
First a coin is flipped to determine who goes first, then the players play a standard match of Gomoku on a 15x15 board, whoever wins that game wins.

### Analysis

Intuitively we'd say that Flip and choose is a game of luck, an average person would figure the optimal play before even playing a single round, so effectively the coin flip is all that matters.

Gomoku is game of skill, an average person will not be able to produce optimal play. Still, starting is an advantage so at least the flip of the coin must count for some luck in the final verdict.

With optimal play Gomoku is a win for the player who goes first, it is also a solved game, so a computer equipped with the solution database will always win if it is allowed to go first. Thus to computer players both games are trivial extensions to a standard coin flip, whoever wins the flip wins the game. This would suggest that they are both games of 100% luck. To reach any other conclusion we must consider a player base of less skill.

• I believe you overstate your case in the second paragraph. While the level of play influences the details of the calculation, and hence the result, the method itself doesn't change. I believe OP is looking for a measurement at some sort of standard level of play, let's call it good club player for sake of starting the discussion. – Pieter Geerkens Aug 31 '13 at 14:46
• I know, the above sounds rather vague. I counter that the solution to all complex problems starts vague, and becomes clearer as the solution slowly acquires its final shape. – Pieter Geerkens Aug 31 '13 at 14:48
• @PieterGeerkens I believe that I phrase it quite accurately, there is no universal way. But we still may be able to find a suitable terran method for selecting a representative player group for determining a ratio. – aaaaaaaaaaaa Aug 31 '13 at 15:30
• That's more like it! – Pieter Geerkens Aug 31 '13 at 16:09
• I am not so interested in rating players but rating games themselves. Checkers for example is a solved game, so if two players play optimally, player two always wins, or maybe p1, I can't remember right now. Something like guess the coin flip has no skill playing a role in determining the winner. – mwjohnson Sep 1 '13 at 0:13

does there exist such a measurement and if so what is it?

No, no such measurement exists. While you may be able to come up with a metric for skill. You'll be hard pressed to come up with a metric for luck (unless it's controlled luck). However, the two metrics would likely be different enough that you're essentially taking the ratio of apples/oranges. Further, the metrics will vary from game to game, so comparing ratios between two games is comparing apples/oranges to GI Joes/cats.

However, there are ways to decide if a game is a game of skill or a game of chance, at least from juridical point of view. Specifically, gambling in law. A number of states in the US allow people to pay money to enter games of skill, but not games of chance (or at least significantly limit the amount of money that can be spent on games of chance). There is a paper on the topic, but the All Games of Chance website has a decent definition of how these are legally categorized:

There are two main differences between games of chance and games of skill. The first difference is who the player is playing against. When a player is playing against the house, it is a game of chance. When the player is pitted against other players, it is considered to be a game of skill. Also, if an individual can prove that a particular game involves the use of skill like strategies, statistics or math along with a factor of luck or chance, the game would be allowed and would be categorized as game of skill.

• As Canadian Bridge Champion and barrister Eric Murray is reputed to have argued in the case of Regina vs St. Clair Bridge Club: "Bridge is only a game of chance when it is played as the justices of the Ontario Supreme Court play it [in chambers]." As the St. Clair Bridge Club still runs rubber bridge for high stakes several decades later, I will take it that they won that case. – Pieter Geerkens Sep 1 '13 at 2:54

An important point to remember is that the importance of skill vs luck in determining the winner of a match increases as the number of games in a match increases. For example, this is why golf tournaments are 4 days long; the influence of luck (at the PGA level of play) is simply too great over a mere 18 holes.

This then provides a means of measuring the relative importance of luck vs skill: the number of matches (or alternatively, hours played) required to accurately determine the better player with a given statistical confidence. (95% would be the usual standard in such a case, as in the familiar 19 times out of 20.)

Then we get:

1. Golf would be rated at 16 rounds (of 18 holes) or 64 hours (16 rounds of 4 standard hours play) if you take the FedEx playoffs as the standard to accurately rate the players.
2. Backgammon is usually played to best of 21 I believe in tournament play, but individual games would be averaging 2 or 3 due to the doubling cube. It's rating would then be about 7 - 10 matches, but only perhaps the same 7 - 10 hours.
3. Duplicate bridge would be rated about 2 sessions of 4 hours each, looking at the elimination rounds of larger team events like Vanderbilt and Spingold.
4. Chess world championships are regularly best of 12 (and I believe Go championships are similar).

Noting particularly from the latter point, even such seminal games of skill as Chess and Go are believed to possess a considerable element of luck per individual game, when played at a professional level. This would seem to be borne out by the extreme rarity of sweeps in such competitions.

Update:
A confound when using number of hours of play is that organizing committees may have unstated reasons for extending the length of individual games. My personal belief is that the overall quality of chess games at the world level would not decrease much if the allotted time were halved. However, there seems to be the unstated intent to showcase all of the individual games as best instances of play, leading to the players having more clock time than might be strictly necessary to determine the best player. (This is not necessarily wrong, simply a complication to note when measuring relative importance of skill vs luck.)

For example, Chess and Go matches extend to an almost obscene number of hours, clearly more than necessary to determine the best player given the, both believed and evidenced, high ratio of skill to luck even in individual games. If the sole purpose of world championship matches were the determination of the best player, the number of play hours, and possibly the number of games, could be reduced for both of these games.

• I think you're confusing "luck" with "ability to reproduce the successful outcome". "Luck" in the question referred to an advantage that you never trained or worked to get -- you just literally got handed "the good card". You are classifying "lucky shots" under "luck", which I wouldn't -- I would classify an unlikely hole in one as skill, not luck (since the player must have trained to be able to do that). – bobobobo Aug 31 '13 at 13:33
• @bobobobo: Strange. My mother played golf regularly for years with a woman who averaged better than one hole-in-one a year. Her comment on them was that every single one was a badly-hit shot that got lucky. – Pieter Geerkens Aug 31 '13 at 14:32
• Yeah yeah. That's what people say to be modest. I personally always hated it when my 3-pointers were called lucky, while I would always want to chalk them up to skill. – bobobobo Aug 31 '13 at 14:49
• 3-pointers are skill, because you aim at the basket (and choose your floor spot carefully). Holes-in-one are luck because (usually, in tournaments at least), the pin placements are such that only weak players aim at the pin. The strong players are fading or drawing to ensure that a miss is into a safe area to save par from. – Pieter Geerkens Aug 31 '13 at 14:51
• I do agree that there is something to be said about skill ending at the same point intent does, and luck beginning. At the same time, I've only played golf once, and not once was my ball ever even near the green let alone the hole, so I will argue it does take skill to get a hole-in-one. Whether that was the precise intent of the player at the time is another story. – mwjohnson Sep 2 '13 at 13:48

Back-of-the-Napkin approach:

1. Need a larger sample size and longer time series than you probably would suspect intuitively.
2. K.I.S.S.: How quickly do the winners and losers "revert to the mean?" If the mean "reversion/regression" is slow then skill plays a larger roll. If mean "reversion/regression" is fast then luck plays a more significant role in the outcome(s).
3. If the game is digital, and the code is locked, then trying to tease apart luck from skill is a waste of your time, since any algorithm imaginable could be shaping the outcomes.

Some measures have been proposed, see

The basic idea from the first paper is to estimate

skill = (potential learning effect) / (potential learning effect + potential random effect)

which gives skill as a number between 0 and 1. Alas, these effects are analytically computable only for "easy" games. For a one-player game, the above equation comes down to

skill = (Gm - G0) / (Gu - G0)

where the G's are the expected net gains of three players

• '0': a beginner who plays the game in the naive way of somebody who has just mastered the rules of the game.

• 'm': a real average player who can be thought to represent the vast majority of players.

• 'u': a virtual average player whom we tell in advance (i.e. before he has to decide) the outcome of the chance elements.

As an example they calculate for American Roulette: Gu = 35 and Gm = -1/74, the latter corresponding a "simple" play (e.g. rouge/noir, pair/impair). The value for G0 is actually a matter of debate, even for this game. If the beginner goes for a simple strategy, then the skill is 0 obviously. However if G0 is for a non-simple strategy (e.g. plein, cheval, carre), then G0 is -1/37 (i.e. worse average loss.) So with the latter assumption, there is a minor potential for learning, so skill is 0.0004. I have to say I'm a little miffed that they use French terminology for the American Roulette; alas they source they cite for further details is in Dutch.

For Blackjack they derive from a computer simulation that Gm = 0.11, Gu = 27, and take G0 = -0.057 for a "mimic the dealer" strategy, and from that obttain a skill of 0.006.

For games where players compete directly and strategies like sandbagging or bluffing matter (these are the only games called multi-player games in game theory by the way), the second paper has a more sensible approach in that it considers players potentially changing strategy a source of randomness. They use the same skill formula as above (except that they calll the three types of players beginner, optimal and fictive player). The difference in their approach is that

the expected gains for player i as an optimal player are given by his expected gains in the Nash equilibrium of the related two-person zero-sum game against the coalition of the other players

and for the "fictive" player they also assume that he knows the the outcome of the randomization process of his opponents.

Alas there aren't any examples that are intersting but simple enough to relate in detail here. They compute for a simplified version of drawpoker a skill of 0.22.

Both papers emphasise however that the exact skill value depends on the definition/assumption of beginners’ behavior.

An experimental approach is needed for more complex games of practical interest, e.g.

Those players identified a priori as being highly skilled achieved an average return on investment of over 30 percent, compared to a -15 percent for all other players. This large gap in returns is strong evidence in support of the idea that poker is a game of skill.

• @AlexandreVaillancourt: I was going to. Is this site Latex-enabled? I'm struggling to type in any equations here. – Fizz Nov 22 '17 at 13:56
• Not yet, sorry :/ – Vaillancourt Nov 22 '17 at 14:01
• This is an excellent response, thank you for your contribution. It's interesting to see how little skill is involved in some of those classic casino games; makes sense, economically. – mwjohnson Nov 22 '17 at 23:22