Ken Larsen's web site

Comparison of Various Backgammon Rating Systems

System Pros Cons Comments
Elo
  • Accounts for the difficulty of opponents.
  • Doing well at a tournament with great players (who have high Elo's) will boost a player's Elo higher than playing well in a tournament with lesser skilled players.
  • Elo is used in other contests and is familar to many people.
  • Places more weight on recent events.
  • Doesn't decay with age.  If a person doesn't play for a protracted length of time, their Elo remains the same.
  • Doesn't remove luck factor.
  • Different stengths of player pools can adversely effect Elo.  An extreme example is this story of Claude Bloodgood.
  • Should be restricted to live tournaments only
Giants Voting
  • Has been in existence since 1993
  • Only done every 2 years

  • Popularity contest

  • Votes are not weighted by the attendance of the voters.  If a voter attended only one tournament their vote is weighted the same as someone who attended 10 tournaments.  That's not right.

  • There hasn't been clear guidance on the criteria that voters should follow.  Some cast their votes based on tournament performance.  Some cast their votes for people who have contributed to promoting the game ... like via a website, tournament commentary, or authoring a book or magazine articles.   Some vote for people who exhibited strong skills many years ago but who haven't played in any major tournaments in years.

Opinions of Ken Larsen:

  • A person should be excluded if they haven't played in a major tournament within the previous two years.  I would even tighten this requirement to stipulate that a person should be excluded if they failed to garner a certain minimum number of Master points in this two year period.
  • Votes should be weighted by the attendance of the voters in major tournaments.  In other words, if a voter has played in 10 major tournaments, their vote should count ten times as much as a voter who has only played in one tournament.  I don't think this is being done.
  • If a voter has played in tournaments on more than one continent, their vote should be accorded more weight, but I'm not sure what the weighting factor should be.
Master Points
  • At least in the U.S.'s ABT system, the size of the field and the skill level of the tournament (Open, Advanced, or Novice) directly influence the quantity of points awarded.
  • Favors attendance
  • Can only go up
  • Doesn't decay with age
  • Doesn't remove luck factor.
  • Treats a 0-3 record and a 2-3 record in a tournament as the same.
  • Should be restricted to live tournaments only. 
  • An example is the ABT system in the U.S.
Larsen-Silliman
  • Limits the effects of attendance
  • Only looks at the last 5 years
  • Places more weight on recent years
  • Only considers main events
  • Currently is only used in the U.S., but it could be expanded to the rest of the world if agreement could be reached on the assignment of Master points worldwide.
  • Doesn't remove luck factor.
  • Treats a 0-3 record and a 2-3 record in a tournament as the same.
  • Is a derivative of Master Points.  It attempts to correct the flaws of the Master Points system.
  • Is restricted to live tournaments only
  • There are a lot of variables that could be tweaked:  number of years considered, weights of each year, weights assigned to attendance.
PR/BMAB
  • Limits the luck factor
  • Unlike Elo, Larsen-Silliman, or Master Points, players from different playing pools may be fairly compared.
  • The BMAB leader (Rick Janowski of the UK) pledges to remove players if they are idle (no tournament competition) for 18 months.
  • Labor intensive to derive
  • subject to cherry picking
  • Due to eXtremeGammon not being perfect, technical issues with calculating PR, and not considering an opponent's deficiencies, playing for a low PR is not the same as playing to win a match.
  • eXtremeGammon's Performance Rating (PR). 
  • Is restricted to live tournaments only

(February 22, 2016) Matt Cohn-Geier's assessment of the Giant's List, Elo, and error rate/PR:  https://www.youtube.com/watch?v=ybc6U0iHSxU&feature=youtu.be  Matt is currently Giant #5.

Ken Larsen's home page