(5.2) Global Rankings (The Collins Rating System – CRS)

The International Axe Throwing Federation (IATF) employs the Collins Rating System to rate players. This system is based on the Elo rating system widely used in chess, online gaming and other kinds of head-to-head competition. The system considers the level of stakes at hand as well as the value of one-on-one competition, and has been adapted by the IATF to accommodate the complexities of axe throwing. The Elo system was selected because it ranks players in head-to-head competition through a mathematical equation.

This system works well for the IATF. We value head-to-head competition, both in our sport and as sports fans, and see the value in a rating system that rates players according to skill and skill improvement. We operate in the spirit of fairness in competition; all venues operate differently, but still have an accurate system of rating for throwers across all IATF organizations. 

Key Principles

• Head-to-head competition should be used to assess skills and determine ratings. (true since version 1.0)
• Beating lower rated players is less valuable than beating higher rated players. (true since version 1.0)
• Higher levels of competition should raise the stakes by allowing larger adjustments, if required to better reflect players’ relative skill. ie. IATC has higher stakes than Regionals which in turn has higher stakes than league playoffs. (true since version 1.0)
• Throwing an 81 and winning the tiebreaker should never result in a downward adjustment (this became clear from version 1.0)
• Throwing more frequently should not disproportionately inflate ratings (true since version 1.0, however version 1.1 created a deviation from this principle, version 1.2 addresses the inflation from version 1.1)

Reviewing How it Works: 

Players start with an initial rating of 1450. 

Each players’ rating goes up or down at the end of a match based on the result of each round. Players have their rating compared to the rating of their opponent. If the higher rated player wins, as expected, adjustments are generally small. However, if the lower rated player wins, an adjustment in ratings moves both players' ratings - up for the lower rated player, and down for the higher rated player.

It should be noted that the first 28 matches a player completes in their career have an additional consideration, specifically, opponents’ ratings in these 28 matches are not adjusted unless the match is also one of the opponents’ first 28 matches. This prevents the uncertainty of new players’ ratings from affecting their opponents’ ratings in their first career season.

CRS 1.0 - The first version of CRS adjusted players rating based on winning or losing rounds within a match. These adjustments were affected by the ratings of their opponents as well as the expected result going into the match. 

For example, in version 1.0 (example A): Alice and Bob are throwing a match. Alice is a higher rated player than Bob. When Alice wins the match, her rating goes up slightly and Bob’s goes down slightly, by the same amount. However, if Bob (as the lower rated player) were to win the match, his rating would go up and Alice’s rating would go down, again by the same amount.

Another example (example B): A highly rated player who ties a lower rated player at 81 and wins the tiebreaker. Based on their rating before the match, the lower rated player has exceeded their expectation by taking the match to a tie breaker, and needed an upward adjustment. Since in CRS 1.0 the adjustments were always equal, but opposite, the higher rated player’s rating would be adjusted downward to match the lower rated player’s increase after the match. The result of which was that the two players were now more closely rated, which was the desired result of the system.

Transition to Version 1.1 - After listening to feedback from the community and observing how the System worked in a live setting, it was clear that the rating calculation was not performing as desired in some match scenarios. It was determined that the cause was that ties at 27 were being treated in the same manner as other ties, which, since this is the maximum score, felt like a penalty for the higher rated player, as outlined in example B, described in the previous paragraph. The natural question was “How could I have done better? Why would my rating go down in that scenario?”

Version 1.1 - To address how ties at 27 were impacting rating adjustments, a modification to the calculation was made to treat these rounds as a win in that round for both players instead of a tie in that round for both players; the rationale being neither player could have done better, so this should count as a win. This meant that in matches with ties at 27, it was now possible for both players to be adjusted positively since both players threw as well as possible in that round. This effectively created new “bonus” rating points, adding them to the system. It also meant that the lower rated players’ adjustments were greater than the higher rated players’.

This modification created a new issue wherein players who were tying at 27 more frequently, especially by throwing in multiple leagues, were receiving a disproportionate boost to their ratings since these “bonus” rating points threw off the original balance of the system. This effect was more pronounced for the lower rated players.

For example, in version 1.1 (example C): Alice and Bob are both rated 1600. Alice and Bob both throw 27s at the same percentage rate, however, Alice throws in one league and Bob throws in four leagues. Bob stands to gain four times the “bonus” rating points.

Following the release of Version 1.1, after listening to community feedback and observing actual match rating adjustments, it was clear that another modification would be needed to align the CRS to some key principles.

Version 1.2 (since July 17, 2020)

A modification to the calculation was made that would treat ties at 27 as follows:

• The higher rated player’s round is treated as their expected result in the round. This means that it is no longer possible to tie at 81 and win the tiebreaker and have the rating adjusted downward.
• The lower rated player’s round is treated as a tie in the round (as with Version 1.0). It should be noted that a tie for the lower rated player is better than expectation. This means that throwing an 81 against a better player can result in a positive adjustment, even if the tie breaker is lost. However, it also avoids the inflated rating adjustments of Version 1.1.

For example, in version 1.2 (example D): Alice and Bob play a match. Alice is rated 1700 and Bob is rated 1500. Alice is expected to win. They both throw three rounds of 27 and tie overall at 81. Alice wins the tiebreaker.

The resulting rating adjustment for each of the CRS versions would be (see The Full Nerd Version below):

For those interested in a more detailed understanding of the rating adjustments calculation, please see The Full Nerd Version below.