Adapting Skill Ratings to Luck-Based Hidden-Information Games

Avirup Chakraborty Shirsa Maitra Indian Statistical Institute Computer Science Engineering Kolkata, India Heritage Institute of Technology, India Tathagata Banerjee Diganta Mukherjee Department of Statistics & Data Science Sampling and Official Statistics Unit National University of Singapore Indian · 2025

Read Adapting Skill Ratings to Luck-Based Hidden-Information Games by Avirup Chakraborty Shirsa Maitra Indian Statistical Institute Computer Science Engineering Kolkata, India Heritage Institute of Technology, India Tathagata Banerjee Diganta Mukherjee Department of Statistics & Data Science Sampling and Official Statistics Unit National University of Singapore Indian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-18858.txt.

Introduction

Avirup Chakraborty and four co-authors fix player ratings for games where luck deals the cards. The Elo system, built for chess, scores only wins and losses and assumes everyone starts equal. In Rummy, a hidden-information game of chance, the starting hand itself bends outcomes. Their paper, posted to arXiv in December 2025, redesigns Elo for such environments.

Their modified system folds in score-based performance and models the quality of each player's initial hand, separating skill from luck. Across 270,000 simulated games played by six strategies of rising sophistication, the new ratings converge stably, separate skill levels more sharply, and predict results better than classic Elo. The framework stays computationally simple, and the authors note it should transfer to other stochastic contests where chance and skill intertwine.

Catalog record

CREATORAvirup Chakraborty Shirsa Maitra Indian Statistical Institute Computer Science Engineering Kolkata, India Heritage Institute of Technology, India Tathagata Banerjee Diganta Mukherjee Department of Statistics & Data Science Sampling and Official Statistics Unit National University of Singapore Indian
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-18858

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adapting Skill Ratings to Luck-Based Hidden-Information Games Avirup Chakraborty Shirsa Maitra Indian Statistical Institute Computer Science Engineering Kolkata, India Heritage Institute of Technology, India Tathagata Banerjee Diganta Mukherjee Department of Statistics & Data Science Sampling and Official Statistics Unit National University of Singapore Indian Statistical Institute, Kolkata Tridib Mukherjee Chief Data Scientist and AI Officer IDfy, India Abstract Rating systems play a crucial role in evaluating player skill across competitive environments. The Elo rating system, originally designed for deterministic and information-complete games such as chess, has been widely adopted and modified in various domains. However, the traditional Elo rating system only considers game outcomes for rating calculation and assumes uniform initial states across players. This raises important methodological challenges in skill modelling for popular partially randomized incomplete-information games such as Rummy. In this paper, we examine the limitations of conventional Elo ratings when applied to luck-driven environments and propose a modified Elo framework specifically tailored for Rummy. Our approach incorporates score-based performance metrics and explicitly models the influence of initial hand quality to disentangle skill from luck. Through extensive simulations involving 270,000 games across six strategies of varying sophistication, we demonstrate that our proposed system achieves stable convergence, superior discriminative power, and enhanced predictive accuracy compared to traditional Elo formulations. The framework maintains computational simplicity while effectively capturing the interplay of skill, strategy, and randomness, with broad applicability to other stochastic competitive environments. Keywords : Elo rating system, skill evaluation, Rummy, stochastic environments, score-based rating 1 Introduction Player skill evaluation is a fundamental component of competitive gaming ecosystems, guiding matchmaking, player progression, and strategic analysis. Traditional rating systems, such as the Elo [ 1 ] , Glicko [ 2 ] , and TrueSkill [ 3 ] frameworks, have proven effective in deterministic or near-deterministic games like chess and Go, where outcomes primarily reflect player skill. However, their direct application to luck-influenced, incomplete-information games such as Rummy, Poker, or Bridge introduces notable limitations. In such games, outcomes are not solely determined by player decisions but also by stochastic elements like initial card distributions and variable round dynamics. Consequently, standard Elo-type formulations tend to overestimate short-term fluctuations and underestimate persistent skill differences. The Rummy environment presents a unique case within this class. It combines probabilistic uncertainty (due to card draws and opponent hands) with strategic depth (melding, discarding, bluffing), leading to outcomes that depend on both tactical skill and random events. This dual dependence makes player-skill inference inherently noisy, requiring rating mechanisms that can disentangle luck from skill and stabilize ratings over repeated play. Existing systems often fail to achieve this balance. Rapid volatility in ratings discourages consistent player engagement, while slow convergence obscures meaningful skill differentiation. Our study will be focused solely on Rummy. Recent advances in rating systems, such as TrueSkill Through Time [ 4 ] and TrueSkill 2 [ 5 ] , have focused on improving dynamic updating and parameter learning; however, these models remain primarily suited to deterministic or team-based settings. In contrast, Edelkamp et al (2021) [ 6 ] proposed an Elo-based framework for Skat, a game characterized by significant random variation, demonstrating the necessity of correcting standard Elo formulations to account for chance effects. Building on the broader discussion of skill versus luck, Banerjee et al. (2021) [ 7 ] introduced a data-driven statistical framework to quantify the relative influence of skill and chance across popular games, revealing systematic differences among Chess, Rummy, Ludo, and Teen Patti. Together, these studies underscore that rating systems for games involving both randomness and hidden information, such as Rummy, must explicitly incorporate chance-adjustment mechanisms to produce fair and stable estimates of player skill. This paper introduces a modified Elo formulation tailored for Rummy, which can be suitably adjusted to other games played in randomized environments. The method is designed to incorporate luck-adjusted weighting and match-context normalization while maintaining computational simplicity. Further, our method is also adaptable to deterministic games as special cases of parameter choices. Our approach aims to (i) model the probabilistic uncertainty intrinsic to Rummy’s deal structure, (ii) reduce the inflation and deflation effects that arise from streaks of fortunate or unfortunate draws, and (iii) preserve interpretability comparable to the classical Elo scale. The proposed method is evaluated through extensive simulations across varying skill distributions, comparing its stability, convergence rate, and predictive reliability against conventional Elo [ 1 ] , Glicko [ 2 ] , and TrueSkill [ 3 ] frameworks. Section 2 outlines the methodological framework, reviewing standard rating systems and highlighting their limitations in games influenced by chance before introducing our proposed formulation. Section 3 details the proposed methodology and the corresponding parameter estimation procedures. Section 4 describes the simulation design and presents the empirical findings. Section 5 interprets the results, emphasizing how the proposed rating system more effectively captures underlying skill in stochastic environments compared to existing approaches. Finally, Section 6 concludes with potential extensions and practical applications, particularly for online Rummy platforms. 2 Different Existing Rating Systems This section provides an overview of several classical rating systems that have been widely used to evaluate player skill. \bullet Elo-Based Rating Systems: Elo-based systems model player skill as a latent variable that is iteratively updated based on game outcomes. In the classical Elo formulation for a two-player match between players i and j , the expected outcome for player i is modeled using a logistic link: \alpha_{i}=\frac{1}{1+10^{(R_{j}-R_{i})/400}}, and the rating update rule is R_{i}^{\prime}=R_{i}+K(S_{i}-\alpha_{i}), where S_{i}\in\{0,0.5,1\} denotes the observed outcome (loss, draw, or win), and K is a fixed sensitivity parameter. Extensions of the classical model address uncertainty and adaptive learning rates. In Bayesian Elo , each player’s rating is treated as a Gaussian random variable, R_{i}\sim\mathcal{N}(\mu_{i},\sigma_{i}^{2}) , allowing probabilistic updates. Elo-MMR further introduces a player-specific sensitivity factor K_{i}=f(\sigma_{i},\sigma_{j},t) that adapts to rating deviation and recency, enabling faster updates for uncertain players and greater stability for consistent ones. \bullet Glicko Systems: While the Elo system assumes a fixed confidence in player skill, it fails to account for uncertainty arising from limited or inconsistent play. The Glicko system addresses this by introducing a rating deviation (RD) that quantifies uncertainty in a player’s true strength. A higher RD indicates that a player’s current rating is less reliable, typical for newcomers or inactive players, while frequent and consistent play reduces RD, reflecting growing confidence in the estimated skill. From the Elo formula, K is scaled by the rating deviation (larger for uncertain players). Glicko-2 extends this framework by adding a volatility parameter to capture fluctuations in player performance over time, enabling faster correction for unstable or streak-prone players. \bullet TrueSkill and TrueSkill-2: TrueSkill and TrueSkill-2: TrueSkill, developed by Microsoft, extends beyond Elo and Glicko in a Bayesian approach, treating each player’s skill as a normal distribution p_{i}\sim N(\mu_{i},\sigma_{i}^{2}) rather than a fixed point estimate. In a two-player match: P(i\text{ wins})=\Phi\left(\frac{\mu_{i}-\mu_{j}}{\sqrt{2(\sigma_{i}^{2}+\beta^{2})}}\right), where \beta represents random performance variation within games. After each match, both \mu_{i} and \sigma_{i} are updated through approximate Bayesian inference, allowing the model to capture learning and uncertainty dynamically. Unlike Elo or Glicko, which assume scalar and independent ratings, TrueSkill supports multiplayer and team-based games by jointly inferring skills from all outcomes. TrueSkill-2 further refines this by enabling multidimensional skill vectors and real-time tracking, though at the cost

Read the complete text