A Finite Sample Analysis of Distributional TD Learning with Linear Function Approximation

Yang Peng, Kaicheng Jin, Liangyu Zhang, Zhihua Zhang · 2025

Read A Finite Sample Analysis of Distributional TD Learning with Linear Function Approximation by Yang Peng, Kaicheng Jin, Liangyu Zhang, Zhihua Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-14172.txt.

Description

Yang Peng and co-authors study how fast a learning method can pin down a full spread of future rewards, in a 2025 paper. The method, called distributional TD learning, aims at the whole return distribution for a set policy. Most past theory covered only simple table-based setups. Far less was known when the learner must use a linear model on streaming data.

The authors give sharp bounds on how many samples the linear method needs. The rate turns out to match the classic method that learns only the average return. In plain terms, learning the full spread costs no more data than learning its mean. To prove this, they study a Bellman-type equation in fine detail and track how random matrix products settle down.

Catalog record

CREATORYang Peng, Kaicheng Jin, Liangyu Zhang, Zhihua Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE156 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-14172

Browse the full library · About the Tower of Babel