A Finite Sample Analysis of Distributional TD Learning with Linear Function Approximation
Yang Peng, Kaicheng Jin, Liangyu Zhang, Zhihua Zhang · 2025
Read A Finite Sample Analysis of Distributional TD Learning with Linear Function Approximation by Yang Peng, Kaicheng Jin, Liangyu Zhang, Zhihua Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-14172.txt.
Description
Yang Peng and co-authors study how fast a learning method can pin down a full spread of future rewards, in a 2025 paper. The method, called distributional TD learning, aims at the whole return distribution for a set policy. Most past theory covered only simple table-based setups. Far less was known when the learner must use a linear model on streaming data.
The authors give sharp bounds on how many samples the linear method needs. The rate turns out to match the classic method that learns only the average return. In plain terms, learning the full spread costs no more data than learning its mean. To prove this, they study a Bellman-type equation in fine detail and track how random matrix products settle down.
Catalog record
| CREATOR | Yang Peng, Kaicheng Jin, Liangyu Zhang, Zhihua Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 156 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-14172