Soft Q ⁡ ( λ ) Q(\lambda) : A multi-step off-policy method for entropy regularised reinforcement learning using eligibility traces

Pranav Mahajan, Ben Seymour · 2026

Read Soft Q ⁡ ( λ ) Q(\lambda) : A multi-step off-policy method for entropy regularised reinforcement learning using eligibility traces by Pranav Mahajan, Ben Seymour (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13780.txt.

Description

Mahajan and Seymour develop Soft Q(lambda) for entropy-regularised reinforcement learning. It is a multi-step, off-policy method using eligibility traces. Soft Q-learning optimizes returns plus a penalty for diverging from a reference policy. Its multi-step extensions were limited to on-policy sampling under the Boltzmann policy. The authors give an n-step formulation, extend it fully off-policy with a new Soft Tree Backup operator, and unify both into an online trace framework for credit assignment under arbitrary behavior policies.

Catalog record

CREATORPranav Mahajan, Ben Seymour
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29528
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13780

Browse the full library · About the Tower of Babel