Do It for HER: First-Order Temporal Logic Reward Specification in Reinforcement Learning (Extended Version)

Pierriccardo Olivieri, Fausto Lasca, Alessandro Gianola, Matteo Papini · 2026

Read Do It for HER: First-Order Temporal Logic Reward Specification in Reinforcement Learning (Extended Version) by Pierriccardo Olivieri, Fausto Lasca, Alessandro Gianola, Matteo Papini (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06227.txt.

Description

Rewards in reinforcement learning often depend on a whole history, not just the last state, which makes them hard to specify. Pierriccardo Olivieri and 3 co-authors propose writing such rewards in a rich temporal logic. Their logic lets predicates be full first-order formulas over real data, not just true-or-false flags.

That richness removes hand-coding but adds cost, so the team isolates a fragment that stays tractable while covering real tasks. On the practical side, reward machines plus a hindsight replay trick turn the logic into learnable signals and fight sparse rewards. Tests in a continuous control setting with non-linear arithmetic showed the approach allows natural wording of complex tasks.

Catalog record

CREATORPierriccardo Olivieri, Fausto Lasca, Alessandro Gianola, Matteo Papini
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-06227

Browse the full library · About the Tower of Babel