Moira: Language-driven Hierarchical Reinforcement Learning for Pair Trading ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Natural language processing CCS: Computing methodologies Planning under uncertainty
Polydoros Giannouris, Yuechen Jiang, Lingfei Qian, Yuyan Wang, Xueqing Peng, Jimin Huang, Guojun Xiong, Sophia Ananiadou · 2026
Read Moira: Language-driven Hierarchical Reinforcement Learning for Pair Trading ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Natural language processing CCS: Computing methodologies Planning under uncertainty by Polydoros Giannouris, Yuechen Jiang, Lingfei Qian, Yuyan Wang, Xueqing Peng, Jimin Huang, Guojun Xiong, Sophia Ananiadou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01954.txt.
Introduction
Polydoros Giannouris and colleagues present MOIRA, a system for pair trading that learns at two levels at once. Pair trading needs a high-level choice of which assets to pair and a low-level plan for when to trade them. Feedback is slow and unclear, so a loss may come from a bad pairing, bad timing, or both.
MOIRA states both levels as language model policies and updates them only by changing prompts, using written feedback from whole episodes. Keeping the pairing choice apart from execution helps assign credit for gains and losses. The framework is built for settings that change over time, where an agent must revise beliefs as well as remember past policy.
Catalog record
| CREATOR | Polydoros Giannouris, Yuechen Jiang, Lingfei Qian, Yuyan Wang, Xueqing Peng, Jimin Huang, Guojun Xiong, Sophia Ananiadou |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.01954
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Moira: Language-driven Hierarchical Reinforcement Learning for Pair Trading ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Natural language processing CCS: Computing methodologies Planning under uncertainty Polydoros Giannouris, Yuechen Jiang, Lingfei Qian, Yuyan Wang, Xueqing Peng, Jimin Huang, Guojun Xiong, Sophia Ananiadou Abstract . Many sequential decision-making problems exhibit hierarchical structure, where high-level semantic choices constrain downstream actions and feedback is delayed and ambiguous. Learning in such settings is challenging due to credit assignment: performance degradation may arise from flawed abstractions, suboptimal execution, or their interaction. We study this challenge through pair trading, a domain that naturally combines long-horizon semantic reasoning for asset pair selection with short-horizon execution under partial observability. We formulate pair trading as a hierarchical reinforcement learning problem and propose a language-driven optimization framework in which both high-level and low-level policies are parameterized by large language models (LLMs) and optimized exclusively through prompt updates. Our approach leverages pretrained LLMs as hierarchical policies and uses trajectory- and episode-level textual feedback to adapt abstractions and execution without gradient-based fine-tuning. By explicitly separating abstraction selection from execution, the framework reduces non-stationarity across hierarchical levels and enables targeted adaptation under delayed feedback. Experiments on real-world market data show consistent improvements over traditional and LLM-based baselines, demonstrating the effectiveness of language-driven hierarchical reinforcement learning. 1. Introduction Many real-world sequential decision-making problems exhibit an inherent hierarchical structure: high-level semantic choices constrain downstream actions, while feedback is observed only after these choices interact with low-level execution in a stochastic environment ( 37 ) . In such settings, the root cause of performance degradation is inherently ambiguous. Failures may come from flawed abstractions, suboptimal execution, or the interplay between the two, yet the environment provides no direct signal to isolate the source of error. This ambiguity poses a fundamental challenge for learning, particularly when abstractions evolve over time, and feedback is delayed and aggregated ( 35 ) . Many agent-based decision systems optimize behavior end-to-end using a single reward-driven policy ( 3 ; 39 ; 8 ) . While effective when behavior and outcomes are tightly coupled, such approaches entangle abstraction and execution: updates affect all components simultaneously, making the source of failure unclear. This limitation is especially severe in hierarchical settings, where errors are revealed only through delayed feedback. Hierarchical reinforcement learning (HRL) explicitly separates abstraction selection from execution, but typically manages credit assignment through strong assumptions such as fixed temporal abstractions or averaging out execution noise ( 44 ; 19 ; 33 ; 28 ) . These choices stabilize learning but limit an agent’s ability to revise abstractions when semantic relationships drift. Pair trading provides a concise stress test for this challenge. Pair selection requires long-horizon semantic reasoning over evolving economic relationships, while profitability depends on short-horizon execution under market noise ( 22 ) . Losses may therefore reflect invalid abstractions, suboptimal execution, or transient noise without clear attribution ( 5 ) , making pair trading a challenging benchmark for hierarchical learning. In this work, we formulate pair trading as a hierarchical sequential decision-making problem ( 9 ; 44 ; 34 ) and study how abstraction, selection, and execution can be adapted under delayed and ambiguous feedback. Rather than enforcing bilevel optimality through explicit algorithmic constraints, we adopt a representational approach: we design the policy class so that hierarchical assumptions are approximately respected by construction. Our key idea is to use language as a shared semantic interface across hierarchical levels. Unlike scalar rewards that obscure the source of error, language enables explicit semantic attribution: textual feedback can distinguish whether a failure stems from a flawed high-level thesis (abstraction) or poor low-level timing (execution). We parameterize both the high-level pair selector and the low-level trader as prompt-conditioned large language models (LLMs), and adapt their behavior solely through prompt updates driven by environment feedback. This induces a natural separation of time scales, distinguishing episodic abstraction updates from intra-episode execution adaptation. This structure aligns credit assignment with the availability of feedback, without requiring gradient-based fine-tuning or expert-labeled data. Building on this design, we propose a language-driven hierarchical optimization framework in which prompt adaptation serves as the sole mechanism for policy improvement. At the low level, the trader performs intra-episode textual policy optimization, using trajectory-level feedback to revise execution rules while conditioning on a fixed selected pair. At the high level, the selector performs episodic textual optimization, receiving delayed, aggregated feedback that reflects the downstream performance of its abstraction choices. By explicitly separating abstraction selection from execution and aligning their adaptation schedules with feedback availability, the framework reduces non-stationarity across hierarchical levels and enables targeted adaptation under ambiguous credit assignment. We evaluate our approach on real-world U.S. equity market data. Empirical results demonstrate consistent improvements over statistical baselines, reinforcement learning methods, and LLM-based alternatives. Ablation studies further show that both hierarchical decomposition and prompt adaptation are necessary: without either component, performance degrades substantially. Together, these results suggest that language-driven hierarchical policy optimization provides a principled and effective approach to sequential decision-making problems characterized by evolving abstractions and delayed feedback. 2. Related Work 2.1. Pair Trading Traditional pair trading relies on statistical heuristics, such as distance minimization ( 15 ) or cointegration tests ( 12 ; 45 ; 23 ) , to identify mean-reverting spreads. However, these parametric methods increasingly suffer from diminishing profitability due to market efficiency and transaction costs ( 4 ; 11 ; 27 ; 38 ) . To address high-dimensional search spaces and non-linear dependencies, recent studies have incorporated machine learning for efficient clustering ( 40 ) and advanced feature engineering ( 18 ; 43 ; 26 ) . Concurrently, the execution paradigm has shifted from static thresholds to dynamic Reinforcement Learning policies ( 17 ; 46 ; 13 ; 6 ) , further enhanced by risk-aware mechanisms like dynamic stop-loss boundaries and position scaling ( 2 ; 52 ; 24 ; 25 ) . Despite these advances, most approaches struggle with structural breaks ( 31 ) and treat pair selection and trading as isolated tasks. While emerging unified frameworks ( 19 ; 51 ) attempt joint optimization ( 42 ) , they rely exclusively on numerical data, neglecting the semantic information essential for understanding fundamental asset correlations. 2.2. LLMs for Trading Early applications primarily utilized LLMs for return prediction through sentiment analysis of financial news or specialized instruction tuning ( 30 ; 54 ; 29 ) . Beyond text analysis, generative approaches have emerged to autonomously mine trading strategies by modeling price histories or synthesizing executable code ( 32 ; 48 ; 47 ) . To handle sequential decision-making in volatile markets, recent research has shifted toward autonomous agents capable of end-to-end execution. Several frameworks enhance decision comprehensiveness by integrating multimodal data, external tools, or system-level simulations ( 57 ; 53 ; 14 ) . More advanced architectures integrate gradient-based reinforcement learning for policy optimization ( 50 ) or employ multi-agent debate mechanisms ( 49 ) . Notably, FinMem ( 55 ) addresses long-term dependency via a layered memory mechanism, while FinCon ( 56 ) improves robustness by simulating an investment firm’s hierarchy through conceptual verbal reinforcement. 2.3. LLMs as Hierarchical Policies Recent work demonstrates that LLMs can be incorporated into trading systems as decision-making components, with several studies reporting promising trading performance ( 10 ; 56