MARS: A Meta-Adaptive Reinforcement Learning Framework for Risk-Aware Multi-Agent Portfolio Management

Jiayi Chen, Jing Li, Guiling Wang · 2025

Read MARS: A Meta-Adaptive Reinforcement Learning Framework for Risk-Aware Multi-Agent Portfolio Management by Jiayi Chen, Jing Li, Guiling Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01173.txt.

Introduction

Jiayi Chen, Jing Li, and Guiling Wang design a team of trading agents in 2025 that shifts with the market's mood. Automated portfolio tools often fail to balance risk and gain when conditions flip. Their frame, MARS, replaces one big model with a mixed crew of agents, each with its own risk bent.

A safety-check network holds each agent to its bent, from guarding capital to chasing growth. A top controller then leans on the cautious agents in downturns and the bold ones in rallies. Tests on big world indexes cut the worst drops and swings while keeping returns in range, they report. The crew's spread of styles, not fancy inputs, is what keeps the portfolio steady.

Catalog record

CREATORJiayi Chen, Jing Li, Guiling Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-01173

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MARS: A Meta-Adaptive Reinforcement Learning Framework for Risk-Aware Multi-Agent Portfolio Management Jiayi Chen, Jing Li, Guiling Wang Abstract Reinforcement Learning (RL) has shown significant promise in automated portfolio management; however, effectively balancing risk and return remains a central challenge, as many models fail to adapt to dynamically changing market conditions. In this paper, we propose Meta-controlled Agents for a Risk-aware System (MARS), a novel RL framework designed to explicitly address this limitation through a multi-agent, risk-aware approach. Instead of a single monolithic model, MARS employs a Heterogeneous Agent Ensemble where each agent possesses a unique, intrinsic risk profile. This profile is enforced by a dedicated Safety-Critic network and a specific risk-tolerance threshold, allowing agents to specialize in behaviors ranging from capital preservation to aggressive growth. To navigate different market regimes, a high-level Meta-Adaptive Controller (MAC) learns to dynamically orchestrate the ensemble. By adjusting its reliance on conservative versus aggressive agents, the MAC effectively lowers portfolio volatility during downturns and seeks higher returns in bull markets, thus minimizing maximum drawdown and enhancing overall stability. This two-tiered structure allows MARS to generate a disciplined and adaptive portfolio that is robust to market fluctuations. The framework achieves a superior balance between risk and return by leveraging behavioral diversity rather than explicit market-feature engineering. Experiments on major international stock indexes, including periods of significant financial crisis, demonstrate the efficacy of our framework on risk-adjusted criteria, significantly reducing maximum drawdown and volatility while maintaining competitive returns. Introduction The application of Deep Reinforcement Learning (DRL) to automated portfolio management has undergone a significant evolution, transitioning from foundational concepts to sophisticated end-to-end trading systems. Early efforts focused on adapting classic DRL algorithms to financial decision-making, establishing DRL as a viable paradigm for learning sequential trading policies directly from market data ( Jiang, Xu, and Liang 2017 ; Bai et al. 2024 ) . The development of open-source libraries such as FinRL played a pivotal role during this period by standardizing the application of DRL algorithms to financial markets and improving reproducibility ( Liu et al. 2020 ) . Despite these advances, the direct application of generic DRL algorithms to financial markets exposed a fundamental mismatch. Financial environments are inherently noisy and exhibit pervasive non-stationarity ( Zhang and Xie 2025 ) , where the statistical properties of financial time series change over time, violating the core Markov Decision Process (MDP) assumption of a stationary environment. As a result, models trained in one market condition, such as a low-volatility bull market, often fail catastrophically when the regime shifts, rendering previously learned patterns obsolete ( Zhang and Xie 2025 ) . The second critical limitation is the superficial treatment of risk . In many DRL models, risk management is handled implicitly through reward shaping, such as using risk-adjusted metrics like the Sharpe ratio as the reward signal or adding penalties for large drawdowns ( Reis, Serra, and Gama 2025 ) . This approach is fundamentally reactive, treating risk as a consequence to be penalized after the fact, rather than as a factor to be proactively managed within the decision-making process, as human traders do. As a result, agents are often vulnerable to tail risks and sudden market shocks. Notably, these two challenges–non-stationarity and risk–are deeply interconnected: an agent that fails to adapt to changing market regimes cannot effectively manage risk. In this paper, we propose MARS ( M eta-controlled A gents for a R isk-aware S ystem), a novel framework that explicitly addresses the dual challenges of non-stationarity and risk. Different from the conventional monolithic agent paradigm, MARS employs a meta-learning-controlled multi-agent architecture that decouples risk preference and management from market adaptation. At the lower level, MARS employs an ensemble of heterogeneous Safety-Critic Agents . Each agent consists of three networks: an Actor, a Critic, and a SafetyCritic that learns to estimate the risk associated with a given state-action pair ( Srinivasan et al. 2020 ) . Crucially, each agent is explicitly configured with a distinct risk tolerance, defined by a safety threshold ( \theta_{i} ) and a safety weight ( \lambda_{i} ), embedding risk management directly into its learning objective in a principled and structural manner. At the higher level, a Meta-Adaptive Controller acts as a meta-policy, learning to dynamically orchestrate the agent ensemble. It takes the current market state as input and outputs a set of weights that determine each agent’s contribution to the final trading decision. By learning to modulate these weights, the meta-controller allows the framework to adapt its aggregate strategy–ranging from “Ultra Conservative” to “Maximum Growth”–in response to the non-stationary dynamics of the financial environment. Specifically, this paper makes the following primary contributions: 1. We propose MARS , a novel meta-learning-controlled multi-agent DRL framework featuring an ensemble of Safety-Critic Agents with explicit risk-profiles, orchestrated by a Meta-Adaptive Controller to address the dual challenges of non-stationarity and risk management. 2. We design a novel, two-part risk management mechanism specific to portfolio management: (1) a custom environmental risk score that provides the Safety-Critic a nuanced, holistic understanding of both structural and market risks, and (2) a rule-based overlay that ensures all executed actions are compliant with practical, real-world trading constraints. 3. Extensive experiments on real-world stock market data show that MARS significantly outperforms both traditional and state-of-the-art DRL baselines, particularly in achieving higher risk-adjusted returns and preserving capital during periods of elevated market volatility. Related Works Recent research in quantitative finance reveals a significant methodological evolution from supervised prediction to end-to-end Reinforcement Learning (RL). This shift is motivated by the “prediction-profitability gap,” where higher prediction accuracy does not reliably translate to better trading returns ( Jiang, Zhu, and Hu 2024 ) , and by RL’s inherent suitability for sequential decision-making. Researchers are applying increasingly sophisticated RL paradigms to tackle financial challenges like market non-stationarity and low signal-to-noise ratios ( Liu et al. 2022 ; Wang et al. 2024 ) , leading to a diverse ecosystem of approaches. Concurrently, the development of standardized benchmarks like FinRL-Meta ( Liu et al. 2022 ) and TradeMaster ( Sun et al. 2023 ) signifies a community-wide push for greater scientific rigor. Model-Free Approaches . Model-free RL approaches learn trading policies directly from market interaction without an explicit market model. Recent advancements focus on augmenting standard RL agents with domain-specific architectures. For instance, DeepTrader is a risk-aware agent using a dual-module architecture to balance return with risk by embedding market conditions and penalizing high portfolio drawdown, allowing it to adapt its strategy to different market regimes ( Wang et al. 2021c ) . Addressing a different challenge, Logic-Q is a knowledge-guided system that injects human-like trading logic into a DRL agent via program sketching ( Li et al. 2025 ) . This helps the agent identify major market trends and prevent catastrophic errors during trend shifts, thereby improving robustness ( Li et al. 2025 ) . Model-Based and Hybrid Approaches . Model-based and hybrid approaches integrate predictive components to provide the RL agent with a richer understanding of the market, aiming to improve sample efficiency. A prime example is StockFormer, which fuses a predictive coding module with an RL agent, using specialized Transformer branches to learn latent representations of future dynamics for a Soft Actor-Critic (SAC) agent ( Gao, Wang, and Yang

Read the complete text