Alpha Discovery via Grammar-Guided Learning and Search

Han Yang, Dong Hao, Zhuohan Wang, Qi Shi, Xingtong Li · 2026

Read Alpha Discovery via Grammar-Guided Learning and Search by Han Yang, Dong Hao, Zhuohan Wang, Qi Shi, Xingtong Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22119.txt.

Introduction

Han Yang and co-authors search for trade rules that work. In stock trade, a rule is a math form that aims to pick stocks that will do well. To find good rules by machine is a prized task. But most search tools roam free, with no check on form. They coin rules that break the rules of trade or mean naught at all.

The team sets a strict grammar for the search. Each rule the tool may coin must fit that grammar. It must be well formed and make sense in trade terms. Their tool is called AlphaCFG. Rules found this way can be read by men and checked by hand. They can also be used with more trust in live trade.

Catalog record

CREATORHan Yang, Dong Hao, Zhuohan Wang, Qi Shi, Xingtong Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22119

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Alpha Discovery via Grammar-Guided Learning and Search Han Yang, Dong Hao, Zhuohan Wang, Qi Shi, Xingtong Li Abstract Automatically discovering formulaic alpha factors is a central problem in quantitative finance. Existing methods often ignore syntactic and semantic constraints, relying on exhaustive search over unstructured and unbounded spaces. We present AlphaCFG, a grammar-based framework for defining and discovering alpha factors that are syntactically valid, financially interpretable, and computationally efficient. AlphaCFG uses an alpha-oriented context-free grammar to define a tree-structured, size-controlled search space, and formulates alpha discovery as a tree-structured linguistic Markov decision process, which is then solved using a grammar-aware Monte Carlo Tree Search guided by syntax-sensitive value and policy networks. Experiments on Chinese and U.S. stock market datasets show that AlphaCFG outperforms state-of-the-art baselines in both search efficiency and trading profitability. Beyond trading strategies, AlphaCFG serves as a general framework for symbolic factor discovery and refinement across quantitative finance, including asset pricing and portfolio construction. 1 Introduction 1.1 Alpha discovery In quantitative finance, alpha factors play a central role in asset management, quantitative trading, and investment decision-making. An alpha factor is an explicit function that maps historical market features, such as prices and volumes, to predictions of future returns. Alpha discovery refers to the systematic identification of such predictive functions from historical data and remains a core challenge due to the vast and complex space of possible functional forms. Beyond their practical importance, alpha discovery poses a fundamental machine-learning challenge: identifying symbolic functions that are both predictive and interpretable under severe combinatorial constraints. Existing approaches to alpha discovery can be broadly classified into three categories. Heuristic or expert-driven methods rely on financial intuition, such as value factors (e.g., price-to-earnings ratios ( 6 ) ) and momentum factors (e.g., past 12-month returns ( 3 ) ), but lack scalability and are quickly arbitraged once widely adopted, reducing predictive accuracy over time. Data-driven learning methods , including regression ( 1 ; 15 ; 5 ; 14 ) , tree-based ensembles ( 19 ; 2 ) , unsupervised learning ( 20 ) , and reinforcement learning ( 12 ) , can capture complex nonlinear patterns, yet often suffer from limited interpretability and overfitting due to their black-box nature. Formulaic alpha methods ( 23 ) emphasize human-readable mathematical expressions composed of predefined operators, offering transparency and interpretability, and have therefore regained recent attention. Table 1 : Comparison of Alpha Discovery Methods Category Pros Cons Heuristic / Expert Intuitive, easy to use Limited, quickly arbitraged Data-driven Learning Captures complex patterns Black-box, less interpretable Formulaic Alpha Interpretable, transparent Computationally expensive Our work lies at the intersection of data-driven learning and formulaic alpha methods, aiming at the automatic discovery of explainable alpha factors . This problem can be viewed as symbolic regression ( 13 ) , which seeks explicit mathematical expressions that fit data while remaining interpretable , but is difficult due to its combinatorial search space and semantic equivalence among expressions. Early approaches such as genetic programming (GP) ( 23 ) evolve expression trees to optimize information coefficients. More recent methods, including AlphaGen ( 22 ) and AlphaQCM ( 24 ) , adopt reinforcement learning to improve scalability. Existing methods face the following fundamental challenges. (1) Lack of linguistic characterization leads to inefficient search in an unbounded space. Automated discovery of formulaic alphas is fundamentally a problem of searching over mathematical languages, yet existing methods lack an explicit linguistic framework to organize and constrain this search. In the absence of formal grammatical structure, current approaches must explore vast, and often effectively infinite, combinatorial spaces of expressions, relying on ad hoc syntactic checks to ensure validity. This unstructured exploration severely limits sample efficiency, degrades model performance, and incurs substantial computational cost. (2) Semantic redundancy causes systematic waste in learning and search. Many syntactically distinct mathematical sequences correspond to the same underlying semantics, but existing methods mostly encode expressions as linear sequences and treat the variants as independent. As a result, semantically equivalent expressions are repeatedly explored and evaluated, leading to significant redundancy in representation learning and search, and greatly reducing efficiency. 1.2 Our Work We propose AlphaCFG, 1 1 1 Our source code is available at https://github.com/HanYang544/AlphaCFG a general linguistic–learning framework for the automatic discovery of interpretable alpha factors. The central idea is to treat alpha discovery as a structured language generation and learning problem, rather than an unstructured search over mathematical expressions. By combining formal grammar with learning and search, AlphaCFG provides a principled way to generate, validate, and optimize human-readable alpha factors. In this way, grammar serves as an explicit inductive bias that shapes both the search space and the learning dynamics. (1) Grammar-Constrained Alpha Factors. From a language-theoretic perspective, we first formalize the space of alpha factors as a structured mathematical language. We propose two formal languages, \alpha -Syn and \alpha -Sem, that integrate context-free grammar (CFG) with finance domain–specific knowledge of alpha factors. \alpha -Syn enforces grammatical correctness, while \alpha -Sem further ensures financial semantic validity. These languages generate alpha expressions recursively in a tree-structured form, making tree-structure–based learning and optimization possible. To control complexity and reduce redundancy, we further enforce (i) length constraints to bound the search space, and (ii) expression-tree pruning to remove syntactically distinct but semantically equivalent factors. (2) Structure Characterization of Alpha Space . Building on this grammar-based language, we cast alpha discovery as a large Tree-Structured Linguistic Markov Decision Process (TSL-MDP), where each state is a partial expression, terminal states represent complete alpha factors, and rewards are given by the information coefficient (IC) on real market data. This formulation transforms alpha discovery from unstructured trial-and-error into a principled sequential decision process over the space of formulaic alpha factors. (3) Reinforcing MCTS with Syntax-Aware Learning . Finally, we design a learning and search algorithm that exploits the grammar-induced structure of the TSL-MDP. We employ a grammar-aware Monte Carlo Tree Search (MCTS), in which action selection is guided by a syntax-aware Upper Confidence Bound (UCB) rule. To generalize across the large state space, each partial expression tree is encoded using a Tree-LSTM, yielding structure-aware representations shared by a value network, which estimates expected performance from historical market data, and a policy network, which predicts promising alpha expansions. Through reinforced interaction between MCTS and these learned models, AlphaCFG progressively refines its search strategy and discovers high-quality alpha factors efficiently. AlphaCFG is not limited to trading strategies and naturally extends to other quantitative finance tasks, by allowing flexible customization of operators, grammatical structures, and objective functions. We use trading as a representative testbed to demonstrate the effectiveness of AlphaCFG. We evaluate AlphaCFG on CSI 300 and S&P 500 stocks, where it consistently outperforms strong baselines across multiple metrics, including returns, information coefficient (IC), Sharpe ratio, and maximum drawdown. Our results show that improved grammar design yields faster convergence and higher-quality factors. Moreover, AlphaCFG effectively refines existing factors and improves their predictive performance , highlighting its utility as a general tool for factor refinement and augmentation. Ablation studies further confirm the critical roles of grammar design and syntax-based representation learning in effective factor discovery. 2 Problem Formulation Consider a market with n stocks over T trading days. On each day t\in\{1,2,\dots,T\} , stock i is associated with a feature matrix \mathbf{x}_{t,i}\in\mathbb{R}^{m\times\tau^{\prime}} , which

Read the complete text