Relational Probing: LM-to-Graph Adaptation for Financial Prediction

Yingjie Niu, Changhong Jin, Rian Dolphin, Ruihai Dong · 2026

Read Relational Probing: LM-to-Graph Adaptation for Financial Prediction by Yingjie Niu, Changhong Jin, Rian Dolphin, Ruihai Dong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10212.txt.

Introduction

Niu, Jin, Dolphin, and Dong turn hidden states straight into link graphs for market forecasts with Relational Probing. Prompting chains pay step-by-step reading costs and split graph building from downstream tuning. Their method swaps the model head for a link head making a graph from hidden states, jointly trained with a stock model. It learns meaning, preserves the induced graph's strict structure, and recasts outputs into task formats. Tests with Qwen3 bases of three sizes beat a pair-count rival each time at low run cost.

Catalog record

CREATORYingjie Niu, Changhong Jin, Rian Dolphin, Ruihai Dong
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30291
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10212

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Relational Probing: LM-to-Graph Adaptation for Financial Prediction Yingjie Niu, Changhong Jin, Rian Dolphin, Ruihai Dong Abstract Language models can be used to identify relationships between financial entities in text. However, while structured output mechanisms exist, prompting-based pipelines still incur autoregressive decoding costs and decouple graph construction from downstream optimization. We propose Relational Probing , which replaces the standard language-model head with a relation head that induces a relational graph directly from language-model hidden states and is trained jointly with the downstream task model for stock-trend prediction. This approach both learns semantic representations and preserves the strict structure of the induced relational graph. It enables language-model outputs to go beyond text, allowing them to be reshaped into task-specific formats for downstream models. To enhance reproducibility, we provide an operational definition of small language models (SLMs): models that can be fine-tuned end-to-end on a single 24GB GPU under specified batch-size and sequence-length settings. Experiments use Qwen3 backbones (0.6B/1.7B/4B) as upstream SLMs and compare against a co-occurrence baseline. Relational Probing yields consistent performance improvements at competitive inference cost. 1 Introduction Stock market prediction is driven not only by a stock’s own momentum but also by inter-stock relationships. Studies ( Pirinsky & Wang 2006 ; Cohen & Frazzini 2008 ; Menzly & Ozbas 2010 ) indicate that related companies tend to co-move and exhibit cross-predictability. To leverage this effect, researchers explicitly construct relational graphs based on stock relationships, including sector information ( Feng et al. 2019 ; Cui et al. 2023 ) , shareholding properties ( Chen et al. 2018 ) , Wikipedia-based relevance ( Sawhney et al. 2021 ) , momentum spillover effects ( Cheng & Li 2021 ) , and price-correlation graphs ( Xiang et al. 2022 ; Xia et al. 2024 ) . However, pre-defined, static relationships (e.g., coarse industry or shareholdings) quickly become outdated in dynamic markets and can be noisy indicators of the true interaction structure. Consequently, recent studies ( Niu et al. 2024 ; Niu et al. 2025 ) construct time-varying relationship graphs from frequently updated news, rather than relying on pre-defined graphs. In practice, news streams are disseminated at near-real-time intervals, making reliable relation extraction at scale a key challenge. Language models have transformed text understanding and information extraction, enabling the identification of relationships between financial entities and the induction of relational graphs from news articles ( Chen et al. 2023 ) . However, even with structured output mechanisms, prompting-based pipelines still rely on autoregressive decoding. Not only does that result in non-trivial latency and cost, but graph construction is also typically treated as a separate stage, decoupled from downstream optimization. For downstream financial prediction, optimizing a language model for either fluency or factual accuracy is insufficient; the predictor ultimately requires structured objects (e.g., graphs or adjacency matrices), the construction of which should be aligned with the end task. Moreover, instruction-following benchmarks suggest that under multi-level or hierarchical constraints, LLMs remain fragile, often violating required formats or priorities. These limitations motivate the induction of relational structure directly from language-model representations and its training jointly with the downstream predictor. Linear probing then fine-tuning (LP \rightarrow FT) is advocated as a low-cost approach for learning readout heads that align general hidden states with task-specific spaces, even when the backbone is frozen or only lightly updated ( Howard & Ruder 2018 ; Tomihari & Sato 2024 ) . Linear probing (LP) first learns a near-optimal linear readout, after which fine-tuning updates features with limited, controlled drift, preserving pre-trained structure while aligning to the task. These findings motivate a shift from unstable prompt formats ( Zhao et al. 2021 ) to stable, trainable adapters that “pull” representations into the target space. We propose Relational Probing , which replaces the standard language-modeling head with a relation head. The relation head generates a relational graph directly from language-model hidden states and is trained jointly with the downstream model for stock-trend prediction. This approach both learns semantic representations and preserves the strict structure of the induced relational graph. It avoids the error-prone process of prompt \rightarrow parsing structured text \rightarrow post-hoc graph and eliminates the runtime and manual effort associated with constrained decoding. Additionally, to improve reproducibility and portability, we present an operational definition of small language models (SLMs) for end-to-end settings. We define SLMs as models that can be jointly fine-tuned with the relation head (RH) and the downstream task model on a single 24 GB GPU under specified batch-size and sequence-length budgets. Our key contributions to the field include: • We generalize linear probing to relational probing, a learnable relation head converts language-model hidden states into relational graphs, aligning outputs to the target structure without constrained decoding. • We jointly train the relation head to produce task-aligned, graph-structured objectives that transfer more effectively to downstream tasks. • We provide an operational definition of SLMs (single-GPU, fixed-budget) and benchmark Qwen3-0.6B/1.7B/4B within a single family for controlled comparisons. 2 Related Work 2.1 LLMs for Relation Extraction Language models (LMs) can read news articles and identify relational signals (e.g., entities, event triggers, and argument roles) with minimal or no task-specific training ( Zhou et al. 2024 ; Wei et al. 2023 ) . For relation extraction (RE), zero- or few-shot settings can be addressed via definition-conditioned prompting ( Zhou et al. 2024 ) and instruction-tuned formulations ( Wang et al. 2023 ) . At the same time, an evaluation ( Hamad et al. 2024 ) reports that financial RE performance remains below that of fully supervised state-of-the-art systems, yet is still practically useful. Furthermore, recent studies show that relation extraction techniques based on language models achieve consistent improvements via prompting ( Wadhwa et al. 2023 ; Li et al. 2023 ) , linear probing ( Alt et al. 2020 ) , and instruction tuning ( Jiao et al. 2023 ; Xu et al. 2024 ) . 2.2 Probing and Parameter-Efficient Adaptation Linear probing freezes the LLM and trains a lightweight head on intermediate states to predict task-specific targets ( Tomihari & Sato 2024 ) . As the head operates on continuous hidden states rather than the tokenizer/decoder, it can produce the diverse outputs required by downstream modules. Low-Rank Adaptation (LoRA) ( Hu et al. 2022 ) modifies the generator itself and remains coupled to token-level decoding. Even ignoring time and memory costs, LoRA outputs are still strings (typically JSON) that must be parsed and validated before any GNN can use them. In contrast, linear-probing outputs are ready for downstream use and can be integrated into joint training with the GAT for the FTP task. Linear probing provides the flexibility and calibration needed to match the exact formats required by downstream tasks ( Tomihari & Sato 2024 ; Niu et al. 2023 ) . These capabilities are beyond the scope of LoRA, which remains within the LLM text channel. While a single linear layer is sufficient in many cases, the recent trend has been toward richer, yet still lightweight, heads when structure plays a key role. Examples include MLP heads for nonlinearity ( Eberts & Ulges 2019 ) and biaffine/bilinear scorers for pairwise relations (entity–entity and token–token) ( Dozat & Manning 2016 ; Verga et al. 2018 ) . These designs retain the core advantages of

Read the complete text