Heterogeneous Graph Pre-training Based Model for Secure and Efficient Prediction of Default Risk Propagation among Bond Issuers
Xurui Li ⋆ † {\dagger}, Xin Shan ⋆, Wenhao Yin, Haijiao Wang · 2025
Read Heterogeneous Graph Pre-training Based Model for Secure and Efficient Prediction of Default Risk Propagation among Bond Issuers by Xurui Li ⋆ † {\dagger}, Xin Shan ⋆, Wenhao Yin, Haijiao Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-03268.txt.
Introduction
Xurui Li, Xin Shan, Wenhao Yin, and Haijiao Wang predict how default risk spreads among companies that issue bonds, in a paper published in 2025. One firm's failure can strain its partners, so judging a firm alone misses the web it sits in. Graph methods read that web, but firms guard their data, and labels for training are scarce.
Their model trains in two stages. First, a masked autoencoder learns from a huge company knowledge graph without needing labels, hiding and rebuilding parts of it. Then a smaller classifier, fed both the learned features and each firm's own traits, estimates default risk spread. The design uses shared structure without pooling raw private data. Tests support the approach for bond market watchers who must flag risky issuers early.
Catalog record
| CREATOR | Xurui Li ⋆ † {\dagger}, Xin Shan ⋆, Wenhao Yin, Haijiao Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-03268
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Heterogeneous Graph Pre-training Based Model for Secure and Efficient Prediction of Default Risk Propagation among Bond Issuers Thanks: ⋆ Equal contribution. † Corresponding author. PubID: pubid: Accepted in NDSS Workshop 2024. Xurui Li ⋆ † {\dagger}, Xin Shan ⋆, Wenhao Yin, Haijiao Wang Abstract Efficient prediction of default risk for bond-issuing enterprises is pivotal for maintaining stability and fostering growth in the bond market. Conventional methods usually rely solely on an enterprise’s internal data for risk assessment. In contrast, graph-based techniques leverage interconnected corporate information to enhance default risk identification for targeted bond issuers. Traditional graph techniques such as label propagation algorithm or deepwalk fail to effectively integrate a enterprise’s inherent attribute information with its topological network data. Additionally, due to data scarcity and security privacy concerns between enterprises, end-to-end graph neural network (GNN) algorithms may struggle in delivering satisfactory performance for target tasks. To address these challenges, we present a novel two-stage model. In the first stage, we employ an innovative Masked Autoencoders for Heterogeneous Graph (HGMAE) to pre-train on a vast enterprise knowledge graph. Subsequently, in the second stage, a specialized classifier model is trained to predict default risk propagation probabilities. The classifier leverages concatenated feature vectors derived from the pre-trained encoder with the enterprise’s task-specific feature vectors. Through the two-stage training approach, our model not only boosts the importance of unique bond characteristics for specific default prediction tasks, but also securely and efficiently leverage the global information pre-trained from other enterprises. Experimental results demonstrate that our proposed model outperforms existing approaches in predicting default risk for bond issuers. I Introduction Bond issuer default risk, a critical aspect of financial analysis, refers to the likelihood that the enterprise which issues a bond will become unable to fulfill its financial obligations, leading to a default [ 1 ] . To make informed investment decisions and maintain market stability, it is crucial to comprehend and assess this risk. When assessing default risk for a bond issuer, traditional methods usually utilize its own operational data for prediction. However, real-world risk encompasses various factors such as the issuer’s financial health, market conditions, industry-specific risks, and overall economic stability, etc. There are often intricate interconnections among enterprises, leading to the spread of risks across the bond issuer landscape [ 2 ] . To effectively utilize information from associate enterprises to assist in assessing default risk for a target enterprise, a common approach is to employ graph-based risk propagation methods such as label propagation algorithm. However, these algorithms merely propagate known risk labels across the graph without effectively incorporating the inherent characteristics of each enterprise into the comprehensive prediction. To address this issue, one solution is to leverage graph embedding information besides the enterprises’ own features when predicting risk propagation probabilities. While this approach merges the enterprises’ inherent traits with their relational topology to some extent, it awkwardly combines these two types of information without fully capturing the dynamic propagation and evolution effects of the enterprises’ features upon the entire corporate network structure. The recently popular graph neural network (GNN) methods such as GAT or Graphsage can well learn the diffusion pattern of node features in the graph [ 3 ] . However, there are several problems with directly using end-to-end GNN algorithms for our scenario. On the one hand, the performance of existing GNNs significantly suffers when distribution shifts occur between training and testing data, or when data scarcity is present. Bond issuers, typically large or even listed enterprises, possess a wealth of business information. However, other non-bond enterprises in the massive graph often lack such detailed features. GNN algorithms struggle to learn effective information from this kind of graph for training specific risk propagation tasks, because they cannot fully leverage the differences between bond issuers and other enterprises. On the other hand, in terms of security and privacy, it is not advisable for some information providers to disclose detailed operational information of related enterprises to business parties. In this paper, we present a two-stage framework that utilizes graph pre-training techniques to predict the propagation probability of default risks among bond issuers. Initially, our model is pre-trained on a comprehensive enterprise knowledge graph (EKG). Subsequently, the embedding from the pre-trained encoder is combined with the unique information of the bond-issuing enterprise to train the default risk prediction model. The main contributions are summarized as follows: \bullet The proposed two-stage method can not only safely and effectively utilize the encrypted information from the massive EKG, but also fully leverage the abundant features of bond-issuing enterprises for risk prediction tasks. \bullet We also introduce a novel Masked Autoencoders for Heterogeneous Graph (HGMAE) model to learn a better representation during the pre-training of heterogeneous graphs. \bullet Extensive experiments show that our method outperforms the state-of-the-art baselines in the risk dataset, indicating the effectiveness of the two-stage framework, as well as the HGMAE model. Fig. 1: The main framework of our two-stage risk propagation prediction method. The right part shows the details for the proposed HGMAE model. Different Colors for HGMAE indicate different edge types. II Related Works Pre-trained Graph Embeddings . The first-generation pre-trained graph models aim to produce effective graph embeddings for diverse tasks [ 4 ] . DeepWalk, a pioneer in this field, introduced the concept of graph embedding by treating the paths traversed by random walks over graphs as sentences. It utilizes skip-gram to learn latent node representations. Following in DeepWalk’s footsteps, Node2vec developed a flexible approach to define a node’s network neighborhood and designed a biased random walk procedure to efficiently explore diverse neighborhoods. Furthermore, several researchers have also attempted to learn embeddings for heterogeneous graphs, sub-graphs, and molecular graphs, such as sub2vec [ 5 ] , sub-graph2vec, etc. Pre-trained Graph Encoders . With the emergence of expressive GNNs and Transformer, recent methods have embraced a transfer learning setting where the goal is to pre-train a generic encoder that can deal with different tasks. Compared to the pre-trained graph embedding methods, pre-trained graph encoders can provide a better model initialization, which usually leads to a better generalization performance and speeds up convergence on the target tasks. In addition, the modern models are usually trained with larger scale database, more powerful or deeper architectures, and new pre-training tasks [ 4 , 6 ] . Their architectures broadly fall into two categories: graph neural networks (e.g., GIN, HAN) [ 7 ] and hybrid of GNNs and Transformer (e.g., MPG, HGT) [ 8 ] . Their pre-training strategies can be widely divided into supervised and unsupervised ones. Although the supervised pre-training brings remarkable improvements, they often require domain-specific knowledge which significantly limits their wider applications. More importantly, some supervised pre-training tasks might be unrelated to the downstream task of interest and can even hurt the downstream performance. As for unsupervised graph pre-training, it can be mainly divided into four categories: 1) Graph autoencoders: typical models such as GAE, VGAE, SIGVAE use self-supervised graph reconstruction for learning discriminative representations [ 9 ] . 2) Graph autoregressive modeling: typical models such as GPT-GNN, MGSSL perform the autoregressive reconstruction on given graphs iteratively instead of reconstruct the graph all at once [ 10 ] . 3) Masked components modeling: typical model such as GROVER [ 11 ] masks