Knowledge Graph Completion by Intermediate Variables Regularization
Changyi Xiao, Yixin Cao · 2025
Read Knowledge Graph Completion by Intermediate Variables Regularization by Changyi Xiao, Yixin Cao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02749.txt.
Introduction
Changyi Xiao and Yixin Cao improve knowledge graph completion in a twenty twenty-five paper. A knowledge graph stores facts as triples, like subject, relation, object. Completion fills in missing facts, and tensor models, which treat the graph as a three-way array, do this well but tend to overfit.
Past fixes just shrink the embedding vectors, which helps little. The pair instead shrinks the intermediate variables that arise in the several ways of computing the predicted tensor. They prove this pushes the tensor toward low trace norm, a known guard against overfitting. The fix fits most tensor models and stays cheap to compute. Tests confirm the gains, and code is shared.
Catalog record
| CREATOR | Changyi Xiao, Yixin Cao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-02749
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Knowledge Graph Completion by Intermediate Variables Regularization Changyi Xiao, Yixin Cao Abstract Knowledge graph completion (KGC) can be framed as a 3-order binary tensor completion task. Tensor decomposition-based (TDB) models have demonstrated strong performance in KGC. In this paper, we provide a summary of existing TDB models and derive a general form for them, serving as a foundation for further exploration of TDB models. Despite the expressiveness of TDB models, they are prone to overfitting. Existing regularization methods merely minimize the norms of embeddings to regularize the model, leading to suboptimal performance. Therefore, we propose a novel regularization method for TDB models that addresses this limitation. The regularization is applicable to most TDB models and ensures tractable computation. Our method minimizes the norms of intermediate variables involved in the different ways of computing the predicted tensor. To support our regularization method, we provide a theoretical analysis that proves its effect in promoting low trace norm of the predicted tensor to reduce overfitting. Finally, we conduct experiments to verify the effectiveness of our regularization technique as well as the reliability of our theoretical analysis. The code is available at https://github.com/changyi7231/IVR . 1 Introduction A knowledge graph (KG) can be represented as a 3rd-order binary tensor, in which each entry corresponds to a triplet of the form (head\ entity,relation,tail\ entity) . A value of 1 denotes a known true triplet, while 0 denotes a false triplet. Despite containing a large number of known triplets, KGs are often incomplete, with many triplets missing. Consequently, the 3rd-order tensors representing the KGs are incomplete. The objective of knowledge graph completion (KGC) is to infer the true or false values of the missing triplets based on the known ones, i.e., to predict which of the missing entries in the tensor are 1 or 0. A number of models have been proposed for KGC, which can be classified into translation-based models, tensor decomposition-based (TDB) models and neural networks models ( Zhang et al. 2021 ) . We only focus on TDB models in this paper due to their wide applicability and great performance ( Lacroix et al. 2018 ; Zhang et al. 2020 ) . TDB models can be broadly categorized into two groups: CANDECOMP/PARAFAC (CP) decomposition-based models, including CP ( Lacroix et al. 2018 ) , DistMult ( Yang et al. 2014 ) and ComplEx ( Trouillon et al. 2017 ) , and Tucker decomposition-based models, including SimplE ( Kazemi and Poole 2018 ) , ANALOGY ( Liu et al. 2017 ) , QuatE ( Zhang et al. 2019 ) and TuckER ( Balažević et al. 2019 ) . To provide a thorough understanding of TDB models, we present a summary of existing models and derive a general form that unifies them, which provides a fundamental basis for further exploration of TDB models. TDB models have been proven to be theoretically fully expressive ( Trouillon et al. 2017 ; Kazemi and Poole 2018 ; Balažević et al. 2019 ) , implying they can represent any real-valued tensor. However, in practice, TDB models frequently fall prey to severe overfitting. To counteract this issue, various regularization techniques have been employed in KGC. One commonly used technique is the squared Frobenius norm regularization ( Nickel et al. 2011 ; Yang et al. 2014 ; Trouillon et al. 2017 ) . Lacroix et al. 2018 proposed another regularization, N3 norm regularization, based on the tensor nuclear p -norm, which outperforms the squared Frobenius norm in terms of performance. Additionally, Zhang et al. 2020 introduced DURA, a regularization technique based on the duality of TDB models and distance-based models that results in significant improvements on benchmark datasets. Nevertheless, N3 and DURA rely on CP decomposition, limiting their applicability to CP ( Lacroix et al. 2018 ) and ComplEx ( Trouillon et al. 2017 ) . As a result, there is a pressing need for a regularization technique that is widely applicable and can effectively alleviate the overfitting issue. In this paper, we introduce a novel regularization method for KGC to improve the performance of TDB models. Our regularization focuses on preventing overfitting while maintaining the expressiveness of TDB models as much as possible. It is applicable to most TDB models, while also ensuring tractable computation. Existing regularization methods for KGC rely on minimizing the norms of embeddings to regularize the model ( Yang et al. 2014 ; Lacroix et al. 2018 ) , leading to suboptimal performance. To achieve superior performance, we present an intermediate variables regularization (IVR) approach that minimizes the norms of intermediate variables involved in the processes of computing the predicted tensor of TDB models. Additionally, our approach fully considers the computing ways because different ways of computing the predicted tensor may generate different intermediate variables. To support the efficacy of our regularization approach, we further provide a theoretical analysis. We prove that our regularization is an upper bound of the overlapped trace norm ( Tomioka et al. 2011 ) . The overlapped trace norm is the sum of the trace norms of the unfolding matrices along each mode of a tensor ( Kolda and Bader 2009 ) , which can be considered as a surrogate measure of the rank of a tensor. Thus, the overlapped trace norm reflects the correlation among entities and relations, which can pose a constraint to jointly entities and relations embeddings learning. In specific, entities and relations in KGs are usually highly correlated. For example, some relations are mutual inverse relations, or a relation may be a composition of another two relations ( Zhang et al. 2021 ) . Through minimizing the upper bound of the overlapped trace norm, we encourage a high correlation among entities and relations, which brings strong regularization and alleviates the overfitting problem. The main contributions of this paper are listed below: 1. We present a detailed overview of a wide range of TDB models and establish a general form to serve as a foundation for further TDB model analysis. 2. We introduce a new regularization approach for TDB models based on the general form to mitigate the overfitting issue, which is notable for its generality and effectiveness. 3. We provide a theoretical proof of the efficacy of our regularization and validate its practical utility through experiments. 2 Related Work Tensor Decomposition Based Models Research in KGC has been vast, with TDB models garnering attention due to their superior performance. The two primary TDB approaches that have been extensively studied are CP decomposition ( Hitchcock 1927 ) and Tucker decomposition ( Tucker 1966 ) . CP decomposition represents a tensor as a sum of n rank-one tensors, while Tucker decomposition decomposes a tensor into a core tensor and a set of matrices. Kolda and Bader 2009 shows more details about CP decomposition and Tucker decomposition. Several techniques have been developed for applying CP decomposition and Tucker decomposition in KGC. For instance, Lacroix et al. 2018 employed the original CP decomposition, whereas DistMult ( Yang et al. 2014 ) , a variant of CP decomposition, made the embedding matrices of head entities and tail entities identical to simplify the model. SimplE ( Kazemi and Poole 2018 ) tackled the problem of independence among the embeddings of head and tail entities