Markov Process-Based Graph Convolutional Networks for Entity Classification in Knowledge Graphs
Johannes Mäkelburg 1∗ ,Yiwen Peng 2∗ , Mehwish Alam 2 , Tobias Weller 3 and Maribel Acosta 1 1 School of CIT, Technical University of Munich, Germany 2 Télécom Paris, Institut Polytechnique de Paris, France 3 Independent Researcher, Germany { johannes.maekelburg, maribel.acosta } @tum.de, { yiwen.pe · 2024
Read Markov Process-Based Graph Convolutional Networks for Entity Classification in Knowledge Graphs by Johannes Mäkelburg 1∗ ,Yiwen Peng 2∗ , Mehwish Alam 2 , Tobias Weller 3 and Maribel Acosta 1 1 School of CIT, Technical University of Munich, Germany 2 Télécom Paris, Institut Polytechnique de Paris, France 3 Independent Researcher, Germany { johannes.maekelburg, maribel.acosta } @tum.de, { yiwen.pe (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-17438.txt.
Introduction
Johannes Mäkelburg and co-authors add a Markov step counter to graph networks in a 2024 paper. Knowledge graphs store vast facts, but the class of many entities stays blank. Graph convolutional networks can guess those classes, yet they treat each guess as a flat one shot task.
Their build folds the guess into a Markov process whose step count is learned during training from a geometric spread. The loss also draws on evidential learning to weigh doubt. Across several network forms and data sets, the Markov form beat plain rivals. The step spread can be tuned to trade speed for accuracy in training.
Catalog record
| CREATOR | Johannes Mäkelburg 1∗ ,Yiwen Peng 2∗ , Mehwish Alam 2 , Tobias Weller 3 and Maribel Acosta 1 1 School of CIT, Technical University of Munich, Germany 2 Télécom Paris, Institut Polytechnique de Paris, France 3 Independent Researcher, Germany { johannes.maekelburg, maribel.acosta } @tum.de, { yiwen.pe |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-17438
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Markov Process-Based Graph Convolutional Networks for Entity Classification in Knowledge Graphs Johannes Mäkelburg 1∗ ,Yiwen Peng 2∗ , Mehwish Alam 2 , Tobias Weller 3 and Maribel Acosta 1 1 School of CIT, Technical University of Munich, Germany 2 Télécom Paris, Institut Polytechnique de Paris, France 3 Independent Researcher, Germany { johannes.maekelburg, maribel.acosta } @tum.de, { yiwen.peng, mehwish.alam } @telecom-paris.fr Abstract Despite the vast amount of information encoded in Knowledge Graphs (KGs), information about the class affiliation of entities remains often incomplete. Graph Convolutional Networks (GCNs) have been shown to be effective predictors of complete information about the class affiliation of entities in KGs. However, these models do not learn the class affiliation of entities in KGs incorporating the complexity of the task, which negatively affects the models’ prediction capabilities. To address this problem, we introduce a Markov process-based architecture into well-known GCN architectures. This end-to-end network learns the prediction of class affiliation of entities in KGs within a Markov process. The number of computational steps is learned during training using a geometric distribution. At the same time, the loss function combines insights from the field of evidential learning. The experiments show a performance improvement over existing models in several studied architectures and datasets. Based on the chosen hyperparameters for the geometric distribution, the expected number of computation steps can be adjusted to improve efficiency and accuracy during training. * * footnotetext: These authors contributed equally to this work. 1 Introduction Knowledge Graphs (KGs) encode factual knowledge in the form of triples (subject-relation-object) and have emerged as a compelling abstraction for organizing semi-structured data, capturing relationships among entities. The facts available in KGs are used in many application areas such as recommendation [ Wang et al., 2019 ] , information retrieval [ Xiong and Callan, 2015 ] , and question answering [ Yasunaga et al., 2021 ] for improving the performance of these systems by providing background or auxiliary information. This relevance imposes a significant importance on a KG providing comprehensive information about the encoded entities. In particular, encoding knowledge about the class affiliation of entities is of great importance for automatic reasoning and inferencing of information contained in a KG. Despite the enormous effort made to keep the knowledge encoded in the KGs up-to-date and consistent [ Heist et al., 2020 ] , KGs are often incomplete majorly due to automated constructions of KGs. To complete missing knowledge, in particular missing class affiliation of entities in KGs, various methods based on machine learning have been introduced [ Bordes et al., 2013a , Kipf and Welling, 2016 ] . Neural networks, especially Graph Convolutional Neural Networks (GCNs), have proven to be very effective in completing class affiliation of entities in KGs [ Schlichtkrull et al., 2018 ] . However, in neural network based methods, the computation cost grows with the size of the input data, but not with the complexity of the problem being learned. In recent developments in automated machine learning, models perform conditional computation based on probabilistic variables that are used to dynamically adjust the number of computation steps [ Banino et al., 2021 ] . The adjustment of the computational budget, in particular the computation steps, is known as pondering. Yet, existing graph-based machine learning algorithms do not consider the complexity of the task to adjust the number of computation steps to learn the parameters. We address this issue and introduce a GCN-based model that learns to adapt the amount of computational steps based on the task at hand. In this work, we propose Markov Process and Evidential with Regularization Loss (MPERL) that builds upon the previous idea of dynamically adjusting the number of computational steps based on the input of the model. We introduce an end-to-end Graph Convolutional Network (GCN) model that is learned within a Markov process and use recent developments in the field of evidential learning [ Sensoy et al., 2018 , Amini et al., 2020 ] . Previous work has demonstrated the high performance of evidence-based models. Unlike models using a softmax function, evidence-based models are effective predictors that do not make overconfident predictions. We therefore follow an evidence-based approach as well. The Markov process in which the model is learned consists of two states: (i) the continue state, indicating further computational steps, and (ii) the halt state, indicating the end of the computational steps. The overall probability of halting at each step is modeled as a geometric distribution. Unlike previous work for entity classification in KGs, MPERL is a graph-based model that dynamically adjusts the number of computational steps according to complexity. MPERL further shows its feasibility on the task of entity type prediction, i.e., inferring the knowledge about the class affiliation of an entity. In the rest of this paper, we are treating entity type prediction as a classification task where we perform single-label classification for smaller datasets (specifically designed for this purpose) and multi-label classification on larger datasets. The experimental results show that MPERL (GCN with markov process and evidential loss) outperforms vanilla GCN as well as various other baselines. The ablation studies show that the use of both markov process and the evidential learning loss provide significant increase in the performance of the MPERL. Overall our paper makes the following contributions: \bullet We introduce a Graph Convolutional Network based model trained within a Markov process, using an evidential loss function. \bullet We demonstrate the performance of the model in predicting missing class affiliations of entities using single- and multi-label classification. \bullet We show the effect on the model when adapting the number of computational or Markov steps. \bullet We show the effectiveness of each of the components of the model on the overall results with the help of an ablation study. The paper is structured as follows: Section 2 discusses the recent works related to representation learning over KGs for entity classification. Section 3 details the proposed approach while Section 4 shows the effectiveness of MPERL with the help of thorough experimentation over various sizes of the datasets as well as the ablation study. Finally, Section 5 concludes the study and discusses future directions. 2 Related Work Different learning approaches have been applied to the problem of entity classification in KGs [ Alam et al., 2024 ] . Relational Graph Convolutional Networks (R-GCN) [ Schlichtkrull et al., 2018 ] uses the structure of KGs to generate embeddings based on local neighbors in order to predict classes [ Kipf and Welling, 2016 , Hamilton et al., 2017 ] . Due to their strong performance on graph-structured data, GCN models have been particularly used and extended in recent years to tackle entity classification [ Schlichtkrull et al., 2018 , Chen et al., 2019 , Zangari et al., 2021 ] , relation classification [ Long et al., 2021 ] , and KG alignment [ Berrendorf et al., 2020 , Wang et al., 2018 ] . Gated Relational Graph Neural Network (GRGNN) [ Chen et al., 2019 ] introduced a gate mechanism to leverage hidden states of current node and its neighbors to target entity classification problem in KGs. Whereas Relational Graph Attention Networks [ Busbridge et al., 2019 ] and Multilayer Graph Attention Network [ Zangari et al., 2021 ] use masked