Harmonizing Metadata of Language Resources for Enhanced Querying and Accessibility

Zixuan Liang · 2025

Read Harmonizing Metadata of Language Resources for Enhanced Querying and Accessibility by Zixuan Liang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-05606.txt.

Introduction

Zixuan Liang tackles the messy metadata that describes language resources such as corpora and lexicons, in work from 2025. Records scattered across repositories use clashing formats, which makes the resources hard to find and query.

Using linked-data methods, Liang merges records from several sources into one model built on standard catalog and language-resource vocabularies. The result powers Linghub, a portal offering text search, faceted browsing, and structured queries. Real questions from a corpus mailing list were used to test the portal: many could be answered, though gaps remain. The study urges open vocabularies and standards, plus machine-readable access for automated use.

Catalog record

CREATORZixuan Liang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-05606

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Harmonizing Metadata of Language Resources for Enhanced Querying and Accessibility Zixuan Liang Abstract This paper addresses the harmonization of metadata from diverse repositories of language resources (LRs). Leveraging linked data and RDF techniques, we integrate data from multiple sources into a unified model based on DCAT and META-SHARE OWL ontology. Our methodology supports text-based search, faceted browsing, and advanced SPARQL queries through Linghub, a newly developed portal. Real user queries from the Corpora Mailing List (CML) were evaluated to assess Linghub’s capability to satisfy actual user needs. Results indicate that while some limitations persist, many user requests can be successfully addressed. The study highlights significant metadata issues and advocates for adherence to open vocabularies and standards to enhance metadata harmonization. This initial research underscores the importance of API-based access to LRs, promoting machine usability and data subset extraction for specific purposes, paving the way for more efficient and standardized LR utilization. I Introduction The study of language and the development of Natural Language Processing (NLP) applications rely on language resources (LRs). Recent initiatives, such as META-SHARE [ federmann2012meta ] , provide digital repositories that index metadata for LRs, supporting their discovery and reuse. META-SHARE offers an open, interoperable infrastructure where LRs can be documented, uploaded, stored, cataloged, exchanged, and discussed. It includes a metadata schema implemented as an XML Schema Definition (XSD), allowing descriptions of LRs throughout their lifecycle. Other repositories, like ELRA 1 1 1 http://www.elra.info , LDC 2 2 2 https://www.ldc.upenn.edu/ , CLARIN VLO [ van2012semantic ] , the Language Grid [ ishida2006language ] , and Alveo 3 3 3 http://alveo.edu.au/ , also provide access to LRs, with varying metadata schemes. However, these sources do not offer integrated and uniform querying. Linked Data offers a solution by linking datasets conceptually, facilitating access and reuse. Applying Linked Data technologies to LRs could create an interconnected ecosystem of resources [ chiarcos2013towards ] . Building on this concept, Linghub integrates metadata from various repositories, harmonizing them to offer a unified entry point for discovering language resources across platforms. Thus, in this paper, we are concerned with how to facilitate the discovery of language resources for a particular task. Given the fact that metadata records for resources are distributed among different catalogs and repositories makes the task of finding a particular resource meeting certain requirements very challenging. This is shown by the fact that many emails to dedicated mailing lists such as the Corpora mailing list contain requests for resources meeting certain desiderata with respect to type of resource (corpus, dictionary, parallel text, etc.), language, size (in tokens or sentences), etc. So far, no repository that allows discovery of resources across repositories has been available. We have closed this gap by developing Linghub, a linked data based portal that indexes and aggregates metadata entries from different repositories including META-SHARE, the CLARIN VLO, LRE-Map and Datahub.io. Parts of the Linghub as a technical system has been described before [ mccrae2015reconciling , mccrae2015linghub ] . However, this manuscript is the only comprehensive description of the principles behind Linghub, a detailed description of the harmonization procedures and this paper includes results of a novel evaluation. Linghub not only indexes the metadata entries, but also harmonizes the information by mapping it to standard semantic web vocabularies as well as to a recently created ontology of language resources 4 4 4 http://purl.org/net/def/metashare that has been developed on the basis of the existing META-SHARE schema [ mccrae2015ontology ] . For this, it relies on state-of-the-art word sense disambiguation methods to support the normalization of data. One of the crucial technologies that enables this integration is that of RDF [ klyne2006resource ] and linked data. Linked data is based on four fundamental princi- pals [ bizer2009linked ] . 1. Use Uniform Resource Identifiers (URIs) to identify everything in a resource, thus ensuring that every element of the resource can be identified in a standard manner. 2. Furthermore, use HTTP URIs as they require the association with domain names, ensuring that the data can be clearly traced to its host and thus someone responsible for that dataset. 3. Ensure that URIs resolve, in the sense that when typed into a web browser an appropriate description of the resource is obtained. Ideally the server should detect (using content negotiation) the type of the user and provide HTML results for humans and RDF (serialised in XML or JSON, for instance) for software agents. 4. Provide links to other resources so that it is possible to identify commonalities between resources and to handle issues of semantic interoperability and provenance. Linked data makes a highly appropriate model for the task of integrating information about language resources as a Web portal allows use to define stables URIs. we harmonize data by attempting to map metadata descriptions from different sources into the same data schema and enforce the use of the same URIs for equivalent metadata elements as well as values. HTML descriptions are provided at the URIs and in order to meet use cases for automatic training of NLP systems, we find the provision of a machine-readable API also of vital importance. Finally, links to other resources are vital to not only provide links back to the source records, but also to ensure that users can find resources for their needs. In this paper, besides describing Linghub and the semantic normalization methods used, we provide an evaluation of the ability of Linghub to answer the needs of actual users seeking language resources meeting certain criteria. For this, we have analyzed user requests for resources issued on the Corpora List and analyzed in how far Linghub is able to answer them. This evaluation is the main contribution of the current paper. We are not aware of any similar evaluation conducted in the context of repositories of language resources, so that to our knowledge this is the first attempt to evaluate the ability of a repository to answer requests for language resources. This paper is structured as follows: Firstly, in Section II we will discuss some of the existing related work in particular focussing on the metadata repositories that we will integrate and in Section III we will describe how we collected the data. In Section IV we derive a single data model based on existing standards that will allow us to combine all the resources and then we will show our procedure for harmonizing these resources. In Section V , we will describe the data portal and provide a thorough evaluation of the system based on real-world queries and thus show the effectiveness of our approach, and finally we conclude in Section VI . 5 5 5 Some results in this paper have been previously published in the following workshop papers [ mccrae2015linghub , mccrae2015reconciling , mccrae2015ontology ] . The results and methods are expanded and combined in this paper in line with the journal guidelines. In addition, Section V provides a novel evaluation of the system. II Related Work Harmonization of data is an important challenge within many application domains. In particular, the integration of data or metadata from different origins is a major challenge, typically requiring a lot of effort. Thus, there is a key interest in developing methods that can support the automatization

Read the complete text