S2Doc - Spatial-Semantic Document Format

2025

Read S2Doc - Spatial-Semantic Document Format (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01113.txt.

Introduction

Sebastian Kempf and Frank Puppe propose S2Doc, a data format for documents and tables that keeps spatial and semantic structure together. Research on documents has no shared model: each project invents its own structure, formats clash, and most models capture either where things sit on the page or what they mean, not both.

S2Doc is designed to be flexible and extendable to new tasks, to support most existing modeling approaches for documents and tables, and to handle multi page documents. The authors describe it as, to their knowledge, the first format to combine all of these aspects in one structure. For document processing research, a common format could end the repeated cost of translating between incompatible representations.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-01113

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

S2Doc - Spatial-Semantic Document Format Abstract Documents are a common way to store and share information, with tables being an important part of many documents. However, there is no real common understanding of how to model documents and tables in particular. Because of this lack of standardization, most scientific approaches have their own way of modeling documents and tables, leading to a variety of different data structures and formats that are not directly compatible. Furthermore, most data models focus on either the spatial or the semantic structure of a document, neglecting the other aspect. To address this, we developed S2Doc , a flexible data structure for modeling documents and tables that combines both spatial and semantic information in a single format. It is designed to be easily extendable to new tasks and supports most modeling approaches for documents and tables, including multi-page documents. To the best of our knowledge, it is the first approach of its kind to combine all these aspects in a single format. Keywords: table understanding, data format, semantic annotation S2Doc - Spatial-Semantic Document Format Sebastian Kempf, Frank Puppe University of Würzburg Center for Artificial Intelligence and Data Science (CAIDAS) Emil-Fischer-Straße 50 97074 Würzburg, Germany {sebastian.kempf, frank.puppe}@informatik.uni-wuerzburg.de Abstract content 1. Introduction Extracting and understanding documents has always been a common task in the domain of artificial intelligence. From OCR and text recognition to document layout analysis and information extraction, many different tasks can be performed on documents. With the rise of large language models, interest in document understanding has increased, since these models benefit greatly from additional context and structure provided by documents. Tables are of particular interest because they contain a large amount of structured information. However, tables are designed for human consumption, so automatic processing and understanding of tables is challenging. While tables and documents are subject to active research, there is no real common understanding of how to model documents and tables in particular. Usually, every approach has its own way of modeling the documents and tables, leading to a variety of different data structures and formats that are not compatible with each other. While some areas have established some kind of standard, like the COCO format for image segmentation, these are task-specific and cannot be easily adapted to other tasks. This lack of standardization causes several problems. Implementing different approaches, or combining them, is often difficult or impossible due to incompatible data structures or missing information. Creating datasets is also more challenging because there is no clear consensus on how to model a table or document. As a result, datasets are often limited to a specific task, so evaluating the performance of whole pipelines is difficult. In addition to these problems, most data models and datasets focus on either the spatial or the semantic structure of a document. However, both are important for a complete understanding of a document and should therefore be jointly considered, modeled, and evaluated. Furthermore, one often overlooked aspect is the multi-page nature of most real-world documents. For example, scientific articles often contain dozens of pages, with tables and figures spread across multiple pages, so a complete understanding of the document requires considering all pages. Most existing data structures focus on individual pages and neglect the overall document context. Because of that, we developed S2Doc 1 1 1 https://github.com/Raynswor/s2doc . It is a modular and flexible data structure for modeling documents and especially tables, combining both spatial and semantic information in a single format. It has been designed to be easily extendable to new tasks and comes with support for most of the modeling approaches of documents and tables as well as multi-page documents. It tries to establish a common ground for modeling documents and tables to make implementing, combining, and evaluating document and table understanding approaches easier. To the best of our knowledge, it is the first approach of its kind. 2. Related Work As previously mentioned, there is no real standard for modeling documents and tables in the scientific community. Most data structures are means to an end to solve a specific task and are implemented to fit these requirements. Because it is not feasible to cover all existing approaches, we will focus on existing document and table formats, focusing on their advantages and disadvantages with regards to the mentioned problems. A tabular comparison of the different approaches can be found in Table 1 . The categorization of existing document and table formats is not trivial, as there are many different ways to categorize them. Because the focus of this work is on practicality and usability, we will loosely categorize them based on their main use case and application. We identified the following categories: (1) OCR-centric, (2) human-centric, (3) data-centric, (4) table-centric, and (5) other. 2.1. OCR-centric These formats are primarily designed to represent the results of digitalization pipelines, including document types such as scans, images, and especially OCR output. Their focus is capturing the physical layout and textual content of documents and uses hierarchies of elements to represent the document structure. They are machine-readable and can be easily processed by algorithms, but focus on layout rather than semantic information. Prominent examples of this category are PAGE , ALTO , and hOCR . PAGE by Pletschacher and Antonacopoulos 2010 is a popular XML standard to encode digitised documents in the domain of document image analysis and OCR. It is designed to support the representation and evaluation of intermediate results of these tasks. In addition to the layout structure and page content, it also provides the possibility to encode image characteristics and processing steps. It has been used in competitions like the ICDAR Page Segmentation. Similarly, ALTO by Library of Congress 2020 is another XML Schema that focuses on layout preservation and content representation. Additionally, it captures structural elements such as margins, headings, and other layout features. It is widely used in libraries and archives for digitized documents, especially in conjunction with the METS format for storing and exchanging digital objects. hOCR by Breuel 2007 is an HTML microformat to represent intermediate and final OCR results. It offers several markups, allowing for modeling the logical structure of documents as well as typographical and layout information. Because it is based on HTML, it can be easily visualized and processed with corresponding tools. 2.2. Human-centric This category includes formats that are primarily designed for human consumption. They focus on the visual appearance of documents and sometimes the ability to edit them. Usually, they are not easily machine-readable and do not provide a clear structure for the document content (with the exception of HTML). They emphasize logical structure and rich text with control over layout. DocX and ODT are widely used formats for word processing documents based on XML. They are designed to be easily editable and provide multiple formatting options. However, any formatting information is only represented visually and not accessible in a structured way, making automatic analysis of these documents difficult and their use in document understanding pipelines limited. Documents in these formats are often exported as PDF documents for sharing and publishing, because PDF preserves the visual appearance across platforms and devices. However, PDF is exclusively focused on the visual representation of documents and does not provide a

Read the complete text