TFLOP: Table Structure Recognition Framework with Layout Pointer Mechanism
Minsoo Khang, Teakgyu Hong · 2025
Read TFLOP: Table Structure Recognition Framework with Layout Pointer Mechanism by Minsoo Khang, Teakgyu Hong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-11800.txt.
Introduction
Khang and Hong present TFLOP, a tool that reads the layout of tables in pictures, in a paper from twenty twenty-five. TFLOP turns a picture of a table into HTML, the code used for web pages. That step lets other tools search the table and reuse it. Older tools first guess where each block of text sits. They then try to match text to those blocks, and the match often goes wrong.
TFLOP skips that match step and points straight at each text block. One step finds the tags that mark the table layout and the text blocks that fit them, all at once. A span-aware training signal makes the pointing stronger on hard, dense tables. TFLOP scores best on the PubTabNet, FinTabNet, and SynthTabNet tests. It also does well on work files with watermarks and on files that are not in English.
Catalog record
| CREATOR | Minsoo Khang, Teakgyu Hong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-11800
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TFLOP: Table Structure Recognition Framework with Layout Pointer Mechanism Minsoo Khang, Teakgyu Hong Abstract Table Structure Recognition (TSR) is a task aimed at converting table images into a machine-readable format (e.g. HTML), to facilitate other applications such as information retrieval. Recent works tackle this problem by identifying the HTML tags and text regions, where the latter is used for text extraction from the table document. These works however, suffer from misalignment issues when mapping text into the identified text regions. In this paper, we introduce a new TSR framework, called TFLOP ( T SR F ramework with L ay O ut P ointer mechanism), which reformulates the conventional text region prediction and matching into a direct text region pointing problem. Specifically, TFLOP utilizes text region information to identify both the table’s structure tags and its aligned text regions, simultaneously. Without the need for region prediction and alignment, TFLOP circumvents the additional text region matching stage, which requires finely-calibrated post-processing. TFLOP also employs span-aware contrastive supervision to enhance the pointing mechanism in tables with complex structure. As a result, TFLOP achieves the state-of-the-art performance across multiple benchmarks such as PubTabNet, FinTabNet, and SynthTabNet. In our extensive experiments, TFLOP not only exhibits competitive performance but also shows promising results on industrial document TSR scenarios such as documents with watermarks or in non-English domain. Source code of our work is publicly available at: https://github.com/UpstageAI/TFLOP . 1 Introduction (a) Dual decoder framework (b) TFLOP Figure 1 : Overview of two TSR frameworks. The dual decoder identifies table cell regions and their HTML structure, requiring further cell and text region mapping for the final output. In contrast, TFLOP utilizes text region information and directly identifies the HTML structure with its corresponding text region relations. Tables are prevalent across a wide spectrum of documents (e.g. business documents, academic papers) for their compact and efficient representation. Such compact representation, however, presents a significant challenge for direct machine parsing. Table Structure Recognition (TSR) aims to digitize table images into machine-readable format (e.g. HTML) representing their structure and text, allowing for various downstream applications such as information retrieval or table QA. TSR often comprises of predicting two sets of structures before constructing the full table structure: logical and physical Huang et al. (2023) . Logical structure represents the semantic organization and relational information between table cells, often represented in the form of HTML or LaTeX. Physical structure, on the other hand, represents the layout information of table cells such as their bounding boxes. Recent works take on the image-to-text approach where both the logical and physical structures are predicted with the latter conditioning on the former. The physical structures (cell bounding boxes) are first mapped to text regions of the table, obtained using OCR engine or through PDF parsing, before combining the matched texts with the logical structure to form the full table. Despite its strong performance in logical structure prediction, these frameworks often suffer from misalignment issues where erroneous texts are matched due to imperfect alignment between the table text regions and the predicted cell bounding boxes. Such approaches require finely-calibrated post-processing during the matching of text regions for satisfactory results. This work, TFLOP, aims to eliminate the need for heuristic-based bounding box matching by leveraging table text regions directly in the framework through layout pointer mechanism. TFLOP reformulates the original bounding box prediction problem to a bounding box pointing problem. In particular, instead of predicting the cell bounding boxes conditioned on the logical structure, it predicts the associations between the bounding boxes and logical sequence through pointer mechanism. TFLOP’s pointer mechanism not only serves as a remedy to the misalignment issue but also eliminates the need for heuristics-based bounding box matching. On top of misalignment issues, recognizing structures of tables with row or column spans (i.e. complex tables) is one of the key challenges of TSR. Capitalizing on the flexibility of our framework, TFLOP employs span-aware contrastive supervision when processing the table text regions to improve its recognition of complex tables. Based on the proposed pointer mechanism and span-aware contrastive supervision, TFLOP achieves state-of-the-art performance across popular TSR benchmarks. In this work, we move beyond the benchmark datasets, and explore the versatility of TFLOP from the industrial perspective. We conduct extensive experiments and show that TFLOP not only has competitive performance but also the versatility in handling industrial document TSR scenarios such as watermarked documents or even non-English tables despite being trained exclusively on English tables. The key contributions of our work are as follows: • We propose a novel TSR framework with layout pointer mechanism which not only remedies the text region misalignment issue but also eliminates the necessity for post-processing when mapping text regions into the predicted cell bounding boxes. • We also present span-aware contrastive supervision in our framework. This supervision enhances the model’s ability in recognizing structures of complex tables involving row or column spans. • TFLOP achieves the state-of-the-art performance across multiple popular TSR benchmarks. • Beyond the benchmark performance, TFLOP has also shown competitive performance and versatility when dealing with industrial TSR document scenarios such as tables with watermark or in non-English domain. 2 Related Work TSR methods cover different variations of handling both logical and physical structure of tables. These methods can be largely categorised into two groups: detection-based and image-to-text methods. 2.1 Detection-based TSR Methods Detection-based TSR is one of the common approaches that recognizes the table structure by leveraging on the table features detected such as separation lines or cell-level features. These methods typically proceed with physical structure understanding first before reasoning with the corresponding logical structure for TSR. Grid-based approach represents methods which utilise the grid representation based on the detected table features. Earlier works Schreiber et al. (2017) ; Paliwal et al. (2019) detects row and column masks through segmentation-based methods before aggregating them to form the table structure. SPLERGE Tensmeyer et al. (2019) then proposed split-and-merge pipeline which first detects the grid structure matching the table before merging adjacent cells to handle spanning entries. Follow-up works improved on top of this grid representation such as TRUST Guo et al. (2022) which proposed query-based splitting and vertex-based merging modules to improve spanning cell prediction, while SEM Zhang et al. (2022) proposed aggregation of both visual and textual features in table grid generation. RobusTabNet Ma et al. (2023) proposed a spatial CNN module which improved physical structure reasoning when predicting separation lines prior to cell grid detection. Follow-up work TSRFormer Lin et al. (2022) reformulated the line prediction task as a regression problem instead of image segmentation through a two-stage DETR Carion et al. (2020) based approach. Recent work, GridFormer Lyu et al. (2023) , proposed a new method which directly predicts the vertexes and edges of the table grid (logical structure) from the table image. Cell-based approach is another type of detection-based methods where cell-level features (physical structure) are first detected, before classifying the relation between cells (logical structure) to form the full table structure. Some of the representative works include TabStructNet Raja et al. (2020) and FLAG-Net Liu et al. (2021a) which are end-to-end frameworks utilizing DGCNN architecture Wang et al. (2019b) to model the relation between the detected cell-level features. More recently, Hetero-TSR Liu