Lightweight and Production-Ready PDF Visual Element Parsing

Meizhu Liu, Yassi Abbasi, Matthew Rowe, Michael Avendi, Paul Li · 2026

Read Lightweight and Production-Ready PDF Visual Element Parsing by Meizhu Liu, Yassi Abbasi, Matthew Rowe, Michael Avendi, Paul Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23276.txt.

Introduction

Liu and four colleagues build a lightweight production PDF parser that detects figures, tables, and forms and links captions to the right elements. Spatial heuristics, layout analysis, and semantic similarity replace parsers that miss complex visuals, grab watermarks, fragment elements, and misplace captions. On benchmarks and internal data, it reaches at least 96 percent visual detection accuracy and 93 percent caption association accuracy. As a preprocessing step for multimodal retrieval, it beats leading parsers and large vision-language models while cutting latency by over two times. It already runs in production.

Catalog record

CREATORMeizhu Liu, Yassi Abbasi, Matthew Rowe, Michael Avendi, Paul Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31611
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.23276

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Lightweight and Production-Ready PDF Visual Element Parsing Meizhu Liu, Yassi Abbasi, Matthew Rowe, Michael Avendi, Paul Li Abstract PDF documents contain critical visual elements such as figures, tables, and forms whose accurate extraction is essential for document understanding and multimodal retrieval-augmented generation (RAG). Existing PDF parsers often miss complex visuals, extract non-informative artifacts (e.g., watermarks, logos), produce fragmented elements, and fail to reliably associate captions with their corresponding elements, which degrades downstream retrieval and question answering. We present a lightweight and production level PDF parsing framework that can accurately detect visual elements and associates captions using a combination of spatial heuristics, layout analysis, and semantic similarity. On popular benchmark datasets and internal product data, the proposed solution achieves ≥ 96 % \geq 96\% visual element detection accuracy and 93 % 93\% caption association accuracy. When used as a preprocessing step for multimodal RAG, it significantly outperforms state-of-the-art parsers and large vision-language models on both internal data and the MMDocRAG benchmark, while reducing latency by over 2 × 2\times . We have deployed the proposed system in challenging production environment. 1 Introduction Accurate understanding of visual elements from PDF documents is a foundational requirement for document analysis, automation, and multimodal retrieval-augmented generation (RAG). Modern PDFs interleave text with tables, figures, forms, and diagrams in highly heterogeneous layouts, making reliable parsing challenging. Errors at this stage—such as missing tables, fragmented images, or incorrect caption alignment—propagate and substantially degrade downstream retrieval and question answering performance. Prior work on PDF parsing can be broadly grouped into heuristic-based tools (e.g., PDFPlumber 24 , PyMuPDF 5 , PDFMiner 23 , Apache Tika 11 ), layout learning approaches (e.g., PubLayNet 30 , DocBank 14 , DocLayout-YOLO 29 , Unstructured 25 ), and vision-language models. Heuristic tools provide efficient access to low-level PDF primitives but struggle with complex layouts, forms, and caption alignment. Layout learning methods improve structural detection but often lack robustness across domains and may not address visual noise or caption association. Recent VLMs demonstrate strong end-to-end PDF understanding ability but incur high computational cost and latency and are unsuitable for large-scale deployment. Our work complements these approaches by combining lightweight heuristics with semantic reasoning, achieving competitive accuracy with substantially lower latency. Our major contributions could be summarized as follows. Table detection with spatial and graphical heuristics. We improve table identification accuracy by combining graphical analysis with spatial alignment heuristics. Bordered tables are detected through vector graphics analysis (e.g., counting straight lines), while unbordered tables are identified by detecting repeated text block alignments into rows and columns. This approach effectively handles both explicitly structured and visually implicit table layouts without requiring predefined templates. Layout-driven form detection via text block profiling. We introduce an adaptive form detection method that profiles text blocks based on density and character length. Pages with a high occurrence of short snippets—typical of form fields and labels—are automatically flagged, removing the need for predefined templates or field dictionaries. Redundancy-resilient image merging. We propose a post-processing mechanism to merge overlapping or duplicate image extractions by comparing bounding box coordinates and applying overlap thresholds. This ensures that only a single, accurate instance of each visual element is preserved, which is crucial for technical diagrams and blueprints where fragmentation can distort meaning. Multi-tiered visual noise filtering. We develop a robust filtering strategy to remove non-essential visual artifacts (e.g., watermarks, logos) by combining: 1) spatial analysis such as position and size, 2) appearance frequency across pages, 3) transparency detection via alpha channels, 4) semantic similarity via embedding vectors. This combination of heuristics and semantic reasoning substantially improves precision in distinguishing useful content from noise. Caption extraction with semantic-layout fusion. We design a hybrid method to associate captions with visuals from the following angles: 1) geometric relationships between visuals and nearby text, 2) spatial clustering of text blocks, 3) stylistic cues including font size and boldness, 4) semantic embedding similarity. This enables accurate title and caption extraction even in poorly structured or inconsistently formatted documents. An efficient, production-ready PDF parsing framework. We integrate all components into a unified, lightweight PDF parsing system that enables accurate detection and semantic association of visual elements at scale. The resulting document representations are cleaner and more structured, substantially benefiting downstream applications such as multimodal retrieval and document question answering. 2 Proposed Method We present a production-ready unified framework for detecting and extracting key visual elements—images, tables, and forms—from PDF documents, while robustly associating each element with its corresponding caption or title under complex and inconsistent layouts. The system consists of four main components: 1. heuristic-based table and form detection, 2. detection and consolidation of duplicate or fragmented images, 3. filtering of non-informative artifacts (e.g., logos and watermarks), and 4. caption association using combined layout and semantic cues. Our pipeline leverages PyMuPDF to parse each PDF page and extract three modalities: embedded images with bounding boxes, text blocks, and low-level drawing primitives (e.g., lines and rectangles). On top of this raw output, we apply heuristic algorithms to infer higher-level structures. The extraction pipeline includes the following modules, each of which will be presented in the coming sections. 2.1 Form Detection • Images: Detected via page.get_images and extracted using doc.extract_image . • Bounding boxes: Obtained from page.get_image_rects . • Text blocks: Retrieved using page.get_text("blocks") with positional metadata. • Tables: Identified through spatial heuristics over text block alignment and density. • Forms: Detected using layout- and widget-based visual heuristics. • Captions and titles: Matched to visual elements using proximity-based layout cues combined with semantic similarity. • Artifact filtering: Discriminative rules separate meaningful figures from decorative or repetitive visuals. Algorithm 1 Detect Table in PDF Page 1: procedure PageContainsTable ( page ) 2: if |\{d\in\text{page.get\_drawings()}:d.\text{type}=\text{"line"}\}|\geq K then 3: return True \triangleright Bordered table 4: end if 5: \text{blocks}\leftarrow\{(x,y):(x,y,t)\in\text{page.get\_text("blocks")},\ t\neq\emptyset\} 6: \text{x\_cnt}\leftarrow\text{Count}(\text{round}(x)),\ \text{y\_cnt}\leftarrow\text{Count}(\text{round}(y)) 7: if |\{v\in\text{x\_cnt}:v\geq 3\}|\geq M and |\{v\in\text{y\_cnt}:v\geq 3\}|\geq M then 8: return True \triangleright Unbordered table 9: end if 10: return False 11: end procedure 2.2 Table Extraction We improve table detection using spatial heuristics that exploit text alignment and graphical structure (Algorithm 1 ), and it could provide layout signals robust to font variation, language, and OCR noise. Using PyMuPDF primitives, the method detects both bordered and unbordered tables. For bordered tables, vector drawings are analyzed to count straight lines; pages with at least K lines are classified as containing tables. For unbordered tables, we examine text block coordinates, where repeated and aligned x and y positions reveal column and row structures. Pages exhibiting at least M distinct vertical and horizontal alignments are classified as containing tables, reducing false negatives and improving robustness across diverse PDF layouts. Algorithm 2 Detection of AcroForm Widgets in PDF Pages 1: PDF document D 2: List of detected form fields and their metadata 3: for all page p\in D do 4: widgets \leftarrow p.widgets() 5: for all widget w\in widgets do 6: field_name \leftarrow w.\texttt{field\_name} 7: field_type \leftarrow w.\texttt{field\_type} 8: rect \leftarrow w.\texttt{rect} 9: output ( field_name , field_type , rect ) 10: end for 11: end for Since PyMuPDF does not natively support form detection, we identify form-like content using three complementary strategies: • Explicit PDF form fields (AcroForms): For PDFs with embedded form fields, we extract interactive elements (e.g., text inputs, checkboxes) via the AcroForm

Read the complete text