PP-FormulaNet: Bridging Accuracy and Efficiency in Advanced Formula Recognition
Hongen Liu, Cheng Cui, Yuning Du, Yi Liu, Gang Pan · 2025
Read PP-FormulaNet: Bridging Accuracy and Efficiency in Advanced Formula Recognition by Hongen Liu, Cheng Cui, Yuning Du, Yi Liu, Gang Pan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18382.txt.
Introduction
Hongen Liu, Cheng Cui, Yuning Du, Yi Liu, and Gang Pan wrote this paper in two thousand twenty-five. It presents PP-FormulaNet, a model that reads math formulas in document scans and writes them out as LaTeX.
Two builds serve two needs. The L build chases top accuracy, and beats a strong rival, UniMERNet, by a clear margin. The S build chases speed, and runs more than sixteen times faster than that pace. A formula mining system rounds out the work, pulling vast troves of clean training data from documents. Both builds ship in the public PaddleOCR and PaddleX toolkits. The aim is smooth use in document flows heavy with math.
Catalog record
| CREATOR | Hongen Liu, Cheng Cui, Yuning Du, Yi Liu, Gang Pan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18382
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PP-FormulaNet: Bridging Accuracy and Efficiency in Advanced Formula Recognition Hongen Liu, Cheng Cui, Yuning Du, Yi Liu, Gang Pan Abstract Formula recognition is an important task in document intelligence. It involves converting mathematical expressions from document images into structured symbolic formats that computers can easily work with. LaTeX is the most common format used for this purpose. In this work, we present PP-FormulaNet , a state-of-the-art formula recognition model that excels in both accuracy and efficiency. To meet the diverse needs of applications, we have developed two specialized models: PP-FormulaNet-L, tailored for high-accuracy scenarios, and PP-FormulaNet-S, optimized for high-efficiency contexts. Our extensive evaluations reveal that PP-FormulaNet-L attains accuracy levels that surpass those of prominent models such as UniMERNet by a significant 6%. Conversely, PP-FormulaNet-S operates at speeds that are over 16 times faster. These advancements facilitate seamless integration of PP-FormulaNet into a broad spectrum of document processing environments that involve intricate mathematical formulas. Furthermore, we introduce a Formula Mining System , which is capable of extracting a vast amount of high-quality formula data. This system further enhances the robustness and applicability of our formula recognition model. Code and models are publicly available at PaddleOCR 1 1 1 https://github.com/PaddlePaddle/PaddleOCR and PaddleX 2 2 2 https://github.com/PaddlePaddle/PaddleX . 1 Introduction (a) Definition of Formula Recognition (b) Comparison of the Avg-BLEU and FPS of Different Models Figure 1 : Definition of Formula Recognition, and Comparison of the Avg-BLEU and FPS of Different Models. GPU Inference Time tested on 32G Tesla V100 with batch size of 15. Formulas, as the core knowledge carriers in scientific literature, technical documents, and educational materials, embody the abstract logic and mathematical expressions of human civilization. With the development of large language models, multimodal AI, and intelligent scientific computation, the structured parsing of formula semantics has become a critical breakthrough for constructing scientific knowledge graphs and enhancing AI’s mathematical reasoning capabilities. As shown in Figure 1 , utilizing formula recognition technology to structurally parse inter-line formulas, inline formulas, and handwritten formulas in contexts such as academic papers and engineering drawings—combined with symbolic semantic understanding and LaTeX generation capabilities—can infuse large model training with precise mathematical prior knowledge. This significantly enhances AI’s problem-solving, theorem derivation, and scientific document generation abilities in the research field. Additionally, formula recognition can be integrated with large models to facilitate the construction of knowledge bases in the scientific domain. In recent years, thanks to the powerful sequence modeling capabilities of transformers, formula recognition algorithms have developed rapidly. However, most algorithms have a limited vocabulary size of only 300 classes and are focused on the singular scenario of handwritten formula recognition, making them difficult to apply to real-world document parsing scenarios. Additionally, these algorithms tend to prioritize improvements in formula accuracy, often at the expense of inference speed and model size. Therefore, enhancing the generalization ability of formula algorithms in complex scenarios, while optimizing inference speed and model size, holds significant research value. To address the aforementioned issues, UniMERNet [ 1 ] uses large-scale pre-training datasets from arXiv to enhance model generalization for complex formulas. Strategies like reducing model dimensions, replacing attention modules with large-kernel convolutions, and using multi-line bidirectional decoding [ 2 ] improve inference speed. However, these methods have limitations. Crawling programs that rely on \begin{equation} tags often discard formulas if rendering fails, missing complex structures defined with commands like \def and \newcommand . Moreover, reducing model size doesn’t always enhance inference speed due to the autoregressive nature of models, where multiple iterative steps are required to predict the formula. However, during model lightweighting, there is an inevitable loss of accuracy, making it challenging for the model to accurately predict the sequence termination point. Consequently, although the number of steps for single inference has decreased, the total number of inference steps has actually increased. Balancing both, there is little change in the overall inference speed of the model. In this paper, we adopt the method of crawling arXiv papers to obtain a large dataset of formula training data. Unlike previous approaches, we develop a more comprehensive formula extraction script that handles user-defined commands and ensured compatibility with various formula identifiers. Based on this program, a dataset of 4 million arXiv formulas is constructed. Additionally, we significantly enhance the CNN network’s capability to model formula structures by distilling the weights of the multi-modal large model GOT2.0-OCR [ 3 ] onto PP-HGNetV2-B4 [ 4 ] . Meanwhile, considering the issue that the original pre-trained weights cannot be loaded after model dimensionality reduction, we propose a dimensional interpolation technique to adjust the pre-trained weights. Finally, we introduce a multi-token prediction method that significantly reduces the number of steps in formula sequence inference, thereby improving inference efficiency. In summary, this work developed an accurate and comprehensive formula mining system and proposed PP-FormulaNet based on techniques such as weight interpolation, knowledge distillation, and multi-token prediction, to meet the performance requirements of practical document processing scenarios. The main contributions are as follows: • We develop an innovative formula mining system that considers various potential formula identifiers and restores complex formulas containing user-defined commands. This significantly enhances the completeness and accuracy of formula dataset construction. • We introduce PP-FormulaNet, a formula recognition model specifically designed for real-world document analysis scenarios. As depicted in Figure 1 , PP-FormulaNet-L delivers precision levels that are 6% higher than those of leading models such as UnimerNet, while PP-FormulaNet-S functions at speeds that are 16 times faster. • We design methods such as weight interpolation and model distillation to effectively utilize the formula representations in pre-trained weights during the model lightweighting process. • We offer a new perspective on accelerating formula inference by employing multi-token prediction techniques, which significantly reduce the number of inference steps for formula sequences. 2 Related Works Existing formula recognition algorithms can mainly be classified into three categories: traditional machine learning methods, deep learning methods, and large model-driven methods. 2.1 Traditional Machine Learning Methods Traditional machine learning methods [ 5 , 6 , 7 ] primarily rely on handcrafted feature extraction and classifiers. These methods typically consist of three main steps: character grouping, character recognition and relationship analysis. [ 7 ] was the first to use a set of replacement rules to model the two-dimensional structure of formula images. [ 6 ] improved the accuracy of formula character recognition by using segmental K-means to initialize the Gaussian Mixture Model parameters and introduced features such as the normalized distance of stroke edges to model formula characters. [ 5 ] used Hidden Markov Models (HMM) to recognize formula characters and employed stochastic context-free grammar to model the structural relationships between characters. While these methods achieved some success in early formula recognition tasks, their generalization ability and robustness are limited due to the reliance on handcrafted features, especially when dealing with complex formulas or images with significant noise. Figure 2 : The overall architecture of PP-FormulaNet. Initially, formula-image pairs are extracted from arXiv paper sources through a formula mining system, resulting in a dataset comprising 4 million formulas. PP-FormulaNet consists of two variants: (1) PP-FormulaNet-L: A vision encoder based on the Vary-VIT-B backbone from GOT-2.0, combined with a 512-dimensional MBart Decoder. (2) PP-FormulaNet-S: A vision encoder based on the distilled PP-HGNetV2-B4, combined with a 384-dimensional MBart