HIPPO: Enhancing the Table Understanding Capability of LLMs through Hybrid-Modal Preference Optimization
Haolan Wang, Zhenghao Liu, Xinze Li, Xiaocui Yang, Yu Gu, Yukun Yan, Qi Shi, Fangfang Li 🖂, Chong Chen, Ge Yu · 2025
Read HIPPO: Enhancing the Table Understanding Capability of LLMs through Hybrid-Modal Preference Optimization by Haolan Wang, Zhenghao Liu, Xinze Li, Xiaocui Yang, Yu Gu, Yukun Yan, Qi Shi, Fangfang Li 🖂, Chong Chen, Ge Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-17315.txt.
Description
Tables carry meaning in their layout, not just their words, and language models often miss that structure. Haolan Wang and co-authors feed models the table twice at once: as text and as an image. Their system, HIPPO, trains with preference optimization on answers sampled from this hybrid view. A sampling rule keeps the two formats balanced, so the model cannot lean on one and ignore the other.
On table question answering and fact checking, HIPPO beat rival table models by about four points. Tests showed the model did more than improve on each format alone. It pulled complementary clues from the pairing, such as layout from the image and wording from the text. The code is public.
Catalog record
| CREATOR | Haolan Wang, Zhenghao Liu, Xinze Li, Xiaocui Yang, Yu Gu, Yukun Yan, Qi Shi, Fangfang Li 🖂, Chong Chen, Ge Yu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-17315