Multimodal Search in Chemical Documents and Reactions DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Informa

Ayush Kumar Shah, Abhisek Dey, Leo Luo, Bryan Amador, Patrick Philippy, Ming Zhong, Siru Ouyang, David Mark Friday, David Bianchi, Nick Jackson, Richard Zanibbi, Jiawei Han · 2025

Read Multimodal Search in Chemical Documents and Reactions DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Informa by Ayush Kumar Shah, Abhisek Dey, Leo Luo, Bryan Amador, Patrick Philippy, Ming Zhong, Siru Ouyang, David Mark Friday, David Bianchi, Nick Jackson, Richard Zanibbi, Jiawei Han (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-16865.txt.

Description

Ayush Kumar Shah and eleven co-authors build a search tool for chemistry that reads pictures and text together. Their 2025 demo paper targets chemical documents and reactions. Chemists store knowledge as diagrams, tables, and prose at once. Searching only words misses the structures that matter most.

Queries can mix a drawn molecule, a text phrase, and reaction data. Behind the scenes, the system pulls diagrams from papers and parses them. Reaction facts are lifted from text into tables. Diagram mentions are linked across to their text. Experts judged the tool's retrievals in tests. The demo shows how cross-modal links turn a library into a workbench.

Catalog record

CREATORAyush Kumar Shah, Abhisek Dey, Leo Luo, Bryan Amador, Patrick Philippy, Ming Zhong, Siru Ouyang, David Mark Friday, David Bianchi, Nick Jackson, Richard Zanibbi, Jiawei Han
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-16865

Browse the full library · About the Tower of Babel