IPBench: Benchmarking the Knowledge of Large Language Models in Intellectual Property

Qiyao Wang, Guhong Chen, Hongbo Wang, Huaren Liu, Minghui Zhu, Zhifei Qin, Linwei Li, Yilin Yue, Shiqiang Wang, Jiayan Li, Yihang Wu, Ziqiang Liu,, Longze Chen, Run Luo, Liyang Fan, Jiaming Li, Lei Zhang, Kan Xu,, Hongfei Lin, Hamid Alinejad-Rokny, Shiwen Ni , Yuan Lin, Min Yang · 2025

Read IPBench: Benchmarking the Knowledge of Large Language Models in Intellectual Property by Qiyao Wang, Guhong Chen, Hongbo Wang, Huaren Liu, Minghui Zhu, Zhifei Qin, Linwei Li, Yilin Yue, Shiqiang Wang, Jiayan Li, Yihang Wu, Ziqiang Liu,, Longze Chen, Run Luo, Liyang Fan, Jiaming Li, Lei Zhang, Kan Xu,, Hongfei Lin, Hamid Alinejad-Rokny, Shiwen Ni , Yuan Lin, Min Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-15524.txt.

Description

Qiyao Wang and co-authors benchmark language models on intellectual property in 2025. Patent and trademark work blends deep legal and technical knowledge. Past test sets covered patents alone or only slices of the field. Their IPBench is billed as the first broad taxonomy of such tasks.

The bilingual set spans eight property mechanisms and twenty distinct tasks. Tasks mirror real work, from drafting to analysis of filings. Leading models were scored to expose gaps in this specialist domain. A shared yardstick should help law and tech teams judge which models can aid real property work safely.

Catalog record

CREATORQiyao Wang, Guhong Chen, Hongbo Wang, Huaren Liu, Minghui Zhu, Zhifei Qin, Linwei Li, Yilin Yue, Shiqiang Wang, Jiayan Li, Yihang Wu, Ziqiang Liu,, Longze Chen, Run Luo, Liyang Fan, Jiaming Li, Lei Zhang, Kan Xu,, Hongfei Lin, Hamid Alinejad-Rokny, Shiwen Ni , Yuan Lin, Min Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE142 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-15524

Browse the full library · About the Tower of Babel