Sino-US-DrugQA: A benchmark for evaluating large language models in cross-jurisdictional pharmaceutical regulation

Xuejing Fu, Zhen Chen, Wentao Lu · 2026

Read Sino-US-DrugQA: A benchmark for evaluating large language models in cross-jurisdictional pharmaceutical regulation by Xuejing Fu, Zhen Chen, Wentao Lu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13577527.txt.

Description

Xuejing Fu, Zhen Chen, and Wentao Lu published this benchmark study in 2026. Drug regulation differs between the United States and China, and companies must satisfy both systems. Language models are being tried for such compliance work without a dedicated test. The team built Sino-US-DrugQA, a bilingual multiple-choice benchmark of 11,444 questions. Items cover single-system knowledge, comparisons between the two regulators, and parallel questions in both languages.

Every item passed checks, semantic screening, and expert review of a risk-weighted sample. Four leading models were tested cold. Accuracy landed between eighty-three and eighty-six percent. The catch was comparison: questions forcing models to contrast the two systems scored five to nine points lower than single-system ones. The two models were statistically tied. The authors conclude cross-jurisdiction reasoning, not recall, is the weak point. They recommend expert-supervised assistance over autonomous regulatory interpretation.

Catalog record

CREATORXuejing Fu, Zhen Chen, Wentao Lu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC13577527

Browse the full library · About the Tower of Babel