HyPAC: Cost-Efficient LLMs–Human Hybrid Annotation with PAC Error Guarantees
Hao Zeng, Huipeng Huang, Xinhao Qu, Jianguo Huang, Bingyi Jing, Hongxin Wei · 2026
Read HyPAC: Cost-Efficient LLMs–Human Hybrid Annotation with PAC Error Guarantees by Hao Zeng, Huipeng Huang, Xinhao Qu, Jianguo Huang, Bingyi Jing, Hongxin Wei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02550.txt.
Description
Hao Zeng and 5 co-authors route annotation work among language models and human experts while guaranteeing the error rate, in a 2026 arXiv paper. Labeling data can use fast cheap models, slower reasoning models, or expensive humans, each with a different cost-quality balance. The challenge is picking the cheapest adequate source per item without losing control of test errors.
HyPAC calibrates two decision thresholds using importance sampling and upper confidence bounds. Inputs split into three uncertainty regions, each routed to an appropriate source. The authors prove the method achieves minimum expected cost with a probably approximately correct guarantee on annotation error. The guarantee is distribution-free and does not depend on pretrained models. Benchmark experiments cut annotation cost by about seventy-nine percent while holding the error tightly at its target. Hybrid labeling becomes a provable saving, not a gamble.
Catalog record
| CREATOR | Hao Zeng, Huipeng Huang, Xinhao Qu, Jianguo Huang, Bingyi Jing, Hongxin Wei |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 86 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-02550