NitiBench: A Comprehensive Study of LLM Framework Capabilities for Thai Legal Question Answering

Pawitsapak Akarajaradwong †, Pirat Pothavorn †, Chompakorn Chaksangchaichot †, Panuthep Tasawong, Thitiwat Nopparatbundit, Keerakiat Pratai, Sarana Nutanong · 2025

Read NitiBench: A Comprehensive Study of LLM Framework Capabilities for Thai Legal Question Answering by Pawitsapak Akarajaradwong †, Pirat Pothavorn †, Chompakorn Chaksangchaichot †, Panuthep Tasawong, Thitiwat Nopparatbundit, Keerakiat Pratai, Sarana Nutanong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10868.txt.

Description

Pawitsapak Akarajaradwong and co-authors built NitiBench in 2025, a test set for large language models that answer Thai legal questions. Thai law is hard to search, and no shared test existed. The set covers money law in one part and real tax cases in another.

The team compared systems that search for text first with systems that read long text at once. Cutting text by legal section helped a lot, they report. Search tools still struggled with hard queries, and long-reading models did worse than search-based ones. The authors also add fair scoring rules and a model-based judge, and they share their code and data in public for others to use.

Catalog record

CREATORPawitsapak Akarajaradwong †, Pirat Pothavorn †, Chompakorn Chaksangchaichot †, Panuthep Tasawong, Thitiwat Nopparatbundit, Keerakiat Pratai, Sarana Nutanong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE150 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10868

Browse the full library · About the Tower of Babel