MHTS: Multi-Hop Tree Structure Framework for Generating Difficulty-Controllable QA Datasets for RAG Evaluation

Jeongsoo Lee*, Daeyong Kwon*, Kyohoon Jin, Junnyeong Jeong Minwoo Sim, Minwoo Kim · 2025

Read MHTS: Multi-Hop Tree Structure Framework for Generating Difficulty-Controllable QA Datasets for RAG Evaluation by Jeongsoo Lee*, Daeyong Kwon*, Kyohoon Jin, Junnyeong Jeong Minwoo Sim, Minwoo Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-08756.txt.

Description

Jeongsoo Lee and co-authors build a way to mint test questions with tuned difficulty for retrieval systems in a 2025 paper. Search augmented chatbots need hard tests, yet most test sets lean easy. Easy sets inflate scores and hide weak spots. The frame grows questions along a tree of linked facts, so each question needs set leaps across chunks.

A grading formula estimates difficulty from leap count and evidence spread. That grade tracks real system scores closely in checks. Sets built this way stay varied and clean while difficulty stays under control. Judges of retrieval and answer quality both gain sharper tools. The method can expose where systems fail on deep questions.

Catalog record

CREATORJeongsoo Lee*, Daeyong Kwon*, Kyohoon Jin, Junnyeong Jeong Minwoo Sim, Minwoo Kim
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-08756

Browse the full library · About the Tower of Babel