BAP v2: An Enhanced Task Framework for Instruction Following in Minecraft Dialogues

Prashant Jayannavar ,1, Liliang Ren ,2, Marisa Hudspeth ∗∗,3, Risham Sidhu 1, Charlotte Lambert ,∗∗,1, Ariel Cordes †,∗∗, Elizabeth Kaplan †,∗∗,4, Anjali Narayan-Chen †,∗∗,5, Julia Hockenmaier ∗,1 · 2025

Read BAP v2: An Enhanced Task Framework for Instruction Following in Minecraft Dialogues by Prashant Jayannavar ,1, Liliang Ren ,2, Marisa Hudspeth ∗∗,3, Risham Sidhu 1, Charlotte Lambert ,∗∗,1, Ariel Cordes †,∗∗, Elizabeth Kaplan †,∗∗,4, Anjali Narayan-Chen †,∗∗,5, Julia Hockenmaier ∗,1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-10836.txt.

Introduction

Prashant Jayannavar and colleagues published a paper in 2025 on teaching software agents to follow spoken instructions. Their setting is a Minecraft building game played by two people. One player, the architect, describes a target structure. The other, the builder, must carry out the directions. The task is predicting the builder's next actions.

The authors rebuilt the task setup and called it BAP version two. They cleaned the test set and designed fairer scoring measures. New measures showed spatial reasoning as the main weakness of current models. To ease scarce training data, they generated synthetic game dialogues. These extra examples taught basic spatial skills. Even strong language models still struggled with the spatial side.

Catalog record

CREATORPrashant Jayannavar ,1, Liliang Ren ,2, Marisa Hudspeth ∗∗,3, Risham Sidhu 1, Charlotte Lambert ,∗∗,1, Ariel Cordes †,∗∗, Elizabeth Kaplan †,∗∗,4, Anjali Narayan-Chen †,∗∗,5, Julia Hockenmaier ∗,1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE143 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-10836

Browse the full library · About the Tower of Babel