The Oracle Has Spoken: A Multi-Aspect Evaluation of Dialogue in Pythia

Zixun Chen, Petr Babkin, Akshat Gupta, Gopala Anumanchipalli, Xiaomo Liu · 2025

Read The Oracle Has Spoken: A Multi-Aspect Evaluation of Dialogue in Pythia by Zixun Chen, Petr Babkin, Akshat Gupta, Gopala Anumanchipalli, Xiaomo Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-16487.txt.

Introduction

What makes a chatbot good at dialogue is rarely measured piece by piece, since most scores blend all traits into one. Zixun Chen and four co-authors score the Pythia model family on many fine-grained dialogue traits drawn from language theory. They vary model size and add fine-tuning on conversation data, then watch each trait move.

Size alone changed most scores only mildly, while fine-tuning lifted them fast until they leveled off. Many scores moved in lockstep, especially when one judge model produced them all, which casts doubt on how separate the traits really are. Extra checks on word use help explain the pattern.

Catalog record

CREATORZixun Chen, Petr Babkin, Akshat Gupta, Gopala Anumanchipalli, Xiaomo Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-16487

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Oracle Has Spoken: A Multi-Aspect Evaluation of Dialogue in Pythia Zixun Chen, Petr Babkin, Akshat Gupta, Gopala Anumanchipalli, Xiaomo Liu Abstract Dialogue is one of the landmark abilities of large language models (LLMs). Despite its ubiquity, few studies actually distinguish specific ingredients underpinning dialogue behavior emerging during post-training. We employ a comprehensive suite of model-based metrics, each targeting a distinct fine-grained aspect of dialogue, motivated by linguistic theory. We evaluate how the performance of pre-trained Pythia models changes with respect to each of those dimensions, depending on model size and as a result of supervised fine-tuning on conversational datasets. We observe only a mild impact of raw model size on most metrics, whereas fine-tuning quickly saturates the scores for all but the smallest models tested. Somewhat contrary to our expectations, many metrics show very similar trends, especially if they are all rooted in the same evaluator model, which raises the question of their reliability in measuring a specific dimension. To that end, we conduct additional analyses of score distributions, metric correlations, and term frequencies in generated responses to help explain our observations. 1 Introduction Large Language Models (LLMs) took the world by storm, in part, thanks to their ability to engage in naturalistic dialogue. While the acquisition of general language fluency and world knowledge by LLMs is commonly attributed to the phase of large scale self-supervised pre-training ( Brown et al. 2020 ) , pinpointing the origins of specific abilities, such as dialogue, remains a challenge. Although it is tempting to take dialogue abilities of LLMs for granted, as naturally emerging from pre-training, there is a well documented behavior of raw base models responding to a prompt by reproducing its likely continuation in the pre-training corpus rather than conversing in a way a human person would 1 1 1 https://openai.com/index/instruction-following/ . A comparatively small-scale phase of post-training, which includes supervised fine-tuning on human-annotated prompt/response pairs, proved essential to ensuring alignment with user intent and preferences — including making them more conversational ( Ouyang et al. 2022 ) . Fine-tuning is a well known technique for improving downstream task performance. Due to its significantly smaller scale, fine-tuning merely “activates” skills and knowledge already learned during pre-training, and aligns them with user preferences; and its effectiveness is upper bounded by the base model’s size ( Zhou et al. 2023 ) . Similarly, LLM performance on downstream tasks is largely determined by the amount of pre-training of the base model ( Gudibande et al. 2023 ) . Furthermore, fine-tuning of smaller models, using imitation data from strong LLMs, teaches them to mimic the style of larger models but is generally unable to compensate for limited pre-training knowledge. Whether or not an LLM’s ability to engage in a conversation merely boils down to stylistic adaptation or constitutes a skill in and of itself, it is clear from linguistic theory that dialogue is predicated on the ability to recognize intent, infer discourse relations between utterances, and to keep track of the evolving state of the conversation ( Horn and Ward 2004 ) . In this work, we investigate the effects of conversational finetuning on an open source LLM Pythia ( Biderman et al. 2023 ) of five different sizes through the lens of these fine-grained linguistic dimensions. We contribute an empirical analysis of changes in model’s text generation behavior, as a result of finetuning on 3 chat datasets: Databricks Dolly 2 2 2 https://huggingface.co/datasets/databricks/databricks-dolly-15k , Open Assistant 3 3 3 https://huggingface.co/datasets/OpenAssistant , and ShareGPT 4 4 4 https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered , with respect to dialogue-specific metrics: UniEval ( Zhong et al. 2022 ) , Themis ( Hu et al. 2024 ) , and a targeted GPT-4-based assessment. 2 Related Work Several works previously evaluated LLMs fine-tuned on conversational data. We summarize them below, while focusing on differences from our work. Alghisi et al. ( Alghisi et al. 2024 ) evaluated different LLM adaptation techniques: in-context learning, fine-tuning and RAG across open domain, knowledge-grounded and task-based dialogue. They find fine-tuning to provide superior lift compared to ICL while retrieval augmentation moderately improves both ICL and SFT. Notably, they used chat/instruct variants of Llama ( Touvron et al. 2023 ) and Mistral ( Jiang et al. 2023 ) without controlling for model size. Whereas, we start with base models of different sizes and directly fine-tune them on each dataset without relying on chat/instruct variants. Additionally the use of perplexity for evaluation is limiting, as it gives an overall measure of gold text’s uncertainty under the model, whereas we look at fine-grained linguistic dimensions with greater interpretability. Mousavi et al. ( Mousavi et al. 2022 ) also conducted dialogue fine-tuning experiments with T5 ( Raffel et al. 2023 ) and GPT-2 ( Radford et al. 2019 ) in Italian. Their work emphasized human evaluation of dimensions such as appropriateness, contextualization, and grammar correctness, which are more informative than perplexity but are quite ad hoc (e.g., “Genericness”) and are not based on any concrete linguistic phenomena. Despite our evaluation using automatic metrics, they are all based on models specifically trained to reproduce human judgements on text annotation tasks. DialogBench ( Ou et al. 2024 ) is a synthetic dataset of dialogue-related tasks encompassing intent recognition, knowledge grounded generation and coherent infilling, rendered as multiple-choice questions. By contrast, we use organic conversations for which we simultaneously compute multiple metrics each targeting a different aspect of dialogue. Authors evaluated a large number of pre-trained and chat/instruction-tuned LLMs from different families and sizes and noted base LLMs do well on correctness-related tasks but struggle with coherence and safety. Our evaluation additionally includes analyses of metric reliability. 3 Methodology We use the Pythia family of models ( Biderman et al. 2023 ) as the testbed for our fine-tuning experiments. These decoder-only LLMs are pre-trained on 1 trillion tokens of the Pile corpus and come in 10 model sizes, ranging from 14 million to 12 billion parameters — all trained on data presented in the same order to facilitate reproducibility research. To reasonably cover the configuration space, we select the following checkpoints: 140m, 410m, 1.3b, 2.8b, 6.9b (all dedup). To ensure diversity of the the fine-tuning datasets, we use a combination of single and multi-turn corpora authored by human annotators, as well as synthesized by ChatGPT: Dolly, Open Assistant, ShareGPT. We sample 10k conversations from each. 3.1 Evaluation Metrics Evaluation of dialogue is inherently challenging due to a large space of valid responses to any given utterance. Rather than capturing general quality of generated responses (which is expected to improve through fine-tuning on any corpus), our goal is to disentangle specific linguistic dimensions that contribute to high-quality dialogue. While human evaluation is considered a gold standard for assessing LLMs, it comes with its own set of challenges, including bias, subjectivity, low reproducibility, and high cost. After an extensive survey of automatic metrics, we adopt the following 3 model-based metrics that most closely reflect qualities characteristic of dialogue. UniEval ( Zhong et al. 2022 ) is based on the T5 model, finetuned in the to predict entailment between a text and a set of interpretable attributes, which for dialogue include: Naturalness, Coherence, Engagingness, Groundedness, and Understandability. Depending on a metric, it can be computed either on

Read the complete text