A Multi-view Discourse Framework for Integrating Semantic and Syntactic Features in Dialog Agents

Akanksha Mehndiratta, Krishna Asawa · 2025

Read A Multi-view Discourse Framework for Integrating Semantic and Syntactic Features in Dialog Agents by Akanksha Mehndiratta, Krishna Asawa (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-09073.txt.

Introduction

Akanksha Mehndiratta and Krishna Asawa present a framework that helps chat programs pick the best reply in a multi-turn talk. Their paper appeared in 2025. Many systems treat all past turns as equal, or ignore how turns relate to each other. The framework first encodes each turn with meaning, word order, and grammar features. It then learns tokens that capture how a turn links to the turns around it.

The two steps build a picture of the flow of the talk, not just its words. The team tested the framework on the Ubuntu Dialogue Corpus, a large set of tech-support chats. It beat older methods on standard scoring measures. The authors argue that tracking how turns connect helps a system choose replies that fit the talk so far.

Catalog record

CREATORAkanksha Mehndiratta, Krishna Asawa
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-09073

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Multi-view Discourse Framework for Integrating Semantic and Syntactic Features in Dialog Agents Akanksha Mehndiratta, Krishna Asawa Abstract Multiturn dialogue models aim to generate human-like responses by leveraging conversational context, consisting of utterances from previous exchanges. Existing methods often neglect the interactions between these utterances or treat all of them as equally significant. This paper introduces a discourse-aware framework for response selection in retrieval-based dialogue systems. The proposed model first encodes each utterance and response with contextual, positional, and syntactic features using Multi-view Canonical Correlation Analysis (MCCA). It then learns discourse tokens that capture relationships between an utterance and its surrounding turns in a shared subspace via Canonical Correlation Analysis (CCA). This two-step approach effectively integrates semantic and syntactic features to build discourse-level understanding. Experiments on the Ubuntu Dialogue Corpus demonstrate that our model achieves significant improvements in automatic evaluation metrics, highlighting its effectiveness in response selection. 1 Introduction Dialogue systems can be categorized into task-oriented dialogue systems [ 23 , 27 , 20 ] , and non-task-oriented dialogue systems [ 29 , 19 , 10 ] , also known as open-domain dialogue systems. Task-oriented dialogue systems aim to assist users in accomplishing specific tasks, such as booking a flight or offering customer support. They rely on predefined dialogs, often enhanced with natural language understanding (NLU) models, to interpret user input and generate responses. As virtual assistants, they excel at handling structured tasks and delivering goal-directed outcomes but lack the flexibility to manage open-ended or non-task-related queries. In contrast, open-domain systems are capable of handling conversations on a wide range of topics that are not restricted to specific tasks or domains. These systems have experienced significant growth due to the availability of large dialogue datasets [ 17 , 4 , 9 , 35 ] and advancements in deep learning and pre-trained models [ 21 , 5 ] . However, they face challenges in maintaining coherence and accuracy during longer conversations. In open-domain dialogue systems, retrieval-based agents have gained substantial attention. In multi-turn response selection, a dialogue context is provided as input, and retrieval-based methods aim to develop systems that can identify relevant responses or information from a large database based on the input context. The dialogue context includes both system and user interactions, providing critical details about the topic and focus of the ongoing conversation. Recent developments in multi-turn retrieval-based agents have focused on language modeling techniques such as sequential models like RNNs and LSTMs [ 12 ] , attention-based models [ 30 ] , and hierarchical architectures (e.g., RNN-RNN [ 28 ] or CNN-RNN [ 36 ] ). More recently, transformers and pre-trained models have significantly boosted the performance of dialogue agents. Although pretrained models have enhanced response quality, they are limited by high computational costs and difficulties in managing long-term context. This paper introduces a novel approach to integrate the dialogue context with discourse-level knowledge by focusing on the syntactic and semantic significance of the input context in a conversation for response selection. The proposed method exploits the multi-view nature of textual data in two key ways and utilizes Canonical Correlation Analysis (CCA) [ 7 ] to develop discourse-level knowledge. The model begins by developing utterance level representations for each utterance in the input context and for candidate responses using Multi-View Canonical Correlation Analysis (MVCCA) to capture a multi-modal perspective of an utterance. Next, it interprets the utterance level representations of an utterance and its surrounding dialogue turns as two sets of variables that have a joint multivariate normal distribution and exploits Canonical Correlation Analysis (CCA) to learn shared intent representations as correlation coefficients. Unique intent representations are treated as discourse tokens. The degree of similarity between the discourse tokens for the dialogue context and the candidate responses determines how aligned a response is. By identifying semantic similarity using discourse tokens, the proposed Multi-view Discourse Framework(MVDF) aims to highlight the importance of filtering out suitable context in fetching responses that are contextually relevant and coherent, ensuring smooth dialogue progression. Experiments conducted with the Ubuntu dataset demonstrate that the proposed model improves automatic evaluation metrics, proving its effectiveness. 2 Related Work A generalized framework for retrieval-based dialogue agents utilizes information retrieval (IR) techniques to rank a set of pre-built candidate responses and select the most appropriate one from the top-ranked candidates. Chatbots built on this framework have garnered significant attention within the information retrieval community, due to their ability to generate fluent and informative responses [ 11 , 30 , 31 ] . Approaches in this field have evolved from sequence-to-sequence and transformer models to methods using pre-trained models such as BERT [ 5 ] . Early research in retrieval-based agents focused on discourse modeling in multiparty dialog systems. Models trained on annotated datasets developed for spoken language, such as the STAC corpus [ 2 ] and the Molweni corpus [ 8 ] , largely focus on detecting links between discourse dependencies and classifying discourse relations. Discourse modeling-based frameworks include approaches such as MST [ 1 ] , ILP [ 18 ] , and Deep Sequential [ 24 ] . The prosperity of the deep learning paradigm led to the conception of multi-turn conversational systems where a dense and continuous vector represents the textual information. The modeling framework can be classified into the following categories: (1) representation-based models [ 12 , 30 , 36 ] and (2) interaction-based models [ 37 , 33 , 32 , 28 , 25 , 26 , 13 ] . Representation-based models focus on generating a representation of the encoding for the input document and query and obtaining a score based on the degree of matching between them. The interaction-based models first build a matching framework between input query and document to compute interaction matrices, then using the interaction matrix as input, a CNN or variant of CNN is utilized to output the similarity score. More recently, pre-trained models have seen significant success in natural language processing tasks. These high-capacity contextual language models are front-runners in almost every task such as Question Answering (QA) and Natural Language Inference (NLI) [ 5 ] . Although these pre-trained models excel at capturing semantic relationships and dependencies within text, these systems are computationally intensive, and their pre-training objective hampers their ability to manage long-term context [ 14 ] . 3 Canonical Corelation Analysis This study exploits CCA to achieve the following objectives: 3.1 Multiview Learning based Language Modeling CCA has been used vividly to analyze datasets with multiple views to learn shared latent structures between the views, enabling better integration of multimodal data. This study aims to leverage an extension of CCA, called Multiview Canonical Correlation Analysis (MCCA) [ 22 ] , to perform exploratory analysis on multi-view data [ 34 ] . Multi-view data refers to datasets where multiple sets of variables (views) describe the same set of entities. MCCA extends standard CCA by enabling the analysis of more than two sets of variables simultaneously, allowing for the exploration of correlations across multiple views or perspectives of the same data. By incorporating several views, MCCA can capture richer relationships within the data. To obtain utterance level representations that effectively integrate semantic and syntactic features, this study utilizes the different views of an utterance to better capture

Read the complete text