Enhancing Dialogue Systems with Discourse-Level Understanding Using Deep Canonical Correlation Analysis
Akanksha Mehndiratta, Krishna Asawa · 2025
Read Enhancing Dialogue Systems with Discourse-Level Understanding Using Deep Canonical Correlation Analysis by Akanksha Mehndiratta, Krishna Asawa (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-09094.txt.
Introduction
Akanksha Mehndiratta and Krishna Asawa teach chat tools to hold the thread of a long talk, in a study from 2025. Most bots grade each line near alone, so old lines fade and picks go off-thread. The pair aim at talk-level sense: what a line means in the whole flow.
Their frame learns talk marks through a deep link tool that ties lines to their kin round them. Marks that earn their keep stay; chaff is sifted out. On a big Ubuntu help-chat bank, pick scores rose with clear room. Hold the thread, the pair shows, and the bot picks the line a human would.
Catalog record
| CREATOR | Akanksha Mehndiratta, Krishna Asawa |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-09094
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhancing Dialogue Systems with Discourse-Level Understanding Using Deep Canonical Correlation Analysis Akanksha Mehndiratta, Krishna Asawa Abstract The evolution of conversational agents has been driven by the need for more contextually aware systems that can effectively manage dialogue over extended interactions. To address the limitations of existing models in capturing and utilizing long-term conversational history, we propose a novel framework that integrates Deep Canonical Correlation Analysis (DCCA) for discourse-level understanding. This framework learns discourse tokens to capture relationships between utterances and their surrounding context, enabling a better understanding of long-term dependencies. Experiments on the Ubuntu Dialogue Corpus demonstrate significant enhancement in response selection, based on the improved automatic evaluation metric scores. The results highlight the potential of DCCA in improving dialogue systems by allowing them to filter out irrelevant context and retain critical discourse information for more accurate response retrieval. 1 Introduction Dialogue systems, such as chatbots or virtual assistants, have made substantial progress in generating contextually appropriate responses. However, these systems face a persistent challenge in maintaining coherence and relevance across multiple turns in longer conversations. This is especially difficult when the context becomes complex, with numerous topics, nuanced references, or shifting conversational goals. With the objective of enhanced language modeling, such models often struggle to effectively utilize the entire discourse history, leading to responses that may be locally appropriate but globally inconsistent or irrelevant [ 8 ] The core issue is how dialogue systems manage and interpret discourse history. Current models typically rely on the immediate context (e.g., the last few utterances) to generate responses, which can lead to a loss of important information from earlier in the conversation. This limitation becomes more pronounced in longer dialogues, where the context is spread across many turns and may involve intricate dependencies between utterances. To address this challenge, this study presents a new framework that leverages Deep Canonical Correlation Analysis (DCCA). DCCA is a technique for modeling relationships between two sets of variables by projecting them into a shared latent space. In this context, DCCA is applied to model the relationship between an utterance and its surrounding turns (i.e., the discourse history). The key idea is to create a shared latent representation that captures the essential elements of the conversation while filtering out irrelevant or redundant information. This allows the model to build a discourse-level understanding of the conversation, enabling it to focus on the most relevant context and select coherent and contextually appropriate responses. This proposed framework enhances coherence by modeling the relationship between an utterance and its discourse history, ensuring a consistent conversational thread even in extended dialogues [ 10 ] . It also improves relevance filtering by leveraging a shared latent space to prioritize essential context while discarding irrelevant information, resulting in more focused responses. Furthermore, the method is highly scalable, effectively managing complex and varied contexts, making it well-suited for longer conversations. The proposed framework is evaluated on the Ubuntu Dialogue Corpus, a widely used benchmark in dialogue system research. This corpus consists of multi-turn technical support conversations, making it an ideal testbed for evaluating the ability to maintain coherence and relevance over extended dialogues. The experiments demonstrate that the framework achieves significant improvements in automatic evaluation metrics for response selection tasks. 2 Related Work Retrieval-based dialog systems maintain conversations by choosing the most appropriate response from a set of candidates, using the conversation history and the user’s latest input as context [ 16 , 19 , 20 , 5 ] . Typically, these systems encode both the conversation and the potential responses as vectors within a shared semantic space and use a classifier to evaluate how well they match [ 7 , 16 , 18 ] . Data-driven dialogue systems generally follow two main approaches: (1) combining all context utterances into a single input [ 7 , 6 ] , or (2) processing each utterance individually and then aggregating the results [ 16 , 20 , 14 ] . Among these methods, RNN-based architectures are commonly used for response selection [ 7 , 6 , 19 , 15 ] . Early single-turn models, such as RNN, CNN, LSTM, BiLSTM, MV-LSTM, and Match-LSTM, treated the entire context as a single document to compute matching scores with candidate responses. More recent approaches focus on multi-turn matching between context and response [ 16 , 20 ] . To better capture interactive information, attention mechanisms have been employed to model relationships across multiple utterances. For instance, Zhang et al. [ 18 ] used self-matching attention to extract key information from each utterance, while Zhou et al. [ 20 ] applied stacked self-attention and cross-attention to represent utterances in a response-aware manner. Tao et al. [ 13 , 14 ] enhanced context-to-response matching by stacking multiple interaction blocks. Recent advancements aim to capture matched information at varying granularities, achieving state-of-the-art performance in multi-turn response selection [ 3 , 13 , 17 ] . In parallel, deep contextualized pre-trained language models (PTMs) have revolutionized NLP by learning universal language representations, significantly improving performance across various tasks. Notable examples include ELMo, GPT, BERT, RoBERTa, XLNet, ALBERT, and ELECTRA. These models differ in their approaches to masking strategies, knowledge injection, language modeling objectives, and efficiency optimization. Despite their success, the potential of PTMs remains underexplored, particularly in adapting them to dialogue systems [ 4 , 11 ] . Limited research has focused on tailoring PTMs to the unique challenges of conversational AI, leaving significant room for innovation in this area. 3 Deep Canonical Correlation Analysis (DCCA) Deep Canonical Correlation Analysis (DCCA) [ 1 ] is a powerful technique that aims to learn a shared representation of two sets of variables by maximizing their correlation in a common latent space. Traditionally used in the domain of unsupervised learning, DCCA can be extended to handle multi-modal data, aligning features from different domains. By learning representations that capture the relationships between two sets of inputs, DCCA helps uncover hidden dependencies between variables, which is especially valuable in dialogue systems [ 2 , 12 ] . With an intent of performing discourse understanding, this study employs DCCA to learn a shared latent space between the utterance and its surrounding context (previous and subsequent turn). This shared representation allows the model to better understand the relationship between an utterance and the broader dialogue history, thereby improving the relevance of the selected response. The key benefit of DCCA in dialogue systems is its ability to map an utterance and its contextual surroundings into a space where their correlations can be maximized. This ensures that the model can distinguish between relevant and irrelevant knowledge from a given context, significantly improving its response selection capabilities. 3.1 Feature Extraction Using Deep Canonical Correlation Analysis In the traditional design of conversational models, the hidden states generated by recurrent neural networks (RNNs) or transformers are often used to represent the contextual information of a dialogue. However, these hidden states alone may not effectively capture all the inter-relationships that are crucial for understanding discourse at a deeper level. By applying Canonical Correlation Analysis (CCA) to the hidden states, we can extract features that better represent the interdependencies between an utterance and its dialogue history [ 9 ] . In our approach, we utilize DCCA to align