Is neural semantic parsing good at ellipsis resolution, or isn’t it?

Xiao Zhang, Johan Bos · 2025

Read Is neural semantic parsing good at ellipsis resolution, or isn’t it? by Xiao Zhang, Johan Bos (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00121.txt.

Introduction

Xiao Zhang and Johan Bos test whether computer programs that translate sentences into meaning can handle ellipsis, in this 2025 paper. Ellipsis lets a speaker drop a whole verb phrase, as in a short reply that borrows its meaning from the sentence before. Machines score above ninety percent on standard meaning tests, but this shortcut construction needs them to copy large chunks of meaning.

The authors built a test set of 120 ellipsis cases, each paired with its full meaning. Parsers that shone on the standard set failed on most of these. Adding more training examples of the same kind helped a good deal. The errors, they find, come less from copying itself than from the tricky sentence settings where ellipsis tends to appear.

Catalog record

CREATORXiao Zhang, Johan Bos
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE16 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-00121

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Is neural semantic parsing good at ellipsis resolution, or isn’t it? Xiao Zhang, Johan Bos Abstract Neural semantic parsers have shown good overall performance for a variety of linguistic phenomena, reaching semantic matching scores of more than 90%. But how do such parsers perform on strongly context-sensitive phenomena, where large pieces of semantic information need to be duplicated to form a meaningful semantic representation? A case in point is English verb phrase ellipsis, a construct where entire verb phrases can be abbreviated by a single auxiliary verb. Are the otherwise known as powerful semantic parsers able to deal with ellipsis or aren’t they? We constructed a corpus of 120 cases of ellipsis with their fully resolved meaning representation and used this as a challenge set for a large battery of neural semantic parsers. Although these parsers performed very well on the standard test set, they failed in the instances with ellipsis. Data augmentation helped improve the parsing results. The reason for the difficulty of parsing elided phrases is not that copying semantic material is hard, but that they usually occur in linguistically complicated contexts, causing most of the parsing errors. 1 Introduction Semantic parsing is the task of providing a formal meaning representation for an input sentence of a natural language such as English, Dutch, or Italian. Semantic parsing is crucial for applications that require the precise translation of unstructured data (i.e., text and images) into structured data (e.g., databases and robot commands). Currently, the most promising approaches to semantic parsing are based on neural models Bai et al. 2022 ; Wang et al. 2023 ; Zhang et al. 2024b ; Zhang et al. 2025 trained or fine-tuned on large semantically annotated corpora Banarescu et al. 2013 ; Abzianidze et al. 2017a , reaching high performance with F scores greater than 90%. Little is known about the ability of neural semantic parsers to cope with ellipsis , a linguistic construction in which elements are omitted and are supplied by the discourse context. In this paper, we will study how neural semantic parsers deal with Verb Phrase Ellipsis (VPE) in English. An example of a VPE is shown in (1) together with its fully expressed surface interpretation in (2). (1) Ann likes grapes, and Bea does, too. (2) Ann likes grapes, and Bea does like grapes , too. As this very simple example already demonstrates, ellipsis interpretation is a challenging task, for the only way to recover the elided material is to consider the discourse context. The (computational) linguistics literature abounds with many more complicated examples of VPE, including sloppy-strict interpretation of pronouns appearing in the elided material, cascaded ellipsis, antecedent contained deletion, gapping, and embedded ellipsis Dahl 1973 ; Williams 1977 ; Roberts 1989 ; Dalrymple et al. 1991 . Nevertheless, our aim is not to focus on these linguistically interesting examples carefully crafted by linguists, but rather to investigate how data-driven semantic parsers deal with instances of VPE found in corpora. As far as we know, this is the first in-depth study of VPE interpretation in neural semantic parsing. Related, but taking a different perspective, is work by Hardt 2023 , who found that large language models have difficulty processing ellipsis. In Section 2 we give an overview of earlier computational approaches to VPE. In Section 3 we introduce the Parallel Meaning Bank (PMB) and a VPE challenge test set distilled from the PMB. In Section 4 we outline our approach to enhance the semantic parsing for VPE, while in Section 5 the parsing results are presented, showing that neural approaches face a difficult time in interpreting elliptical constructions, even with substantial fine-tuning, but not for the reasons we initially thought would cause the difficulty. 2 Background VPE interpretation has drawn considerable attention in formal linguistics Dahl 1973 ; Sag 1976 ; Klein 1987 ; Dalrymple et al. 1991 . These early approaches can be summarized as identifying an antecedent verb phrase in the context, providing a logical form while abstracting over the subject, and applying the result to the subject noun phrase of the elided verb phrase. Computational approaches were introduced later Alshawi 1992 ; Kehler 1993 ; Bos 1994 ; Crouch 1995 ; Hardt 1997 , with the landmark paper by Dalrymple et al. 1991 introducing a set of benchmark VPE examples and a sophisticated algorithm based on higher-order unification to construct fully resolved meaning representations for elliptical phrases. These approaches, although computational of nature, still required external modules to identify the source verb phrase and the parallel elements between source and target phrase. Data-driven approaches based on annotated corpora Nielsen 2005 ; Bos and Spenader 2011 ; Bos 2016 demonstrated the large gap between theoretical ideas and practical implementations McShane and Babkin 2016 ; Kenyon-Dean et al. 2020 ; Zhang et al. 2019 , and were considered to be specific tasks rather than an integral part of wide-coverage semantic parsing. In this paper, we take a different computational perspective and depart with an overall well-performing general-purpose semantic parsing and investigate how well it succeeds on ellipsis data. 3 Data The Parallel Meaning Bank The PMB Abzianidze et al. 2017a is a multilingual corpus enriched with semantic annotations, covering a wide range of linguistic phenomena. It contains a substantial set of parallel texts, each paired with a formal meaning representation known as a Discourse Representation Structure (DRS) based on Discourse Representation Theory (DRT, kampreyle:drt ). While DRSs are typically presented in a human-readable box format, a clause-based linear representation was introduced by van Noord et al. 2018 to enable their use in sequence-based models. More recently, Bos 2023 proposed Sequence Box Notation (SBN), a simplified, variable-free version of DRS aimed at further facilitating sequence processing. In this paper we use SBN as meaning representation format (see Figure 1 ). Annotated VPE Instances As occurrences of VPE are relatively rare Bos and Spenader 2011 , it is rather challenging to yield a reasonably sized corpus. A total of 120 cases were identified in the PMB and their corresponding meaning representations manually corrected. Slightly more than half of the cases (71) contained some kind of negation in the elided construction (e.g., ”and neither am I”, ”Greenland is not”, ”but she didn’t”). Half of the instances are accompanied by the auxiliary verb to do , a third by to be , and the remaining cases are formed by other auxiliary verbs, the infinitival particle to or instances of gapping. An annotated example taken from the corpus is shown in Figure 1 , where the elliptical phrase ”life does” is semantically interpreted as ”life does end”. Figure 1: An example sentence with Verb Phrase Ellipsis and meaning representation in sequence notation and drawn as a directed acyclic graph. 4 Experimental Setup Training Sets For training our neural semantic parsers, we consider two settings: (1) the Standard Training Set , the default training data provided by PMB version 5.1.0, with all texts that are included in the VPE test set removed; and (2) the Augmented Training Set , an augmented dataset to enhance the model’s ability to handle verb phrase ellipsis. We construct the augmented dataset applying the following data augmentation strategies: • We employ GPT-4 to generate 600 sentence pairs, each

Read the complete text