Revisiting Absence with Symptoms that *T* Show up Decades Later to Recover Empty Categories
Emily Chen Nicholas Huang Casey Robinson Kevin Xu, Zihao Huang Jungyeul Park · 2024
Read Revisiting Absence with Symptoms that *T* Show up Decades Later to Recover Empty Categories by Emily Chen Nicholas Huang Casey Robinson Kevin Xu, Zihao Huang Jungyeul Park (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01109.txt.
Introduction
Emily Chen and co-authors work on restoring silent elements in parsed sentences across English, Chinese, and Korean, in a 2024 paper. Such null elements carry real grammar and meaning, yet parsers usually strip them out. Restoring them helps machines read structure correctly.
A rule-based method that uses language context, once built for English only, was extended to Chinese and scored on par with past results. A neural sequence model was also trained for all three languages, reaching higher scores with added function labels. The authors say this is the first side-by-side study of null elements in three languages. Both routes show the gaps can be filled with fair accuracy.
Catalog record
| CREATOR | Emily Chen Nicholas Huang Casey Robinson Kevin Xu, Zihao Huang Jungyeul Park |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-01109
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Revisiting Absence with Symptoms that *T* Show up Decades Later to Recover Empty Categories Emily Chen Nicholas Huang Casey Robinson Kevin Xu, Zihao Huang Jungyeul Park Abstract This paper explores null elements in English, Chinese, and Korean Penn treebanks. Null elements contain important syntactic and semantic information, yet they have typically been treated as entities to be removed during language processing tasks, particularly in constituency parsing. Thus, we work towards the removal and, in particular, the restoration of null elements in parse trees. We focus on expanding a rule-based approach utilizing linguistic context information to Chinese, as rule based approaches have historically only been applied to English. We also worked to conduct neural experiments with a language agnostic sequence-to-sequence model to recover null elements for English (PTB), Chinese (CTB) and Korean (KTB). To the best of the authors’ knowledge, null elements in three different languages have been explored and compared for the first time. In expanding a rule based approach to Chinese, we achieved an overall F1 score of 80.00, which is comparable to past results in the CTB. In our neural experiments we achieved F1 scores up to 90.94, 85.38 and 88.79 for English, Chinese, and Korean respectively with functional labels. nanummj 1 Introduction Null elements, or empty categories, are elements that exist within a parse tree that do not correspond to a surface level word. Chomsky outlined how certain elements could be invisible or null in surface structure ( Chomsky, 1981 ) . These null elements serve an important role in syntax, especially in explaining movement and transformational grammars ( Chomsky, 1965 ) . In the context of dependency parsing, particularly in the context of predicate elision, null elements have predominantly surfaced in the enhanced representation of universal dependencies. 1 1 1 https://universaldependencies.org/u/overview/enhanced-syntax.html Such elements can encode additional semantic information of sentences by recording non-local dependencies, enabling interpretation of constructions such as WH-questions ( Johnson, 2002 ) . Conversely, within constituency parsing, null elements have typically been treated as preprocessed entities to be removed ( Collins, 1999 ; Bikel, 2004 ) . However, excluding null elements can be problematic when attempting to parse more than just the syntactic information from a surface sentence, since the semantic information they provide is often lost. Certain languages, such as Japanese, Korean, and Chinese, more readily omit specific arguments within sentences compared to English when these arguments have been previously referenced or can be inferred. As an example, when attempting machine translation from pro-drop languages such as Chinese and Korean, where pronouns are often dropped, into English, problems arise because English pronouns must be inferred from non-existent words to preserve meaning ( Chung and Gildea, 2010 ) . Therefore, recovering null elements from a parsed tree without traces may enable us to resolve null anaphora, where omitted pronouns or other referential expressions are understood from context. Outside of practical applications, null elements have a rich history in syntax, and are an essential aspect of explaining transformational grammars. By restoring null elements, parse trees may better reflect a linguistic understanding of movement, as well as many other linguistic phenomena relevant to the use of null elements. Previous work has approached the recovery of null elements in different ways, ranging from more rule based approaches to statistical approaches. For example, Johnson (2002) introduced a pattern-matching algorithm aimed at recovering empty nodes within the Penn English Treebank (PTB) ( Marcus et al., 1993 ) . Their exploration involved scrutinizing parsing outcomes for various null elements such as *T* (WH trace) and *U* (empty units), demonstrating their potential to enhance parsing results. Campbell (2004) proposed a linguistically oriented approach grounded in Government-Binding theory ( Chomsky, 1981 ) , which forms the basis of syntactic annotation in treebanking. Despite the inclination towards a rule-based method for empty category retrieval, reported parsing outcomes have been less compelling ( Campbell, 2004 , p.650) . Pattern matching algorithms have also been utilized to restore Restoring null elements for Chinese and Korean using a similar algorithm to Johnson, but included more context in the generated patterns to better recover null elements in Chinese, since Chinese contains null elements (*PRO* and *pro*) which have similar syntactic contexts so need wider patterns to be effectively recovered. ( Chung and Gildea, 2010 ) . In contrast, Levy and Manning (2004) introduced an algorithm based on statistical features for reconstructing non-local dependencies concerning null elements. Non-local dependencies are syntactic constructs where two words cannot be obviously related based on typical syntactic rules – null elements that are indexed to the dependant word are a way to represent non-local dependencies using existing syntactic rules. Previous work on null element restoration has primarily focused on English – we hope to expand the scope of studies by implementing algorithms for null element recovery for English, Chinese, and Korean. Furthermore, we aim to take a a comparative approach by implementing linguistically-motivated algorithms for removing and restoring null elements, as well as expanding to a language agnostic neural experiments. Considering the general success of neural experiments in outperforming traditional statistical parsers, applying neural experiments to null element recovery may allow for greater accuracy than previous approaches. 2 Previous Work After being introduced by Chomsky, null elements have become a standard feature of syntax and have been a topic of further exploration. Reeves (1992) discusses in depth the legitimacy and theory behind null elements in a purely theoretical context by generating a framework in which null elements are not allowed and testing if such a framework fails. Reeves concludes that while sometimes disruptive to work with, null elements are ultimately necessary for systematically representing and analyzing the English language. Previous works on the computational approach to restoring null elements generally choose one of three methods for inserting null elements; pre-processing, in-processing, and post-processing. Such previous works and their respective approaches are noted in Table 1 . We utilize the post-processing approach. In this section we give a broad overview of each approach and justify our decision to use post-processing. As noted in Table 1 , Dienes and Dubey (2003a) inserted null elements as a step before parsing (thus, preprocessing). However, preprocessing approaches to null element insertion have been linked to worse results for general parsing than with post-processing approaches ( Johnson, 2002 ) . Advantages of pre-processing versus post-processing or in-processing include runtime efficiency. ( Dienes and Dubey, 2003a ) . Other work utilized in-processing, making this the most popular approach from previous attempts of inserting null elements ( Dienes and Dubey, 2003b ; Schmid, 2006 ; Kato and Matsubara, 2016 ; Hayashi and Nagata, 2016 ; Kummerfeld and Klein, 2017 ) . In in-processing, null elements are integrated into the parsing algorithm. Generally, in-processing results from previous work are relatively similar to post-processing results. However, in-processing is a more complex task than post-processing, and is less conducive to a rules based approach, since a rule-based approach to null element recovery uses existing syntactic structure to predict the position of null elements. Finally, Johnson (2002) and Campbell (2004) utilized post-processing, during which null elements are inserted into the parse tree. Results from post-processing are favourable to some in-processing results Campbell (2004) . Furthermore, post-processing avoids problems where the null element is