Comparable Corpora: Opportunities for New Research Directions

Kenneth Church · 2025

Read Comparable Corpora: Opportunities for New Research Directions by Kenneth Church (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-14721.txt.

Introduction

Church calls peers to widen what paired text banks can do in this twenty twenty-five talk paper. Such banks hold like texts in two tongues with no line match. Most work mines them for word pairs and flat drill files.

Inside, the talk tours the banks' past from their first builds on. Fresh paths then fan out for peers to take up and test. He frames the banks as a field ripe for new hands and aims. The piece is a key talk text from a shop meet that year. It asks the field to think past stock uses of the banks.

Catalog record

CREATORKenneth Church
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-14721

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Comparable Corpora: Opportunities for New Research Directions Kenneth Church Abstract Most conference papers present new results, but this paper will focus more on opportunities for the audience to make their own contributions. This paper is intended to challenge the community to think more broadly about what we can do with comparable corpora. We will start with a review of the history, and then suggest new directions for future research. 1 Introduction The success of chat bots in many languages demonstrates the power of comparable corpora (CC) and pivoting via English. We will start with a review of the history of CC, and then suggest new directions for future research: 1. More depth: CC are normally used for simple tasks such as bilingual lexicon induction (BLI), but CC can be used for much more interesting views of lexical semantics. 2. Compare and Contrast: CC are normally used to make simple comparisons over language pairs, but they can be used for contrasts as well as comparisons (in monolingual settings as well as multilingual settings). 3. More modalities: Now that vectors encode everything (text in many languages, pictures, audio, video), we can compare and contrast everything with everything. 4. Bursting filter bubbles: bots made in America are trained on corpora from an American perspective with American biases. We should not impose American values on others. 2 Historical Background Resource Cites Europarl Koehn 2005 4634 OPUS Tiedemann 2012 2255 HF: Helsinki-NLP/opus-100 Resnik and Smith 2003 848 MultiUN Eisele and Chen 2010 327 HF: Helsinki-NLP/multiun Bible Pratap et al. 2024 254 Akerman et al. 2023 HF: Flux9665/BibleMMS LDC: Hansard French/English LDC: Hong Kong Hansards HF: NilanE/ParallelFiction-Ja_En-100k HF: sentence-transformers/parallel-sentences HF: tiagoblima/bible-ptbr-gun-gub-aligned HF: dsfsi/vukuzenzele-sentence-aligned Marivate et al. 2023 Table 1: Examples of parallel corpora English French Sense bank banque money banc river duty droit tax devoir obligation drug médicament medical drogue illicit land terre property pays country language langue medium langage style position position place poste job sentence peine judicial phrase grammatical Table 2: Using Hansards for Word Sense Disambiguation (WSD), based on Table 2 in Gale et al. 1992 2.1 Parallel Corpora Table 1 shows some examples of parallel corpora. HF and LDC in Table 1 refer to HuggingFace 1 1 1 https://huggingface.co/ and the Linguistic Data Consortium, 2 2 2 https://www.ldc.upenn.edu/ respectively. More parallel corpora can be found on HF by searching for parallel , aligned and translation . The main application of parallel corpora has been machine translation Brown et al. 1993 . Shannon’s noisy channel model Shannon 1948 was originally motivated for applications in communication (telephones), but it has been used for many other applications including machine translation. That is, to translate from English to French, one imagines that French speakers think in English, like English speakers do, but for some reason, when French speakers talk, the noisy channel converts their English to French. E\rightarrow Noisy~Channel\rightarrow F (1) The task of the translation system is to recover the original English, E , from the observed French, F . These days, it has become standard practice to use neural networks for translation, but it used to be popular to use Hidden Markov Models (HMMs) to find the most likely English, \hat{E} , based on a prior (language model), Pr(E) , and a bilingual dictionary, Pr(F|E) . \hat{E}=\mathrm{argmax}_{E}Pr(E)Pr(F|E) (2) Much of the discussion below will focus on the bilingual lexicon, Pr(F|E) . Pr(E) , the language model in Eqn ( 2 ), is relatively well estimated because we can re-use monolingual LLMs that have been developed for other applications. The bilingual lexicon, Pr(F|E) , assigns probabilities to all sequences of English, E , and French, F . It was standard practice, at least at first, to estimate Pr(F|E) from parallel corpora such as the English-French Canadian Hansards. The rest of this section on history will largely focus on the lexicon. After introducing comparable corpora as an alternative to parallel corpora, we will motivate WSD (word-sense disambiguation). Much of the research on WSD started with bilingual word-senses, but it should be noted that word-senses are different in monolingual and bilingual dictionaries. There has also been considerable work on transferring monolingual lexical resources such as WordNet and VAD to more languages. Unfortunately, much of this work uses translation to pivot out of English in inappropriate ways, as we will see. This section will end with a review of BLI (bilingual lexicon induction). The BLI literature uses more modern methods in machine learning than previous methods for inducing lexicons from CC, but BLI benchmarks (such as MUSE) may not be as effective as older WSD methods for addressing classic challenges with translations of ambiguous words. A classic example is bank , which is translated as banque and banc in the Canadian Hansards, depending on the sense. Unfortunately, this ambiguity is not captured in the MUSE benchmark where bank translates to banque (but not banc ). In the reverse direction, MUSE has translations for both banque and banc , but they translate to different English words, bank and bench , respectively. Comparisons of ambiguities in Hansards ( Table 2 ) and MUSE ( Table 5 and Table 6 ) suggest that MUSE is not testing WSD as much as the older literature. Another concern with MUSE is that most words in the benchmark translate to themselves. These concerns suggest that there may be room to introduce a new benchmark that would make a stronger case for comparable corpora (CC). After discussing history, the next section will discuss more radical challenges for the future: lexical semantics, transfer learning, filter bubbles and connections between academic search and CC. 2.2 Comparable Corpora (CC) The term, comparable corpora , was introduced in Fung and Church 1994 ; Rapp 1995 ; Fung and Yee 1998 ; Fung 2000 to address limitations with parallel corpora. Parallel corpora are available for a few genres such as parliamentary debates (Hansards) and religion (Bible), as shown in Table 1 . Since most texts and most genres are not translated, we can collect larger and more diverse corpora if we relax the restriction on translation. CC replace a single parallel corpus with two monolingual corpora, ideally on similar (comparable) topics. 2.3 Word-Sense Disambiguation (WSD) In addition to machine translation applications mentioned above, parallel corpora have also been used to disambiguate ambiguous words such as bank , as illustrated in Table 2 . Bar-Hillel 1960 thought machine translation was impossible when he could not figure out how to disambiguate words such as those in Table 2 . It was obvious that the translation depends on a solution to WSD. Gale et al. 1992 used this argument in reverse to obtain large quantities of labeled text for WSD research. They used parallel corpora such as Hansards to find instances of ambiguous words such as bank , and use the French translations to label each instance of bank as either “money” sense or “river” sense. After labeling the English in this way, they threw away the French and used the sense-labeled text to train and test machine learning methods for WSD. 2.4 Monolingual Senses != Bilingual Senses This approach was successful in reviving interest in WSD research, though it should be mentioned that bilingual lexicography is different from monolingual lexicography. Consider the

Read the complete text