Handling Korean Out-of-Vocabulary Words with Phoneme Representation Learning

Nayeon Kim 1†, Eojin Jeon 1†, Jun-Hyung Park, SangKeun Lee · 2025

Read Handling Korean Out-of-Vocabulary Words with Phoneme Representation Learning by Nayeon Kim 1†, Eojin Jeon 1†, Jun-Hyung Park, SangKeun Lee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04018.txt.

Introduction

Kim, Jeon, Park, and Lee teach Korean word models to handle words they have never seen by listening to how the words sound. Korean writing mirrors speech closely, so the sound units of a word carry much of its meaning cues. Their 2025 framework, KOPL, blends a word's letter-based form with its sound-based form to build richer word vectors.

The module plugs into both fixed and context-based Korean word models without redesign. Across Korean language tasks, models with the module scored higher, beating the best prior model by about two percent on average. The authors release their code publicly. They argue sound-aware vectors are a cheap fix for the unknown-word problem.

Catalog record

CREATORNayeon Kim 1†, Eojin Jeon 1†, Jun-Hyung Park, SangKeun Lee
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-04018

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Handling Korean Out-of-Vocabulary Words with Phoneme Representation Learning Nayeon Kim 1†, Eojin Jeon 1†, Jun-Hyung Park, SangKeun Lee Abstract In this study, we introduce KOPL, a novel framework for handling K orean O OV words with P honeme representation L earning. Our work is based on the linguistic property of Korean as a phonemic script, the high correlation between phonemes and letters. KOPL incorporates phoneme and word representations for Korean OOV words, facilitating Korean OOV word representations to capture both text and phoneme information of words. We empirically demonstrate that KOPL significantly improves the performance on Korean Natural Language Processing (NLP) tasks, while being readily integrated into existing static and contextual Korean embedding models in a plug-and-play manner. Notably, we show that KOPL outperforms the state-of-the-art model by an average of 1.9%. Our code is available at https://github.com/jej127/KOPL.git . † † footnotetext: Equal contribution. * * footnotetext: Corresponding author. 1 Introduction Handling Out-of-Vocabulary words, which are the words unseen during training, remains a longstanding challenge, as they can degrade the performance of downstream tasks [ 8 , 14 , 29 ] . Various approaches have dealt with OOV words based on pre-training subword-level representation from scratch [ 2 , 9 , 18 ] or on encouraging learned representations for OOV words to be close to well-trained word embeddings [ 4 , 16 , 21 , 23 , 35 ] . However, since the formation of OOV words could be language-dependent, existing approaches focusing on English OOV words can struggle to address OOV words in other languages including Korean. In this paper, we focus on handling Korean OOV words by considering the linguistic property of Korean, the high correlation between phonemes and letters. Our work is aligned with the recent studies that have considered Korean-specific linguistic knowledge such as the writing system [ 9 , 10 , 18 ] or syntactic features [ 12 , 17 , 19 , 26 ] , to improve the performance of various Korean NLP tasks. Since the Korean letter Hangeul as a phonemic script [ 22 ] has a systematic internal structure correlated with the feature of phonemes, phonemes can contain useful information to understand Korean. Regarding Korean OOV words, this property of Hangeul makes us pay attention to the theory of psycholinguistics, the interactions between phonemes and letters in forming the meaning of words [ 3 ] . Figure 1 : Illustration of humans’ forming the word meaning. When encountering the unseen word “*마싯다[mAsitt’A] *dilisious ”, humans can recall its phoneme information “[mAsitt’A]” to infer the correct word “맛있다[mAsitt’A] delicious ”. For instance, Figure 1 shows one possible case where phoneme information plays a crucial role in understanding the meaning of words. When faced with “*마싯다[mAsitt’A] *dilisious ’’ 1 1 1 Asterisk(*) denotes ungrammatical words. , a misspelling of “맛있다[mAsitt’A] delicious ”, humans can recognize its spelling and subsequently recall its pronunciation. They can then infer “맛있다 delicious ”, the similarly pronounced word, as the correct word. Based on these observations, we introduce KOPL, a novel framework for handling K orean O OV words with P honeme representation L earning. KOPL learns representations of phonemes and words, and combines them to connect both types of representations with the word meaning. In addition, we propose two key strategies: phoneme-aware multimodal learning and cross-modal ensemble techniques. Experimental results demonstrate that KOPL outperforms the previous state-of-the-art model by an average of 1.9% across five Korean NLP downstream tasks featuring real-world OOV examples. KOPL can be seamlessly integrated into existing static and contextual Korean embedding models in a plug-and-play fashion. To the best of our knowledge, KOPL is the first work to leverage phoneme information to improve Korean OOV word embeddings. Although there have been works to utilize phoneme information to make NLP models robust to input perturbations, these methods require either pre-training the dedicated transformers on the large-scale corpora [ 28 , 30 ] , or the separate detection and correction of misspellings [ 13 , 15 , 34 ] . Other works use phoneme representations by themselves or combining them with speech representations to address the sound sequence alignment [ 27 ] or forced alignment problem [ 36 ] . Unlike these works, we focus on how to jointly utilize phoneme and text representations to address the OOV problem in Korean. In summary, we introduce KOPL, a novel framework to enhance Korean OOV word representations based on the linguistic property of Korean, the high correlation between phonemes and letters. We propose novel phoneme-aware multimodal learning and cross-modal ensemble strategies to effectively combine textual and auditory modalities in downstream tasks. Moreover, KOPL outperforms previous OOV methods on various Korean NLP tasks and can be employed in a plug-and-play manner to improve existing Korean embedding models. 2 Methodology Figure 2 : Illustration of KOPL. The meanings of inputs “맛있다” and “강아지 진짜 귀여움, 굿!” are “delicious” and “A puppy is really cute, good!”, respectively. (a) In the pre-training stage, KOPL jointly learns phoneme and word representations using linear interpolation. (b) In the fine-tuning stage, KOPL uses phoneme, word, and mixed representations as separate inputs to the downstream model. During inference for downstream tasks, we average the prediction scores obtained from each type of representation to make the final prediction. This section details our proposed KOPL. The core idea of KOPL is to learn phoneme representations for words that may not be present in a vocabulary (Section 2.1 ), and then to use these representations to fine-tune the models (Section 2.2 ). An overview of KOPL is illustrated in Figure 2 . 2.1 Pre-training Phoneme Representations In this section, we describe how to pre-train phoneme representations. First, we consider a vocabulary \mathcal{V} with a fixed size, and the pre-trained word embedding \mathbf{v}_{w}\in\mathbb{R}^{d} for each word w\in\mathcal{V} . We aim to learn phoneme and word representations that not only capture the unique characteristics of phonemes and words but also contain complementary information present in both. To this end, we jointly train two neural networks F_{P} and F_{W} , which output phoneme and word representations as separate d -dimensional vectors, respectively. We use a self-attention module with positional embeddings [ 32 ] for both neural networks F_{P} and F_{W} . The input p_{w} for F_{P} is a list of phonemes corresponding to the word w , 2 2 2 We use the hangul_to_ipa program ( https://github.com/stannam/hangul_to_ipa ) to obtain phoneme sequence. which is constructed by International Phonetic Alphabet (IPA) symbols [ 1 ] . Furthermore, F_{W} takes a mixed sequence of subcharacters and morphemes constituting w , which is the same as the input method for the Korean version of LOVE [ 4 ] detailed in Table 2 . To ensure that the two types of representation contain complementary information in phonemes and words, we seek to fuse them during pre-training. To this end, we interpolate the phoneme representation F_{P}(p_{w}) with the word representation F_{W}(w) : F(w)=\lambda F_{P}(p_{w})+(1-\lambda)F_{W}(w), (1) where 0\leq\lambda\leq 1 is a hyperparameter to derive the mixup ratio of each representation. The advantage of the interpolation is to balance the weights between phoneme and word representations, enabling their effective combination. While some words have similar meaning and pronunciation but different spellings, others have similar pronunciation 3 3 3 In

Read the complete text