SCRIPT : A Subcharacter Compositional Representation Injection Module for Korean Pre-Trained Language Models
SungHo Kim, Juhyeong Park, Eda Atalay, SangKeun Lee · 2026
Read SCRIPT : A Subcharacter Compositional Representation Injection Module for Korean Pre-Trained Language Models by SungHo Kim, Juhyeong Park, Eda Atalay, SangKeun Lee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12377.txt.
Introduction
SungHo Kim and three colleagues tackle Korean language models, which usually split text into subwords. Subword splitting misses how each Korean character is built from smaller jamo units. Their SCRIPT module injects that subcharacter structure into existing models. It needs no architecture change and no retraining from scratch. SCRIPT lifts every baseline tested on Korean understanding and generation. It also reshapes embeddings to capture grammar patterns more cleanly.
Catalog record
| CREATOR | SungHo Kim, Juhyeong Park, Eda Atalay, SangKeun Lee |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89713 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.12377
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SCRIPT : A Subcharacter Compositional Representation Injection Module for Korean Pre-Trained Language Models SungHo Kim, Juhyeong Park, Eda Atalay, SangKeun Lee Abstract Korean is a morphologically rich language with a featural writing system in which each character is systematically composed of subcharacter units known as Jamo. These subcharacters not only determine the visual structure of Korean but also encode frequent and linguistically meaningful morphophonological processes. However, most current Korean language models (LMs) are based on subword tokenization schemes, which are not explicitly designed to capture the internal compositional structure of characters. To address this limitation, we propose SCRIPT , a model-agnostic module that injects subcharacter compositional knowledge into Korean PLMs. SCRIPT allows to enhance subword embeddings with structural granularity, without requiring architectural changes or additional pre-training. As a result, SCRIPT enhances all baselines across various Korean natural language understanding (NLU) and generation (NLG) tasks. Moreover, beyond performance gains, detailed linguistic analyses show that SCRIPT reshapes the embedding space in a way that better captures grammatical regularities and semantically cohesive variations. Our code is available at https://github.com/SungHo3268/SCRIPT . 1 Introduction In human writing systems, the grapheme, the smallest unit of written language that encodes linguistic information, plays a crucial role in shaping how meaning is represented and processed 7 ; 39 ; 10 . In many alphabetic systems, such as English, graphemes typically correspond to atomic letters (e.g., a, b, c), and words are formed through linear combination (e.g., “cat” consists of c, a, and t). However, not all alphabetic systems operate in such a linear manner, nor do they necessarily treat characters as minimal units of written composition. Korean, in particular, employs a unique featural writing system, Hangul , in which each character is a structured composition of smaller subcharacter units known as Jamo. As illustrated in Figure 1 , each character consists of three Jamo units: Choseong (initial consonant), Jungseong (vowel), and Jongseong (final consonant), following fixed spatial arrangements and a strict compositional order. These principles were explicitly defined in Hunminjeongeum 1 1 1 Hunminjeongeum explains the letter design and well-formed combinations; modern encoding schemes and keyboard input sequences arise from contemporary standards. 32 ; 33 , the historical document detailing the invention principles of Hangul , including the design and combination rules for the subcharacters. Figure 1: (a) Examples of the components of Hangul . This figure illustrates two characters, such as ‘춥 {}_{\text{cold}} ’ and ‘다 {}_{\text{ending suffix}} ’, with each subcharacter highlighted in blue . (b) Examples of linguistic phenomena arising from the inflection of predicate ‘춥다 {}_{\text{be cold}} ’ at the subcharacter-level, with the transformed subcharacters highlighted in red . Crucially, this compositional structure is not merely orthographic. As a morphologically rich and agglutinative language, Korean exhibits extensive morphophonological alternations across morpheme boundaries 26 ; 28 ; 17 . Predicate inflection, for example, often triggers systematic subcharacter-level alternations, such as the addition of the final consonant ‘ㅆ’ to mark past tense or ‘-ㅁ’ for nominalization, as shown in Figure 1 . Additionally, phonological assimilation between adjacent syllables frequently alters subcharacters to facilitate natural pronunciation 43 ; 41 . These phenomena highlight that subcharacter-level features in Korean are tightly linked to grammatical, semantic, and morphophonological functions 24 . Despite this linguistic reality, most contemporary Korean PLMs, including advanced off-the-shelf LLMs 50 ; 27 , rely almost exclusively on subword-based tokenization. While subword modeling effectively captures lexical semantics from large corpora, it struggles to reflect Hangul ’s compositional structure, limiting sensitivity to fine-grained morphosyntactic variations 2 ; 20 . In contrast, a few subcharacter-based LMs 30 ; 5 ; 21 show strong robustness to such variations but often underperform on downstream tasks due to weaker semantic representations and increased computational cost. To leverage complementary strengths, we propose SCRIPT , a lightweight, plug-and-play module that injects subcharacter-level structural knowledge directly into existing subword-based PLMs. SCRIPT attaches to the embedding layer of a PLM and operates through a dual-channel strategy. It compresses subcharacter sequences into structure-aware subword representations grounded in Hangul ’s compositional principles, and then fuses them with the PLM’s original subword embeddings. This design enables the model to capture fine-grained subcharacter compositionality while preserving the rich semantic information learned from large-scale corpora, without modifying the PLM architecture or requiring additional pre-training. The main contributions of this work are summarized as follows: • We empirically show that most Korean morphological variations occur at the subcharacter-level, motivating subcharacter-aware modeling. • We introduce SCRIPT , a model-agnostic module that injects structure-aware subcharacter compositional representations into existing PLMs via embedding-level integration. • We show that SCRIPT improves performance across a wide range of Korean NLU and NLG benchmarks, while effectively capturing key morphosyntactic phenomena. Figure 2: Morphological modifications in a large-scale Korean POS-tagged corpus: the left panel distinguishes subcharacter- and character-level MOD cases, and the right lists the ten most frequent MOD types with raw target characters and corresponding factors. 2 Motivation In this section, we present our empirical observations on the pervasiveness of diverse morphophonological changes at the subcharacter-level in real Korean usage, highlighting the importance of modeling the subcharacter structure of Hangul . 2.1 Setup Specifically, we conduct a large-scale corpus-based analysis quantifying their frequency. We used the Korean Part-of-Speech Tagged Corpus 2 2 2 Part-of-Speech Tagged Corpus (v1.1) from ModuCorpus , provided by the National Institute of Korean Language (2020) , which contains 3M words annotated with morpheme-level and POS information. This corpus provides a solid basis for estimating the frequency of subcharacter-level alternations in real usage. To systematically capture these alternations, we adopted a simple annotation scheme, following 28 , which marks how each character relates to its base form (lemma). Each character is assigned to one of three categories: • KEEP : unchanged with respect to the base form. • MOD : modified from the base form. • NOOP : omitted in the base form. For example, in ‘했다 {}_{\text{did}} ’, whose base form is ‘하다 {}_{\text{do}} ’, the segment ‘했 {}_{\text{did}} ’ is labeled as MOD because it reflects a tense change arising from the combination of the verb stem ‘하’ and the past tense marker ‘-었-’, which undergoes phonological contraction (‘하 + 었 → 했’). In contrast, ‘다’ is labeled as KEEP since it remains unchanged. In this paper, we focus our analysis on characters tagged as MOD , as they directly encode morphological alternations. Each MOD character is further classified by its level of granularity 3 3 3 Detailed tagging procedures are provided in Appendix B : • Subcharacter-level MOD : when only part of a character is altered. (e.g., the change from ‘하’ to ‘한’, adding the final consonant ‘ㄴ’). • Character-level MOD : when the entire character is changed into a different character. (e.g., ‘이’ is replaced with ‘라’). 2.2 Observation As shown in Figure 2 , the overwhelming majority of characters tagged as MOD (92.75%) involved subcharacter-level modifications, while only a small fraction (7.25%) represented character-level changes. These findings underscore the importance of modeling subcharacter-level alternations for a deeper and more comprehensive understanding of Korean, especially in adapting to diverse usage and morphological variation. This aligns with prior work 21 ; 25 , which observed that jamo-based language modeling demonstrates robustness in handling character-level conjugation changes and exhibits strong performance