A Task-Oriented Evaluation Framework for Text Normalization in Modern NLP Pipelines
Md Abdullah Al Kafi, Raka Moni, Sumit Kumar Banshal · 2025
Read A Task-Oriented Evaluation Framework for Text Normalization in Modern NLP Pipelines by Md Abdullah Al Kafi, Raka Moni, Sumit Kumar Banshal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-20409.txt.
Description
Kafi, Moni, and Banshal warn that shrinking words can silently destroy meaning. Their 2025 paper evaluates text normalization in language processing pipelines. Stemming chops words down to a root form. It reduces vocabulary size and can speed up tasks. But judges rarely check whether aggressive stemming harms meaning. The team built a three-part evaluation framework.
One score measures stemming effectiveness through word reduction. A second tracks downstream task performance changes. A third measures meaning distance between stemmed and original words. Two stemmers were compared: Bangla's BNLTK and English's Snowball. The Bangla tool scored 1.67 on effectiveness, the highest mark recorded. Its compression ratio reached 1.90 across the test corpus used. Yet its meaning distance of 0.26 in the same tests exposed harmful over-stemming. Downstream performance dropped accordingly in those runs. Snowball scored 1.31 on effectiveness with a safer distance overall, 0.14 in the final comparison table. The lesson: check meaning safety before chasing compression.
Catalog record
| CREATOR | Md Abdullah Al Kafi, Raka Moni, Sumit Kumar Banshal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-20409