Seeing the Poem: Image-Semantic Detection of AI-Generated Modern Chinese Poetry with MLLMs

Shanshan Wang Fengying Ye Hanjia Lyu Caiwen Gou Junchao Wu, Jingming Yao Chengzhong Xu Jiebo Luo Derek F. Wong · 2026

Read Seeing the Poem: Image-Semantic Detection of AI-Generated Modern Chinese Poetry with MLLMs by Shanshan Wang Fengying Ye Hanjia Lyu Caiwen Gou Junchao Wu, Jingming Yao Chengzhong Xu Jiebo Luo Derek F. Wong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22654.txt.

Introduction

Wang and others test whether language models can spot machine made modern Chinese poetry, a case prior detector studies had skipped. Their image semantic method pairs each poem with a picture of its content, then blends meaning, imagery and feeling from the image with the poem text into one judgment. The image guided detectors beat plain text baselines and even the best traditional detector, RoBERTa. A Gemini detector using the method reaches a Macro F1 score of 85.65 percent, a state of the art mark for this task.

Catalog record

CREATORShanshan Wang Fengying Ye Hanjia Lyu Caiwen Gou Junchao Wu, Jingming Yao Chengzhong Xu Jiebo Luo Derek F. Wong
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45798
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22654

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Seeing the Poem: Image-Semantic Detection of AI-Generated Modern Chinese Poetry with MLLMs Shanshan Wang Fengying Ye Hanjia Lyu Caiwen Gou Junchao Wu, Jingming Yao Chengzhong Xu Jiebo Luo Derek F. Wong Abstract Previous detection studies have shown that LLMs cannot be effectively used as detectors, but these studies have not addressed modern Chinese poetry. Moreover, no relevant research has explored the performance of LLMs in detecting modern Chinese poetry. This paper evaluates and enhances the performance of LLMs as detectors for modern Chinese poetry, and proposes an image-semantic guided poetry detection method. Compared with traditional detection approaches, our method innovatively incorporates images that reflect the content of the poetry. Through example-driven approaches, our method effectively integrates information such as meaning, imagery, and feeling from the image, then forms a complementary judgment with the poem text. Experimental results demonstrate that the LLM detectors based on our method outperform baseline detectors based on plain text, and even surpass the best-performing traditional detector, RoBERTa. The Gemini detector using our method achieves a Macro-F1 score of 85.65%, reaching the state-of-the-art level. The performance improvements of different LLM detectors on multiple LLMs-generated data prove the effectiveness of our method. † † footnotetext: ∗ Corresponding Author 1 Introduction The rapid advancement of Large Language Models (LLMs) has significantly improved the quality of AI-generated text, making it increasingly similar to human-written content Lai et al. 2024 . While this progress demonstrates notable technical achievements, it has also raised concerns about the growing difficulty in distinguishing AI-generated text from genuine human-written text Jakesch et al. 2023 ; Porter and Machery 2024 ; Hayawi et al. 2024 ; Najjar et al. 2025 . Figure 1: The relationship between human poetry and images. In this example, key nouns from the poem are underlined, and their corresponding visual elements are highlighted with yellow bounding boxes in the image. The blurring boundary between human and AI-generated text now poses a particularly pressing challenge, extending even into the domain of poetry Wang et al. 2025 . Poetry, as a precious part of human cultural heritage, carries deep historical and cultural significance, permeating various aspects of our lives comprehensively Ren et al. 2023 ; Zhang et al. 2019 . With a history spanning thousands of years, it has profoundly influenced the development of different nations and cultures Yi et al. 2018 ; Zhao et al. 2022 . Alarmingly, the total amount of AI-generated poetry has been predicted to exceed all human-composed poetry throughout history Huo 2020 . Figure 2: Comparison between the framework of the traditional detection method and our proposed IMAGINE method. In China, since the release of DeepSeek-R1 Guo et al. 2025 , the quantity of LLMs-generated poetry has grown rapidly. Some individuals now use LLMs to imitate the style of famous poets to generate modern Chinese poems and submit them to regular literary journals. This phenomenon creates considerable confusion and disruption for editors, readers, and even professional poets, as humans often struggle to distinguish AI-generated poems from those written by humans Köbis and Mossink 2021 ; Jakesch et al. 2023 ; Darda et al. 2023 . Therefore, the development of effective automatic detection methods has become not only necessary but also urgent Wu et al. 2025b . LLMs have been applied to various tasks and have demonstrated varying degrees of performance Lin et al. 2025 ; Ye et al. 2025 ; Lyu et al. 2025 ; Shen and Ying 2025 ; Lan et al. 2025a ; Lan et al. 2025b ; Shen et al. 2026 , making them promising as detectors for AI-generated text. Previous research in the NLP field on modern Chinese poetry was limited, mainly focusing on translation Song et al. 2023 ; Wang et al. 2024 and comprehension Wang et al. 2026 . A recent benchmark study Wang et al. 2025 has revealed that traditional text detectors such as Fast-DetectGPT Bao et al. 2023 , Binoculars Hans et al. 2024 , LRR Su et al. 2023 , Log-Rank Gehrmann et al. 2019 , Log-Likelihood Solaiman et al. 2019 , and RoBERTa-based classifiers Liu et al. 2019 , which perform well in general text domains, are not reliable for detecting AI-generated modern Chinese poetry. In fact, most detectors other than RoBERTa achieve less than 70% accuracy in detecting LLM-generated poems. In the real world, poetry often originates from the poet’s interaction with visual scenes, a principle summarized in the famous concept of “expressing emotions through scenery.” In other words, the poet sees the image first, and then creates the poem. Specifically, poetry is the artistic presentation of subjective emotions inspired by objective external objects Hu 2003 . Images have long served as a significant source of inspiration for Chinese poetry writing Cheng et al. 2018 ; Li et al. 2021 . Expressing emotions through scenery is a common method in poetry writing Wang 2003 ; Xiong 2001 . Poets often experience poetic feelings when seeing and contemplating natural scenery Li et al. 2021 , prompting them to write poems that express their emotional or aesthetic perceptions Xu et al. 2018 ; Cheng et al. 2018 . However, the essence of poetry lies not in describing the objective facts in an image, but in condensing the objects, scenes, and emotions within them to reveal the deeper meaning and poetic essence behind them Liu et al. 2018 . Images, in this context, carry rich information, extending beyond immediate visual details to encompass higher-level semantic themes Xu et al. 2018 . Take Figure 1 as an example, imagery often emerges across scattered lines and through seemingly simple nouns such as river, cliff, eagles, fish bones, and yellow leaves. They collectively form a coherent emotional landscape or artistic conception. This emotional core is rarely stated explicitly, but rather constructed through the subtle interplay of visuals, themes, and affect. Building on this poetically grounded generation setting where visual and emotional imagery plays a central role, rather than relying solely on textual analysis, models can benefit from visual semantics to better grasp the implicit meanings and emotional depth encoded in poetry. So we propose a novel image-semantic guided poetry detection framework based on Multi-modal Large Language Models (MLLMs) , termed IMAGINE , which integrates multimodal information to further refine LLM-based detection. Specifically, we first provide the model with paired examples of AI and human poems along with corresponding images reflecting the content of the poem. The model then receives both the poem under examination and a related image to guide its classification. This framework leverages image-semantic guidance to deepen the model’s understanding of poetic nuance and origin. Experimental results demonstrate that our image-semantic guided method yields substantial improvements in poetry detection task. Gemini, when enhanced with our approach, achieves a state-of-the-art Macro-F1 score of 85.65% across all LLM-generated and human-written poems. Our findings also reveal intriguing trends: different LLMs vary in their detection performance, suggesting that not all are equally suitable for this task. Furthermore, different categories of detectors exhibit varying performance when distinguishing between LLM-generated and human-written poems. Notably, our method significantly narrows the performance gap within LLM-based detectors. The consistent improvements observed across both overall and fine-grained metrics demonstrate the robustness and effectiveness of our image-semantic guided detection framework. Main contributions of our

Read the complete text