Application of deep learning for transformation of Chinese traditional cultural narrative patterns and enhancement of cultural identity empowered by AIGC Authors: Yan Liu, Hongmei Wang Published: 2025 ABSTRACT This study aims to achieve controllable generation of Chinese traditional cultural narrative content and enhance cultural identity. First, it constructs a tri-modal generation framework of text-image-style based on Stable Diffusion v2.1 and Contrastive Language-Image Pretraining (CLIP) models, realizing the joint modeling of traditional cultural semantics and visual imagery. Second, the study introduces the Low-Rank Adaptation (LoRA) mechanism to embed traditional cultural style features in a lightweight manner, improving the model’s style adaptability under small sample conditions. Finally, a three-level evaluation system of “generation quality-semantic consistency-cultural identity” is built, covering both objective indicators and user feedback, to systematically verify the model’s performance. Results show that the proposed model significantly outperforms existing methods in multiple dimensions: in terms of image quality, the Fréchet Inception Distance (FID) is 22.85, the Learned Perceptual Image Patch Similarity (LPIPS) is 0.298, and the style recognition accuracy reaches 86.4%. Regarding narrative consistency, the Bilingual Evaluation Understudy (BLEU) score is 0.325, the CLIP text-image similarity is 0.793, and the Narrative Style Match is 82.3%. On the cultural perception level, the average user narrative resonance is 4.32 points, the imagery accuracy score is 0.748, and the question-answer task pass rate is 82.6%. The comparative results indicate that the proposed method has significant advantages in expressive diversity and depth of cultural communication. When properly designed, Artificial Intelligence Generated Content (AIGC) technology can be effectively used for the generation and identity reconstruction of Chinese traditional cultural narrative content. This study provides a scalable technical path for the integration of AI and traditional culture, and expands the boundaries of digital humanities in content generation and reception research.Supplementary InformationThe online version contains supplementary material available at 10.1038/s41598-025-32302-5. Introduction Research background and motivations Currently, Artificial Intelligence Generated Content (AIGC) technology is evolving at an unprecedented rate. From text generation and image synthesis to multimodal interaction, it is gradually becoming a significant force in content production1. Its widespread application in fields such as media, art, and education herald the emergence of a new digital narrative paradigm2. Against this backdrop, AIGC provides an efficient tool for information delivery and paves an innovative path for cultural expression3. Meanwhile, Chinese traditional culture is facing the dual challenges of “discourse failure” and “audience disconnection”. On the one hand, traditional culture often relies on historical contexts and symbolic systems, and its narrative methods can be difficult to understand and may have aesthetic gaps in modern communication4. On the other hand, the younger generation has significant differences in media preferences and reception psychology compared to traditional forms of expression, leading to a gradual weakening of the effectiveness of cultural dissemination5. Therefore, there is an urgent need to use technology to digitally reconstruct the narrative patterns of traditional culture to enhance its vividness and approachability in the contemporary context of expression. In multi-modal research such as Visual Question Answering (VQA), existing studies have significantly improved the semantic consistency and expression stability of generated content by enhancing visual semantic robustness and the language-image collaboration mechanism. This has inspired the design of cultural semantic alignment and narrative control mechanisms in this study. In recent years, style transfer technology in deep learning has made significant progress in image generation and artistic style simulation fields. Particularly, the new generation of generative models represented by diffusion models and Contrastive Language-Image Pretraining (CLIP) semantic guidance has been able to achieve high-quality semantic control and visual generation across modalities6. This technological approach provides the possibility for the digital translation of traditional cultural elements, allowing originally static and abstract cultural symbols to be dynamically interpreted in multimodal forms7. Based on this, the integration of AIGC and style transfer technology is expected to break through the “form-semantics” disconnection in the digitalization of traditional culture, achieving effective coupling between cultural core and contemporary expression. Research objectives This study aims to explore the implementation mechanisms and practical effectiveness of AIGC models based on deep learning style transfer technology in the narrative transformation of Chinese traditional culture. The specific goals include: constructing a multimodal generative model that integrates image generation, text semantics, and style control, and analyzing its pathways and methods in cultural narrative construction; proposing a generation control mechanism centered on narrative style labels and cultural identity feedback to achieve controllable generation and expressive diversity of cultural content. Literature review The introduction of the large pre-trained model Generative Pre-trained Transformer (GPT)-3 achieves context-relevant language generation through learning from massive amounts of text data. It also provides a foundational semantic engine for the reconstruction of cultural texts8. In the context of cultural dissemination, Wadhwani et al. (2021) examined the narrative reproduction of AIGC in museum settings, highlighting its significant potential for cross-cultural interpretation and audience emotional activation9. Assael et al. (2022), in their research on culturally-oriented content generation for educational purposes, used AIGC to automatically write traditional festival stories, enhancing cognitive engagement and emotional identification among young people10. In the field of digital dissemination of Chinese traditional culture, researchers have mostly focused on aspects such as media translation, content redesign, and user acceptance11. Zhang et al. (2024) analyzed user preferences for traditional culture-themed short videos based on user interviews and found that visual imagery and emotional narration are key to attracting young audiences12. Sinhal et al. (2020) pointed out through questionnaires and web corpus analysis that current traditional cultural dissemination faced issues such as “heterogeneity of audience cultural backgrounds” and “high barriers to understanding classical texts”. It could easily lead to disconnection in communication and cultural fragmentation13. Style transfer, as a core branch of computer vision, has evolved through several stages. It began with the Adaptive Instance Normalization (AdaIN) model based on convolutional feature matching, progressed to the Style Generative Adversarial Network (StyleGAN) series with image style control capabilities, and then to diffusion models with cross-modal capabilities14. Yao et al. (2025) proposed text-guided style reconstruction in the CLIP space, effectively achieving triple consistency between text, image, and style. This opened a new direction for the integration of style transfer and semantic control15. In recent years, related research in the VQA field has put forward higher requirements for the semantic control ability of multi-modal generation. Chowdhury and Soni (2025) proposed a dynamic semantic enhancement mechanism based on the Reasoning-based Visual Question Answering (R-VQA) model to improve the robustness of image-text alignment16. Chowdhury and Soni (2024) optimized visual perception accuracy by weakening language prior bias17. Studies on Environmental Visual Question Answering (ENVQA) and Question-Specific Feature Visual Question Answering (QSFVQA) have shown that high-quality image semantic modeling and cross-modal feature fusion are key paths to ensure generation consistency18,19. Research on VQA further pointed out that logical combinatorial reasoning ability was the foundation for achieving stable narrative expression20. The above studies provide methodological inspiration for the construction of the traditional cultural semantic fusion model in this study. In addition, existing studies have also explored semantic mapping and emotional symbol transfer issues in more complex scenarios. Bai et al. (2024) established a deconstruction and reconstruction mechanism of cultural symbolic schemas in medical image semantic generation21. Zermatten et al. (2025) verified semantic transfer ability under open vocabulary conditions in remote sensing multi-semantic scenarios, providing references for the cross-semantic label generation strategy of this study22. Wei et al. (2023) pointed out that emotional consistency control in the AIGC generation process was of great significance for the construction of cultural resonance11. These studies have further enriched the cultural adaptability, emotional regulation ability, and pan-context transplantation paths of AIGC-generated content. Existing research has made significant progress in AIGC generation and style transfer technology, and has shown good application potential in cultural dissemination. However, it still falls short in the reconstruction of the deep narrative logic of traditional culture, the expressive labeling of narrative styles, and the quantitative evaluation mechanism of cultural identity. Research model Research framework This study develops a holistic research framework with cultural element analysis, style feature transfer, narrative context generation and cultural identity feedback (Fig. 1). Fig. 1Framework of holistic research. Framework of holistic research. The research framework consists of four core modules: (1) Traditional Cultural Corpus Input Module, (2) Style Transfer and Semantic Generation Module, (3) Narrative Structure Reconstruction Module, and (4) Cultural Identity Evaluation and Feedback Module. The framework takes the deep learning style transfer model (based on Stable Diffusion v2.1) as the technical core. It integrates three key elements (text, image, and style) as the basic pathway. This forms an automatic generation mechanism for the contemporary expression of traditional culture. Model core mechanism Based on Stable Diffusion v2.1, the proposed generative model integrates the CLIP model to achieve semantic consistency constraints. It also introduces the Low-Rank Adaptation (LoRA) mechanism to embed traditional cultural styles through low-rank regulatory embedding, thereby enhancing the integration capability of the generated content in terms of visual representation and cultural imagery. Stable Diffusion is a type of Latent Diffusion Model (LDM), which performs denoising generation in the latent space of images to achieve efficient modeling of high-resolution images23,24. Given an initial noise distribution zT, the model generates the final image \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\widehat{\mathbf{x}}$$\end{document} through an iterative process, as shown in Eq. (1):1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\widehat{\mathbf{x}}={D}_{\theta\:}\left({z}_{0}\right),\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}{z}_{0}={\mathrm{L}\mathrm{D}\mathrm{M}}_{\theta\:}\left({z}_{T}\right)$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{D}_{\theta\:}$$\end{document} represents the decoder and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{L}\mathrm{D}\mathrm{M}}_{\theta\:}$$\end{document} represents the latent space representation generated by a series of back diffusion steps. In the text control generation, the condition vector c (generated by the text encoder) is introduced, and the generation process can be expressed as Eq. (2):2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\widehat{\mathbf{x}}={D}_{\theta\:}\left({\mathrm{L}\mathrm{D}\mathrm{M}}_{\theta\:}\right({z}_{T}\mid\:c\left)\right)$$\end{document} In this study, c is generated by Text Encoder of CLIP model, which ensures the continuous control and semantic guidance of traditional cultural text semantics in the process of diffusion generation. To ensure the semantic correspondence between the generated images and cultural narrative texts, this study introduces the joint embedding space in the CLIP model as a cross-modal constraint mechanism. CLIP achieves cross-modal alignment by maximizing the similarity of text-image pairs, with the training objective function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathcal{L}}_{CLIP}$$\end{document} shown in Eq. (3):3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathcal{L}}_{CLIP}=-\frac{1}{N}\sum\:_{i=1}^{N}\:\mathrm{l}\mathrm{o}\mathrm{g}\frac{\mathrm{e}\mathrm{x}\mathrm{p}(⟨{f}_{t}({x}_{i}^{t}),{f}_{i}({x}_{i}^{i})⟩/\tau\:)}{\sum\:_{j=1}^{N}\:\mathrm{e}\mathrm{x}\mathrm{p}(⟨{f}_{t}({x}_{i}^{t}),{f}_{i}({x}_{j}^{i})⟩/\tau\:)}$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f}_{t}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f}_{i}$$\end{document} represent text and image encoders respectively, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\tau\:$$\end{document} is the temperature coefficient. Considering that the Stable Diffusion model is huge, the training cost is high, and the traditional cultural styles are sparsely distributed in the data set, this study uses LoRA technology to locally tune the pre-training model25,26. LoRA introduces low rank decomposition into the weight matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathbf{W}\in\:{\mathbb{R}}^{d\times\:k}$$\end{document} of the original model, as shown in Eq. (4):4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathbf{W}}^{{\prime\:}}=\mathbf{W}+{\Delta\:}\mathbf{W},{\Delta\:}\mathbf{W}=\mathbf{A}\mathbf{B},\mathbf{A}\in\:{\mathbb{R}}^{d\times\:r},\mathbf{B}\in\:{\mathbb{R}}^{r\times\:k},r\ll\:min(d,k)$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathbf{A}$$\end{document} is one of the trainable weight matrices, which is used to map the input to a low-dimensional space. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathbf{B}$$\end{document} is one of the trainable weight matrices, which is used to map the low-dimensional spatial results back to the original dimension. r represents the rank value in low rank decomposition and the embedding space dimension, which is much smaller than d and k. d represents the dimension of the original input. k is the dimension of the original output. In this study, r is set to 8, and the style transfer ability is injected into the key attention module of Unet structure by layer-by-layer injection. Intuitively, the core idea of the LoRA mechanism is as follows: Without modifying most parameters of the original large model, only “lightweight” style adjustment structures are inserted into specific sub-modules. These structures contain only a small number of trainable parameters (low-rank matrices A and B), which learn the feature transfer capability of cultural styles during the training phase. Thus, fine-grained style control is achieved while maintaining the overall capabilities of the pre-trained model. Compared with full-parameter fine-tuning, LoRA greatly reduces training overhead and avoids the “catastrophic forgetting effect”. The final generation framework combines the above three mechanisms to construct a conditional generation function in the form of Eq. (5):5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\widehat{\mathbf{x}}={D}_{\theta\:}\left({\mathrm{L}\mathrm{D}\mathrm{M}}_{\theta\:}\right({z}_{T}\mid\:{f}_{t}\left(c\right),{\mathrm{L}\mathrm{o}\mathrm{R}\mathrm{A}}_{\varphi\:}\left)\right),\varphi\:\subset\:\theta\:$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f}_{t}\left(c\right)$$\end{document} is the cultural text coding vector. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{L}\mathrm{o}\mathrm{R}\mathrm{A}}_{\varphi\:}$$\end{document} is the embedded traditional cultural style adjustment module, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\theta\:$$\end{document} represents the whole model parameter set. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{D}_{\theta\:}$$\end{document} is the decoder part of the diffusion model, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{L}\mathrm{D}\mathrm{M}}_{\theta\:}$$\end{document} represents the underlying latent space diffusion model. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{z}_{T}$$\end{document} represents the initial noise tensor obtained by sampling Gaussian distribution. The design of narrative transformation mechanism To achieve style control and generative expression of traditional cultural content, this study constructs a system of five representative narrative style labels based on preliminary corpus analysis and image semantic clustering. This system integrates visual features, cultural semantics, and application scenarios, aiming to provide the model with a clear and controllable style guidance mechanism. Each type of style label refines several “core images” and associates them with specific “cultural orientations” and “application dimensions”, thereby ensuring the consistency and expressiveness of the model in the process of image generation and text guidance. In Table 1, these five types of labels include “Symbolism of Confucianism, Buddhism, and Taoism”, “Landscape Painting Style”, “Artistic Conception of Calligraphy”, “Legends and Myths”, and “Folk Life Scenes”. They cover a wide cultural space from philosophical symbols and natural landscapes to mythological narratives and folk experiences, providing clear semantic coordinates and expression boundaries for multi-modal content generation. Table 1Five kinds of narrative style labels.Style labelCore imageCultural orientationApplication dimensionSymbolism of Confucianism, Buddhism, and TaoismConfucius, Laozi, Buddha, Taiji, etc.Philosophical Thought, Ethical Order, Self-cultivationText/Image/Semantic GuidanceLandscape painting styleMountain Peaks and Ridges, Ink Wash Painting, Leaving Blank, Multi-point PerspectiveUnity of Heaven and Humanity, Self and Object Both ForgottenImage style transferArtistic conception of calligraphyRunning-Cursive Script, Seal-Clerical Script, Brush-Tip StructureEmotional Expression and Personality ExpressionLine Generation/Content LayoutLegend mythNüwa Mends the Heavens, Sun Wukong, Kunpeng, etc.Imagination, Primitive Narrative StructureCharacter Generation/Scene FusionFolk life sensePaper-Cutting, New Year Pictures, Temple Fairs and Customs FestivalsRegionality, Sense of Ceremony, Daily ExperienceScene rendering/scene simulationIn the concrete implementation, each style label is expressed by embedding vectors in the model input stage, which is recorded as Eq. (6): 6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathbf{l}}_{i}=\mathrm{E}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{d}\left({s}_{i}\right),i=1,\dots\:,5$$\end{document} Table 1Five kinds of narrative style labels.Style labelCore imageCultural orientationApplication dimensionSymbolism of Confucianism, Buddhism, and TaoismConfucius, Laozi, Buddha, Taiji, etc.Philosophical Thought, Ethical Order, Self-cultivationText/Image/Semantic GuidanceLandscape painting styleMountain Peaks and Ridges, Ink Wash Painting, Leaving Blank, Multi-point PerspectiveUnity of Heaven and Humanity, Self and Object Both ForgottenImage style transferArtistic conception of calligraphyRunning-Cursive Script, Seal-Clerical Script, Brush-Tip StructureEmotional Expression and Personality ExpressionLine Generation/Content LayoutLegend mythNüwa Mends the Heavens, Sun Wukong, Kunpeng, etc.Imagination, Primitive Narrative StructureCharacter Generation/Scene FusionFolk life sensePaper-Cutting, New Year Pictures, Temple Fairs and Customs FestivalsRegionality, Sense of Ceremony, Daily ExperienceScene rendering/scene simulationIn the concrete implementation, each style label is expressed by embedding vectors in the model input stage, which is recorded as Eq. (6): Five kinds of narrative style labels. In the concrete implementation, each style label is expressed by embedding vectors in the model input stage, which is recorded as Eq. (6): \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{s}_{i}$$\end{document} represents the text prompt or image style identification corresponding to the i-style tag. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathbf{l}}_{i}$$\end{document} is the style control vector, which is jointly input with the text encoder or the image condition module to adjust the narrative direction and visual style of the generated content. This study proposes a technology-text-image three-mode fusion mechanism, which includes the following three key paths: (1) Semantic-driven Path. Generate corresponding image content \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\widehat{\mathbf{x}}$$\end{document} based on the text prompt c. Control the generated image to be consistent in style \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{l}}_{i}$$\end{document} and semantic \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f}_{t}\left(c\right)$$\end{document}, and the expression form is as shown in Eq. (7)27,28:7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\widehat{\mathbf{x}}={D}_{\theta\:}\left({\mathrm{L}\mathrm{D}\mathrm{M}}_{\theta\:}\right({z}_{T}\mid\:{f}_{t}\left(c\right),{\mathrm{l}}_{i}\left)\right)$$\end{document} To enhance the model’s ability to understand complex semantics (such as metaphors and irony), this study introduces a text data subset containing “ironic sentences” during the training phase. The subset is extracted from typical implicit semantic paragraphs in Zhuangzi and A New Account of the Tales of the World. A BERT-based semantic discriminator is used for pre-classification of text prompts (literal vs. ironic). When the system detects potential ironic structures, it automatically switches to a context-nested image generation path to avoid generating “misinterpreted images.” The contrastive learning strategy in the CLIP embedding space is reweighted to reduce the priority of literal semantics in ironic contexts, thereby being more in line with the author’s intent. This mechanism effectively improves the accuracy of the model in generating ironic and metaphorical images and avoids literal misinterpretation of surface semantics. (2) Image Feedback Path (Vision-to-Text). The generate image is input to that CLIP image encoder again to obtain an image semantic table \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f}_{i}\left(\widehat{\mathbf{x}}\right)$$\end{document}. Calculate the semantic similarity \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{txt-img}$$\end{document} between the generated image and the original text as the semantic consistency evaluation index, as shown in Eq. (8):8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{txt-img}=\mathrm{c}\mathrm{o}\mathrm{s}\left({f}_{t}\right(c),{f}_{i}(\widehat{\mathbf{x}}\left)\right)$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{c}\mathrm{o}\mathrm{s}$$\end{document} represents cosine similarity, which is used to measure the semantic space alignment between modes. (3) Style Regularization. The style of the generated image is judged, and its consistency with the target style label \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathbf{l}}_{i}$$\end{document} is judged through the pre-trained style recognition network \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{G}_{s}$$\end{document}29,30. Use the loss function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathcal{L}}_{style}$$\end{document} for optimization, as shown in Eq. (9):9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathcal{L}}_{style}=\parallel\:{G}_{s}\left(\widehat{\mathbf{x}}\right)-{\mathbf{l}}_{i}{\parallel\:}_{2}^{2}$$\end{document} The style recognition network Gs is constructed based on a lightweight Convolutional Neural Network (CNN) architecture, with an overall structure similar to ResNet-18. It consists of four residual convolution modules and outputs the probability distribution of five types of style labels (corresponding to the five styles in Table 1). The training data is derived from the self-built “Cultural Style Image Dataset” in this study, totaling 6200 images covering five style types (about 1200 images per type), and the image samples are manually annotated by cultural experts. To verify the classification ability of the style recognition network Gs, the CrossEntropy loss function is used for training, achieving a Top-1 accuracy of 94.8% on the validation set. The confusion matrix shows that most styles have good separability, while certain cross-misclassification exists between individual styles (e.g., Calligraphy Artistic Conception and Landscape Painting Style), mainly due to the similarity in texture structure. Evaluation path of cultural identity This study constructs a cultural identity evaluation path based on a deep semantic matching mechanism, which mainly includes three indicator dimensions: imagery accuracy, narrative resonance, and cognitive engagement. It combines multimodal semantic space and user evaluation mechanisms for quantitative and qualitative analysis. (1) Image accuracy. Imagery accuracy aims to measure whether the AIGC-generated content visually and accurately presents the core imagery elements of the target traditional culture31. First, a traditional cultural imagery lexicon is constructed, including imagery units such as figures, artifacts, patterns, and compositions. The CLIP image encoder is used to extract the semantic vector of the generated image32. Meanwhile, the CLIP text encoder is used to generate the vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f}_{t}\left({v}_{j}\right)$$\end{document} for the target imagery keywords, and the matching degree is calculated, as shown in Eq. (10):10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{\mathrm{i}\mathrm{m}\mathrm{g}-\mathrm{a}\mathrm{c}\mathrm{c}}=\frac{1}{n}\sum\:_{j=1}^{n}\:\mathrm{c}\mathrm{o}\mathrm{s}\left({f}_{i}\left(\widehat{\mathbf{x}}\right),{f}_{t}\left({v}_{j}\right)\right)$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{\mathrm{i}\mathrm{m}\mathrm{g}-\mathrm{a}\mathrm{c}\mathrm{c}}$$\end{document} is the score of image accuracy. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{c}\mathrm{o}\mathrm{s}$$\end{document} is cosine similarity function. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f}_{i}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f}_{t}$$\end{document} are image and text coding functions in CLIP. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:n$$\end{document} is the number of image tags. (2) Narrative resonance. Narrative resonance aims to measure whether the audience can generate emotional resonance and cultural identity responses during the reception of the generated content33,34. The user feedback text \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{u}_{k}$$\end{document} and the original cultural narrative text \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:c$$\end{document} are jointly projected into the CLIP semantic space to construct the semantic resonance score \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{\mathrm{r}\mathrm{e}\mathrm{s}}$$\end{document}, as shown in Eq. (11):11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{\mathrm{r}\mathrm{e}\mathrm{s}}=\mathrm{c}\mathrm{o}\mathrm{s}\left({f}_{t}\left({u}_{k}\right),{f}_{t}\left(c\right)\right)$$\end{document} Supplemented by Likert’s five-level subjective evaluation questionnaire, the target users (N = 100) are invited to rate the “cultural resonance” to obtain the distribution of artificial scores35. Equation (12) shows the comprehensive calculation. To enhance the representativeness and credibility of subjective scoring results, this study recruits 100 participants in total, including a youth group (N = 60) and an expert group (N = 40). The youth group, aged between 18 and 25, come from different universities and urban backgrounds, covering related majors such as digital art, journalism and communication, and cultural industry. The expert group consists of university teachers and researchers with a traditional cultural background. The selection criteria include: priority given to those who have published relevant papers in CSSCI or core journals, and those who have participated in national/provincial-level digital cultural projects. A double-blind evaluation mechanism is adopted during the scoring process. All evaluation objects are presented anonymously, and experts used a unified scale for scoring, including dimensions such as image style accuracy, semantic consistency, and cultural resonance. The average value of each dimension is finally taken as the input of the manual resonance score \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\stackrel{\prime }{R}}_{manual}$$\end{document}. Although the total sample size is 100, including 60 in the youth group and 40 in the expert group, this study focuses on qualitative feedback and cross-group comparison, emphasizing the representativeness of the sample structure rather than large-scale quantitative statistics. This sample size is at a medium level in AIGC-related subjective evaluation studies and can effectively reveal major trends. To make up for the limitation of sample size, it is planned to expand the sample to 300 people in the follow-up and introduce more regions and age groups to enhance the universality and promotion of the conclusions.12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{Narrative\:Resonance\:Score}={\lambda\:}_{1}{S}_{\mathrm{r}\mathrm{e}\mathrm{s}}+{\lambda\:}_{2}{\stackrel{-}{R}}_{\mathrm{m}\mathrm{a}\mathrm{n}\mathrm{u}\mathrm{a}\mathrm{l}}$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\lambda\:}_{1}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\lambda\:}_{2}$$\end{document} are weight parameters, which satisfy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\lambda\:}_{1}$$\end{document}+\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\lambda\:}_{2}$$\end{document}=1. In this study, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\lambda\:}_{1}$$\end{document}=0.6 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\lambda\:}_{2}$$\end{document}=0.4. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\stackrel{-}{R}}_{\mathrm{m}\mathrm{a}\mathrm{n}\mathrm{u}\mathrm{a}\mathrm{l}}$$\end{document} is the average value of manual evaluation. (3) Cognitive participation. Cognitive engagement aims to measure the depth of understanding and proactive response ability that users demonstrate when facing the generated content36,37. An interactive question-answering task is designed (posing 3–5 related questions based on the image/text-generated content)38,39. Users complete the task and provide free-text answers \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{k}$$\end{document}. A pre-trained language understanding model (BERT-based QA (Question Answering) evaluator) is used to analyze the logical completeness and contextual match of the answers40,41, and a score \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{\mathrm{e}\mathrm{n}\mathrm{g}}$$\end{document} is calculated, as shown in Eq. (13):13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{S}_{\mathrm{e}\mathrm{n}\mathrm{g}}=\frac{1}{m}\sum\:_{k=1}^{m}\:\mathrm{B}\mathrm{E}\mathrm{R}\mathrm{T}\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}({a}_{k},{\mathrm{A}\mathrm{n}\mathrm{s}\mathrm{w}\mathrm{e}\mathrm{r}}_{\mathrm{r}\mathrm{e}\mathrm{f},k})$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\mathrm{A}\mathrm{n}\mathrm{s}\mathrm{w}\mathrm{e}\mathrm{r}}_{\mathrm{r}\mathrm{e}\mathrm{f},k}$$\end{document} is the standard reference answer (provided by cultural experts). m is the total number of answers. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{B}\mathrm{E}\mathrm{R}\mathrm{T}\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}$$\end{document} is a semantic similarity evaluation function based on context embedding. Model verification strategy To systematically verify the effectiveness of the proposed AIGC generation model in the narrative expression of traditional Chinese culture, this study constructs a three-level and multi-dimensional verification mechanism covering semantic consistency, style alignment, and user perception responses, forming a complementary strategy of “automatic evaluation + subjective assessment”, as detailed below: (1) Semantic Consistency Verification Path: Construct cross-modal image-text similarity metrics (e.g., CLIP-Sim, Simg−acc) using the CLIP semantic space to evaluate whether the generated images accurately convey the cultural images described in the text. Among them, CLIP-Sim is used for overall text-image matching, and Simg−acc focuses on the reducibility of image keyword terms. (2) Style Consistency Verification Path: Perform style classification on the generated images through the style recognition network Gs, compare them with the input style labels to obtain Lstyle loss and accuracy metrics; combine the confusion matrix to evaluate the discriminative ability between various styles, and verify the stability and generalization of the style transfer module. (3) User Subjective Cognition Evaluation Path: Set up user question-answering tasks and subjective scoring of narrative resonance, calculate Seng and Narrative Resonance Score to reflect the effectiveness of the generated content in cognitive processing and cultural empathy, considering both deep semantic understanding and emotional responses. To further verify the adaptability and stability of the model in multi-style label scenarios, this study sets two cultural themes, namely “Confucianism-Buddhism-Taoism Images” and “Legends and Myths”, for full-process evaluation. It conducts comparative analysis on the generalization performance of various indicators under different cultural structures, reflecting the cultural transferability of the model structure. Code availability All the main model structures, style control module (LoRA injection), style recognition network (Gs), and semantic guidance path (CLIP-BERT path) involved in this study are written in Python by the authors and implemented based on the PyTorch framework. All core training and inference codes have been organized in a reproducible form and stored in a private GitHub repository. Access links can be provided upon request from editors or reviewers. Meanwhile, some core modules are additionally attached in the Supplementary Code document for review and verification. Experimental design and performance evaluation Datasets collection and experimental environment The dataset used in this study includes both images and texts, all constructed around the core imagery of Chinese traditional culture. The image data is mainly collected from publicly available resources such as ink wash paintings, intangible cultural heritage crafts, calligraphy, paper-cutting, and folk New Year paintings, totaling 12,000 images. All images are high-quality Red Green Blue (RGB) images with a unified resolution of 512 × 512, covering typical visual styles and completed with style label annotations. The text data is selected from classic literature such as " Classic of Mountains and Seas” (Shan Hai Jing), " The Analects of Confucius” (Lun Yu), “Dream of the Red Chamber” (Hong Lou Meng), and " Investiture of the Gods” (Feng Shen Yan Yi). It focuses on paragraphs that are symbolic, narrative, and philosophical, constructing a traditional narrative corpus of 300,000 words, used for CLIP text embedding and semantic guidance. The experimental environment configuration of this study is shown in Table 2. Table 2Experimental environment.CategoryConfiguration detailsGraphics Processing Unit (GPU)Nvidia A100 × 2, single card memory 80GB (Gigabyte)Central Processing Unit (CPU)AMD (Advanced Micro Devices) EPYC (Embedded Processor for x86-64 Architecture) 7742 × 1, 64 cores and 128 threadsMemory128 GB ECC (Error-Correcting Code) DDR4 (Double Data Rate Fourth Generation)Operating systemUbuntu 22.04 LTS (Long Term Support)Deep learning frameworkPyTorch 2.0.1Natural language processingHugging Face Transformers 4.36.2Diffusion model toolDiffusers Library (version 0.21.0)Model infrastructureStable Diffusion v2.1 + CLIP ViT (Vision Transformer-Base)-H/14Acceleration and distributionDeepSpeed (enable zero (zero redundancy optimizer) stage 2), Mixed Precision training Table 2Experimental environment.CategoryConfiguration detailsGraphics Processing Unit (GPU)Nvidia A100 × 2, single card memory 80GB (Gigabyte)Central Processing Unit (CPU)AMD (Advanced Micro Devices) EPYC (Embedded Processor for x86-64 Architecture) 7742 × 1, 64 cores and 128 threadsMemory128 GB ECC (Error-Correcting Code) DDR4 (Double Data Rate Fourth Generation)Operating systemUbuntu 22.04 LTS (Long Term Support)Deep learning frameworkPyTorch 2.0.1Natural language processingHugging Face Transformers 4.36.2Diffusion model toolDiffusers Library (version 0.21.0)Model infrastructureStable Diffusion v2.1 + CLIP ViT (Vision Transformer-Base)-H/14Acceleration and distributionDeepSpeed (enable zero (zero redundancy optimizer) stage 2), Mixed Precision training Experimental environment. Parameters setting Based on the methodological design, this study has systematically set parameters for key modules. The following parameters correspond to the diffusion generation process, the style regulation module (LoRA), and the semantic control module (guided by CLIP), and their optimal value ranges have been verified through preliminary experiments. The main parameter settings for the proposed model are shown in Table 3. Table 3Main parameter settings.Parameter nameSetting valueBelonging moduleDiffusion step number50Stable DiffusionCFG Scale7.5Classifier-Free GuidanceLoRA injection weight0.75Style control module (LoRA)LoRA rank value8Style control module (LoRA)Text guidance weight0.6CLIP semantic guidanceImage discriminator threshold0.85Style consistency constraint Table 3Main parameter settings.Parameter nameSetting valueBelonging moduleDiffusion step number50Stable DiffusionCFG Scale7.5Classifier-Free GuidanceLoRA injection weight0.75Style control module (LoRA)LoRA rank value8Style control module (LoRA)Text guidance weight0.6CLIP semantic guidanceImage discriminator threshold0.85Style consistency constraint Main parameter settings. Performance evaluation Evaluation of image quality and style expression ability Starting from the objective quality of image generation and the accuracy of traditional cultural style expression, a systematic comparison is made between the proposed AIGC model and existing mainstream image generation models. The evaluation indicators cover: Fréchet Inception Distance (FID): Measures the overall distribution difference between generated images and real images; lower values are better. Learned Perceptual Image Patch Similarity (LPIPS): Assesses the perceptual similarity between images; lower values indicate closer resemblance to real samples. Style Recognition Accuracy: Uses a pre-trained style classifier (with five style labels) to judge the consistency between the generated image style and the target label, reflecting the style transfer effect. The comparison models include: Stable Diffusion v2.1 (SD 2.1), ControlNet + Style Reference Image, and StyleGAN-NADA (No Additional Domain Adaptation) (semantic style transfer). The results of image generation performance evaluation are shown in Fig. 2. Fig. 2The comprehensive performance comparison between the model and three contrast models in image quality, perceptual similarity and style expression accuracy. The comprehensive performance comparison between the model and three contrast models in image quality, perceptual similarity and style expression accuracy. In Fig. 2, the proposed model has significant advantages in all three indicators: the lowest FID score (22.85), indicating that the generated images are closest to real traditional images in terms of statistical distribution and have the most natural and stable quality. The lowest LPIPS value (0.298), which means the image detail quality is optimal at the perceptual level. A style recognition accuracy of 86.4%, which is significantly higher than other comparison models. This indicates that the traditional cultural style features embedded by LoRA have good transferability and that the semantic and style consistency is further enhanced by CLIP guidance. Consistency of text narration and the evaluation of semantic control Two types of tasks are set: “Text-Image Consistency Generation Task” and “Narrative Style Prompt Rewriting Task,” and four indicators are used for evaluation: Bilingual Evaluation Understudy (BLEU): Measures the degree of n-gram overlap between the generated text and the reference text. Recall-Oriented Understudy for Gisting Evaluation (ROUGE): Assesses the coverage of the original narrative by the generated text. Text-Image Semantic Similarity (CLIP-Sim): Measures the consistency between the semantics of the generated image and the text. Narrative Style Match: Evaluates the consistency of narrative style in terms of structure, emotion, and imagery in the generated text. The comparison models include: Stable Diffusion v2.1, ControlNet + Style Reference Image, and StyleGAN-NADA. The results are shown in Fig. 3. Fig. 3Evaluation of text generation ability of four models under the indicators of narrative consistency, semantic alignment and style matching. Evaluation of text generation ability of four models under the indicators of narrative consistency, semantic alignment and style matching. In Fig. 3, the proposed multimodal AIGC model significantly outperforms the three comparison models in terms of text-image semantic alignment and narrative style matching capabilities: The BLEU and ROUGE scores have increased significantly, indicating that the model has a stronger ability to retain the content of the input text and reproduce the language style. The rewritten content is closer to the original cultural context. The Narrative Style Match reaches 82.3%, proving that the model is more adjustable in narrative style control. The CLIP-Sim score is as high as 0.793, indicating that the generated images more accurately reflect the cultural imagery contained in the text, enhancing the advantage of coordinated semantic construction between text and images. Cultural image restoration and user perception evaluation To enhance the representativeness and scientific nature of subjective scoring, this study recruits 100 participants in total, including 60 in the youth group and 40 in the expert group. The youth group, aged between 18 and 25, mainly comes from professional backgrounds such as digital art, journalism and communication, and cultural industry, covering different universities and regions. The expert group consists of cultural university teachers, researchers from art institutions, etc. The selection criteria include: priority is given to those who have published papers related to traditional culture in core journals in the past five years or participated in provincial-level or higher digital cultural projects. A double-blind design is adopted in the scoring phase. Participants conduct evaluations based on a unified scoring scale without knowing the configuration of the generative model. The scale includes three dimensions: ① Semantic Consistency (whether the content accurately expresses the text semantics); ② Image Style Expressiveness (whether it restores cultural style characteristics); ③ Cultural Resonance Degree (whether it triggers the audience’s cultural emotional identity). Each item adopted a 1–5 point Likert scale scoring, and the average value is finally taken as the input for the subjective scoring part of the Narrative Resonance Score. Although the total sample size is 100, it is above the medium level in AIGC-related cultural generation research, and the structural distribution is representative, which helps capture the perception differences between young users and professional scholars. To improve the consistency of expert scoring, the expert group receives unified training before the official scoring, clarifying the scoring dimensions, scoring standards, and typical examples to ensure a consistent evaluation scale. In addition, each generated sample is independently scored by at least two experts, and the final score was the average of their scores. If the score difference between the two experts exceeds 1 point, a third expert conducts a review and judgment to ensure the stability and reliability of the evaluation results. This study sets three evaluation indicators from both the model and user levels: Imagery Semantic Similarity of Images (Simg-acc): Reflects whether the generated images accurately reproduce the cultural imagery keywords. Narrative Resonance Score: Combines CLIP semantic similarity and user subjective ratings. Human Evaluation: Invites two representative user groups (youth and cultural scholars) to evaluate from three dimensions: “Cultural Expression,” “Aesthetic Identification,” and “Style Understanding,” using a 5-point scale. To enhance the explanatory power of the analysis, this study selects two cultural style labels to model and output, and conducts a detailed comparison: Category A: Confucianism, Buddhism, and Taoism Imagery-Emphasizes philosophical nature and abstract symbolism (Tai Chi, lotus, the Buddha’s smile when holding a flower). Category B: Legends and Myths-Focuses on character images and dynamic narratives (Nüwa Mends the Heavens, Nezha Stirs Up the Sea). The specific results are shown in Fig. 4. Fig. 4Cultural image restoration degree, user subjective resonance score and group difference analysis of four models under different cultural style labels. Cultural image restoration degree, user subjective resonance score and group difference analysis of four models under different cultural style labels. In Fig. 4, the proposed model significantly outperforms the other three comparison models in all dimensions, demonstrating the effectiveness of the multimodal style fusion and semantic control strategy in traditional cultural generation tasks. In terms of imagery accuracy, the proposed method achieves the highest semantic reproduction scores in both “Symbolism of Confucianism, Buddhism and Taoism " (0.722) and “Legends and Myths” (0.748) tasks, significantly higher than the baseline models. This indicates that the culturally embedded style features, fine-tuned by LoRA, can effectively drive the model to accurately generate cultural visual elements. Regarding narrative resonance, the model also shows strong control over text-guided narrative expression. The average resonance scores for the two tasks are 4.28 and 4.35, respectively, both significantly higher than those of the comparison models, indicating that the generated content is more likely to evoke cultural and emotional resonance among users. In the subjective evaluation, both the scholar group (4.4) and the youth group (4.3) give high overall ratings to the proposed model, while the SD native model only receives 2.4 and 2.8, respectively. This confirms that cultural embedding and narrative consistency have a significant impact on audience acceptance. Generative diversity and cognitive participation assessment This study designs the following evaluation system: Self-BLEU: Used to assess the semantic diversity of generated content under the same input prompt. Lower values indicate greater differences, representing richer content. Seng (Cognitive Engagement Score): Based on user free-form question-and-answer tasks, it evaluates the semantic match of users’ reactions to the generated content. To ensure the comparability of the experiments, this study generates outputs based on the following two settings: Standard Template Content Group: Normative cultural description paragraphs written by experts, used to simulate “traditional static content. AIGC Generated Diversity Group: The model generates three stylistic variants under the same text prompt (corresponding to different narrative labels: lyrical, fable, and epic). Question-Answer Task Pass Rate: The accuracy with which users’ complete questions derived from the generated content, serving as an indirect reflection of depth of understanding. Three comprehension questions are designed for each content segment, and data is collected from N = 60 users (youth group) for answering and free expression, followed by scoring. The specific evaluation results are shown in Fig. 5. Fig. 5The performance of the content generated by the model in three indicators: semantic diversity, self-understanding response ability and user’s correct answer rate. The performance of the content generated by the model in three indicators: semantic diversity, self-understanding response ability and user’s correct answer rate. In Fig. 5, the proposed AIGC model demonstrates significant advantages in supporting diverse cultural expression and user cognitive engagement: The Self-BLEU score (0.278) is significantly lower than that of the template group (0.412), indicating that while maintaining semantic consistency, the generated content is more diverse in language construction, perspective switching, and style expression, avoiding “formulaic output”. The increase in user free-form question scores (0.743) indicates that the generated content better stimulates understanding and thinking. The CLIP semantic consistency and style label synergy mechanism makes the text logic more coherent and the narrative clearer. The question-answer pass rate of 82.6% verifies the understandability of the generated content. It indicates that it has certain knowledge content and expressive logic, and can serve as a trigger for user cognitive processing. Analysis of style discrimination and narrative differentiation To evaluate the performance of the style recognition network Gs in the narrative style classification task, this study conducts a systematic analysis from three dimensions: accuracy, confusion matrix, and style distinguishability. The Gs network adopts the ResNet-18 architecture and is trained on the five-category cultural style image dataset constructed in this study, which contains a total of 6200 manually annotated samples covering five style types: “Symbolism of Confucianism, Buddhism, and Taoism”, “Landscape Painting Style”, “Artistic Conception of Calligraphy”, “Legends and Myths”, and “Folk Life Scenes”. Table 4 shows the classification accuracy of each style type. Table 4Comparative analysis of Top-1/Top-3 accuracy and main misjudged categories of styles in style classification task.Style labelTop-1 accuracyTop-3 accuracyObject liable to misjudgmentLandscape painting style89.20%97.80%Artistic conception of calligraphyArtistic conception of calligraphy88.60%96.90%Landscape painting styleSymbolism of Confucianism, Buddhism, and Taoism94.10%99.00%No obvious misjudgmentFolk life scenes93.70%98.50%Legends and myths (small amount)Legends and myths94.80%98.40%Folk life scenes (mild) Table 4Comparative analysis of Top-1/Top-3 accuracy and main misjudged categories of styles in style classification task.Style labelTop-1 accuracyTop-3 accuracyObject liable to misjudgmentLandscape painting style89.20%97.80%Artistic conception of calligraphyArtistic conception of calligraphy88.60%96.90%Landscape painting styleSymbolism of Confucianism, Buddhism, and Taoism94.10%99.00%No obvious misjudgmentFolk life scenes93.70%98.50%Legends and myths (small amount)Legends and myths94.80%98.40%Folk life scenes (mild) Comparative analysis of Top-1/Top-3 accuracy and main misjudged categories of styles in style classification task. In Table 4, the “Legends and Myths” category shows the most stable performance in complex scenarios and character modeling. Meanwhile, analysis of the confusion matrix reveals a certain degree of misclassification between “Calligraphy Artistic Conception” and “Landscape Painting Style”. The reason is that both contain visual features such as ink texture and spatial blank leaving, resulting in blurred boundaries at the convolutional perception level. In contrast, “Folk Life Scenes” and “Symbolism of Confucianism, Buddhism, and Taoism” achieve higher classification accuracy due to their distinct image features and prominent visual symbols. In summary, the style recognition network exhibits excellent overall classification performance and can effectively support the style normalization mechanism. Especially in the process of cultural narrative generation, its responsiveness to fine-grained style semantics provides a fundamental guarantee for the cultural consistency of generated content, and verifies the distinguishability and practicality of style label settings. Discussion The root of the generated image quality and style recognition advantage lies in the low-rank fine-tuning strategy of LoRA for Stable Diffusion. This approach retains the original model’s efficient expressive power and achieves precise style embedding through a small number of traditional cultural images. This “light regulation, deep transfer” method has stronger stability and cultural detail reproduction capabilities compared to GAN-type models, avoiding the common issues of symbol drift and style mixing in image generation. The improvement in narrative consistency and text-image semantic alignment is thanks to the CLIP model’s deep semantic deconstruction ability for cultural texts. CLIP provides a consensus reference system in the semantic space and forms a joint guidance path with style labels. This allows the generated images to perform flexible style interpretation while retaining the semantic core, achieving “double alignment” of symbolic imagery and emotional intent. Moreover, the fundamental reason for the excellent performance in cultural identity and user perception is that the model generates “visually appealing content” and “content that can be understood and resonated with”. By binding style labels with narrative structures, the system’s output has a clear cultural positioning and emotional suggestion, enhancing the activation of cultural schemas and emotional resonance in the user’s cognitive process. The advantages in generation diversity and cognitive engagement reflect the model’s ability to balance “expressive control” and “user stimulation.” That is, the model can control the style of content expression and provide an open space for understanding. This balance is especially critical for cultural AIGC systems. Further, the model’s advantages are explained from the perspectives of parameter settings and comparison paths: In image quality and style expression tasks, LoRA’s injection weight (0.75) and rank value (8) control the fusion depth between style features and the backbone model, ensuring high-fidelity style transfer while maintaining the original diffusion capability. In text-image consistency tasks, the CLIP text guidance weight is set to 0.6. This relatively high weight effectively enhances cross-modal collaborative expression in the semantic space, with a CLIP-Sim score of 0.793. Compared with comparative models: Stable Diffusion 2.1 lacks a cultural fine-tuning mechanism; StyleGAN-NADA has style transfer capability but weak text constraints; ControlNet relies on external reference images, resulting in limited generation quality. The proposed model achieves comprehensive performance improvement through multi-path evaluation (FID, LPIPS, BLEU, Style Match, etc.), leading in all four main indicators and demonstrating the integrated advantages of cross-modal style control and cultural image expression. Conclusion Research contribution This study constructs an AIGC model framework for the narrative generation of Chinese traditional culture based on deep learning style transfer and multimodal semantic control technology. By introducing Stable Diffusion, CLIP, and the LoRA mechanism, the model achieves precise integration and controllable generation between cultural text, images, and style. The model performs excellently in terms of image quality, semantic consistency, cultural reproduction, and user perception, providing a new paradigm for the deep integration of digital humanities and intelligent communication. Future works and research limitations This study still has certain limitations: the current model still has generation boundaries for highly abstract imagery styles, and the sample size for user subjective evaluation is relatively limited. Future work will combine more extensive population data and human-computer interaction mechanisms to optimize the path for constructing cultural identity. Future research can be expanded in the following two directions: First, further introduce multilingual cross-cultural diffusion mechanisms to enhance the model’s cultural adaptability in international contexts. Second, strengthen emotional modeling and virtual narrative character functions to achieve a more immersive cultural experience. Supplementary Information Below is the link to the electronic supplementary material. Supplementary Material 1 Supplementary Material 1 Supplementary Material 2 Supplementary Material 2