A hybrid intelligent model that performs product evaluation via semantic mining and optimized decision processing Authors: Dezheng Wu, Xueqian Jiang, Yawen Hu, Kexiang Li Published: 2026 ABSTRACT The current product evaluation systems, constrained by fixed indicators and subjective weighting, struggle to adapt to dynamic market demands. This study proposes a data-driven framework integrating latent Dirichlet allocation, fuzzy analytic hierarchy process, and particle swarm optimization. User reviews and product parameters are collected from e-commerce platforms, with latent Dirichlet allocation extracting latent themes to build a multi-dimensional indicator system. The fuzzy analytic hierarchy process quantifies qualitative indicators to form an initial judgment matrix, while particle swarm optimization performs global optimization and consistency correction to minimize expert bias. An empirical case on smartwatches verifies the feasibility and effectiveness of the proposed framework, demonstrating its ability to capture user-centered demand characteristics and produce comprehensive evaluation results. The proposed evaluation system effectively combines demand insights with market adaptability, offering a robust theoretical and methodological foundation for product design optimization and differentiated strategy formulation.Supplementary InformationThe online version contains supplementary material available at https://doi.org/10.1038/s41598-026-55759-4. Introduction The smart wearable device industry, particularly the smartwatch market, is experiencing sustained and rapid growth. As user scenarios and lifestyles diversify, consumer demands for smartwatches have become increasingly multifaceted. These demands extend beyond core functional requirements such as health monitoring accuracy, communication capabilities, and battery life to encompass subjective preferences like interactive experience, ecosystem compatibility, and emotional design. These needs exhibit distinct dynamism and show significant variation across different user groups. Recent advancements in hybrid decision models across various domains demonstrate potential for addressing similar complexities. For instance, the application of spherical fuzzy SWARA-WASPAS in prioritizing low-carbon policies1 and the ESG evaluation framework based on fuzzy Delphi and cluster analysis showcase robust approaches for handling uncertainty and conflicting criteria2. These studies provide a solid theoretical foundation for fuzzy judgment and optimization techniques. However, their application in the dynamic evaluation of smart wearable products incorporating real-time user feedback remains to be explored. Current research on smart wearable devices, particularly smartwatches, primarily focuses on two directions: first, exploring consumers’ purchase intentions and usage attitudes toward smartwatches3–5; second, examining technological advancements in smartwatches, such as sensor technology, interactive features, or algorithmic performance6–8. However, systematic design evaluation research remains relatively scarce for consumer electronics like smartwatches—products characterized by multidimensionality, strong interactivity, and rapid iteration. Although some scholars have attempted to incorporate statistical methods into product design evaluation, such as Yonaha9 who developed a design support tool using product databases and design feature metrics. Baghirov and Zhang,10 explored the relationship between aesthetic design and product attitude through mediating and moderating effect analyses. Wei and Shi11 employed FMEA-TOPSIS to construct a perceived evaluation system for intelligent product service failures. Sun et al.,12 proposed an image fuzzy set method integrating TOPSIS-GRP to enhance the accuracy of conceptual design evaluation. However, most of the aforementioned studies have failed to transcend the limitations of static metric systems. Their evaluation dimensions are often predefined, unable to automatically capture emerging themes from vast volumes of user reviews. Simultaneously, these approaches exhibit significant shortcomings in integrating objective parameters with subjective textual feedback, rendering them ill-equipped to address the multimodal data evaluation challenges posed by complex products like smartwatches. To address these challenges, this study employs an LDA–FAHP–PSO integrated evaluation model for smartwatch design assessment, aiming to resolve key issues in this field such as dynamic shifts in user needs and difficulties in multimodal data fusion. As a comprehensive wearable device integrating health monitoring, communication, and entertainment, smartwatch evaluation encompasses not only hardware performance but also subjective factors like interaction experience, ecosystem compatibility, and emotional design. Traditional methods struggle to provide a comprehensive characterization. Based on this, this study constructs a comprehensive evaluation system spanning from data-driven indicator generation to weight optimization: First, LDA is employed to automatically extract latent themes from user reviews, establishing a multidimensional indicator system aligned with genuine needs. Subsequently, FAHP integrates experts’ fuzzy judgments on qualitative indicators. Finally, PSO is introduced to perform global consistency optimization on the judgment matrix, thereby significantly reducing subjective bias and enhancing the robustness of weight allocation. Although previous research13 has applied this model to the evaluation of walking aids for the elderly and validated its universality. However, the application of this study within the smartwatch context highlights the model’s distinctive value: firstly, the smartwatch user base is broader and more diverse in its requirements, evolving rapidly with technological iterations. LDA’s capability for dynamic theme identification enables it to effectively capture multi-dimensional concerns such as health monitoring, battery performance, and system ecosystem. Secondly, this study further constructs a comprehensive closed-loop system encompassing ‘theme extraction—fuzzy judgement—consistency optimisation’, incorporating additional elements such as robustness validation and sensitivity analysis to enhance the transparency and reproducibility of this composite methodology. Finally, this study also demonstrates a comprehensive end-to-end evaluation process, encompassing dynamic indicator generation, weight optimisation, fuzzy comprehensive evaluation, and multi-source validation mechanisms. This approach differs from the more theory-framework-oriented presentation adopted by13. By focusing on smartwatch application scenarios and refining validation mechanisms, this study not only achieved effective identification of user requirements and quantitative assessment of key design factors, but also demonstrated the advantages of the LDA–FAHP–PSO evaluation model in addressing complex, dynamic product evaluation tasks. It provides an actionable theoretical foundation and practical pathway for smartwatch product iteration and design innovation. In summary, the main contributions of this study are as follows: Proposing the LDA–FAHP–PSO integrated framework for application in the field of smartwatch design evaluation;Conducting empirical validation based on real smartwatch data to demonstrate the model’s effectiveness in practical scenarios;Presenting a transparent, reproducible hybrid MCDM workflow that can be extended to the comprehensive evaluation of other complex systems. Proposing the LDA–FAHP–PSO integrated framework for application in the field of smartwatch design evaluation; Proposing the LDA–FAHP–PSO integrated framework for application in the field of smartwatch design evaluation; Conducting empirical validation based on real smartwatch data to demonstrate the model’s effectiveness in practical scenarios; Conducting empirical validation based on real smartwatch data to demonstrate the model’s effectiveness in practical scenarios; Presenting a transparent, reproducible hybrid MCDM workflow that can be extended to the comprehensive evaluation of other complex systems. Presenting a transparent, reproducible hybrid MCDM workflow that can be extended to the comprehensive evaluation of other complex systems. The remainder of this paper is structured as follows: section “Methods” reviews relevant theoretical and methodological research; section “Approach overview” introduces the proposed LDA–FAHP–PSO product evaluation framework; section “Test steps and data check” details data sources and experimental setup; section “Tests and findings” presents empirical results; section “Discussion and outlook” summarizes the findings and discusses future research directions. Methods Latent dirichlet allocation (LDA) The topic model is a three-layer Bayesian model consisting of “document-topic-word,” which ultimately reveals the text information in the form of “document-topic” distribution and “topic-vocabulary” distribution. As a bag-of-words model, LDA ignores the order in which words appear in a document. Additionally, the LDA model assumes that each document contains multiple hidden topic information, with each document following a multinomial distribution of topics, and each topic following a multinomial distribution of vocabulary. In recent years, an increasing number of studies have sought to integrate user-generated content (UGC) with multi-criteria decision-making (MCDM) methodologies for product evaluation and decision support. Some research employs sentiment analysis or topic extraction techniques to identify user concerns from online reviews. For instance, Huang et al.,14 utilised LDA to discern technological topic evolution trends. Xue,15 applied LDA topic mining to circular economy product research. Within consumer product contexts, Parker et al.,16 and Tian et al.,17 demonstrated how topic modelling can extract latent user demand structures from large-scale textual data. However, most of these studies employ text mining merely as a descriptive tool or as supplementary input for simple ranking methods, without directly embedding textual themes into a systematic framework for determining MCDM weights. More importantly, existing text-assisted MCDM methods typically rely on predefined lists of metrics or simplistic sentiment scores, making it difficult to dynamically reflect shifts in user demand trends. Most frameworks lack mechanisms to address inconsistencies in expert judgements and, more critically, lack a unified evaluation system that integrates textual insights with structured product parameters. Consequently, these approaches struggle to meet the evaluation demands of complex products like smart wearables within multimodal, high-dimensional, and rapidly evolving data environments. Therefore, the LDA topic model, as a mature and reliable probabilistic statistical tool, is particularly suitable for topic identification and feature mining in large-scale text. The principle and process of text recognition using the LDA topic model are illustrated in Fig. 1. However, in product design research, there are still shortcomings in constructing a systematic mechanism for identifying product attributes from the user’s perspective. This study employs smartwatches as a case study, applying the LDA model to extract latent themes from user-generated content. This process establishes a systematic evaluation framework, providing both data support and methodological foundations for product refinement and user experience optimisation. Fig. 1LDA text extraction diagram. LDA text extraction diagram. At its heart, LDA operates on the principle that how often a keyword pops up in a document can be explained by a joint probability distribution. This distribution is swayed both by how likely that word is to show up in a particular topic and how likely that topic is to be present in the document itself. Think of a dataset of shopping reviews: if we let M stand for the total number of reviews, W for the complete vocabulary of keywords, and T for the number of underlying topics, then φt helps us pin down the distribution of keywords within each topic, while θm gives us a sense of the topic mix in any given review. When we zoom in on the individual words, we can say that ωm, n represents the nth word within the mth comment. Think of τm, n as the topic tag slapped onto that specific word. With these labels in hand, we can build a model to understand how keywords and topics intertwine. The probability of seeing a particular combination can then be worked out using this model:1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P\left( {\theta \left. {,\tau ,\omega } \right|\alpha ,\beta } \right) = p\left( {\left. \theta \right|\alpha } \right)\mathop \prod \limits_{n}^{{N_{m} }} p\left( {\left. {\tau _{n} } \right|\theta } \right)p\left( {\left. {\omega _{n} } \right|\tau _{n} ,\beta } \right)$$\end{document} In this probabilistic model, the hyperparameters α and β are used to regulate the sparsity of the topic distribution and term distribution, respectively. Specifically, first, a Dirichlet distribution with α as the prior can be used to generate the corresponding topic distribution θm for each document. Then, each word in the document is assigned to a specific topic based on the probability structure of θm, thereby obtaining the corresponding topic label τm, n. Additionally, a Dirichlet distribution with β as the prior determines the word distribution φt within each topic, and based on this, specific terms in the document are sampled and generated, resulting in the final word ωm, n. Particle swarm optimization (PSO) In the mid-1990s, inspired by Heppner’s bird flock habitat model, Kenney and Eberhart proposed the particle swarm algorithm18. The concept behind the algorithm is this: initially, each particle’s location within the population is tossed into the decision vector space of the issue at random. Afterward, the particles take to the skies and navigate through the search space using a specific set of rules. This biomimetic algorithm, which uses particles to mimic the foraging behavior of birds, represents a solution to the optimization problem based on the coordinates of the particles’ positions. The fitness value of the particles determines the quality of the solutions they represent19. In algorithm design, Kenmey mimics the information-sharing behavior among bird individuals to establish rules for information exchange between particles. In each generation selection, all particles must calculate their fitness values based on the fitness function at each iteration step. By sorting the fitness values from the historical records of all particles, the global optimal position gBest is determined, and the individual optimal position pBest is determined based on the historical records of each particle. As can be seen, the PSO algorithm’s optimization process is similar to birds searching for food, both guided by the best individuals and their own best positions within the population20. Figure 2 illustrates the execution process of PSO. Fig. 2PSO algorithm flowchart. PSO algorithm flowchart. The Particle Swarm Optimisation algorithm, as a quintessential swarm intelligence-inspired method, is widely applied to complex optimisation problems owing to its structural simplicity, high search efficiency, and robust global optimisation capabilities. Pham and Raahemi,19 note in their systematic review that PSO demonstrates robust performance when handling high-dimensional, multi-constraint, and non-linear optimisation tasks. Consequently, it is frequently employed for weight learning and consistency correction within multi-criteria decision-making (MCDM). Against this backdrop, PSO has been widely employed to optimise weight allocation in supplier selection within supply chain systems21,22. These studies essentially utilise PSO to address expert subjectivity and optimise consistency in decision matrices. This, of course, represents a key strength of PSO. Meanwhile, the application boundaries of PSO continue to expand: in engineering optimisation, Mesquita and Gaspar20, Liu et al.,23, and Norouzi and Bazargan24 have respectively applied PSO to unmanned aerial vehicle path planning, water pollution simulation, and permafrost distribution prediction, thereby enhancing computational efficiency. In the field of image and signal processing, Li et al.,25 enhanced image restoration performance based on PSO. Regarding network and system optimisation, Guler,26 introduced PSO for spectrum scheduling in blockchain-enhanced optical networks, while Carstensen and Lin27 employed PSO for efficient utility set mining. Puma et al.,28 employed HT-PSO to improve convergence stability in power grid reconfiguration. The aforementioned results demonstrate the PSO’s capability in handling non-convex search spaces, complex constraints, and noisy data, further validating its theoretical reliability as a tool for optimising judgement matrix consistency and weight learning. Building upon the aforementioned research foundations, this study incorporates the Particle Swarm Optimisation (PSO) algorithm into the Fuzzy Analytic Hierarchy Process (FAHP). By implementing parametric encoding of the fuzzy complementary judgement matrix, the PSO performs global search under consistency constraints. This approach mitigates expert subjectivity, thereby enhancing the accuracy and stability of weight allocation. Unlike traditional FAHP, which relies on manual calibration for consistency, the PSO-driven collaborative optimisation mechanism is better suited to high-dimensional scenarios involving multiple expert inputs. This constitutes the key technological foundation of the LDA–FAHP–PSO integrated evaluation framework. In the PSO algorithm, a swarm consisting of N particles is formed in a D-dimensional target search space. Each particle in the search space corresponds to a solution to an optimization problem. Each particle has two attributes: velocity and position. Velocity represents the speed at which the particle moves, while position represents the direction of the particle’s movement. The particle position is expressed as in Formula (2):2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${X_i}=[{x_{i1}},{x_{i2}},{x_{i3}},...,{x_{iD}}]$$\end{document} In this context, D denotes the dimension of the search space; N denotes the number of particles; and Xi is a D-dimensional vector representing the position of the i-th particle. At the initial stage, all particles in the search space are randomly initialized, each with a random initial velocity and position. Subsequently, they calculate their fitness values based on the objective function and update their positions accordingly. Then, multiple selections are performed. After each selection, each particle finds an individual optimum pbesti. The entire population converges on a global optimum, represented by the gbesti. Each particle combines these two optima for the next iteration, updating its velocity and position. The update process is shown in Eqs. (3) and (4).3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\nu _i}=\omega \times {\nu _i}+{c_1} \times rand\left( \cdot \right) \times \left( {pbes{t_i} - {x_i}} \right)+{c_2} \times rand\left( \cdot \right) \times \left( {gbes{t_i} - {x_i}} \right)$$\end{document}4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${x_i}={x_i}+{v_i}$$\end{document} In this context, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$rand\left( \cdot \right)$$\end{document}denotes a random number in the range [0,1]; vi represents the velocity of the i-th particle; c1 and c2 are the individual learning factor and global learning factor, respectively, which enhance the particles’ ability to optimize; ω is the particle inertia weight, which controls the rate of change in particle motion, thereby balancing local and global optimal solutions. As can be seen from the update formula, the updated velocity vi consists of three parts. The first part is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\omega \times {\nu _i}$$\end{document}, which is used to maintain the original trend of the particle and can control the rate of change in particle motion, thereby balancing the local optimal and global optimal solutions. The second part is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${c_1} \times rand\left( \cdot \right) \times \left( {pbes{t_i} - {x_i}} \right)$$\end{document}, where the learning factor reflects the particle’s preference for the individual extreme value, and its value affects the convergence of the algorithm. The third part is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${c_2} \times rand\left( \cdot \right) \times \left( {gbes{t_i} - {x_i}} \right)$$\end{document}, which represents the particle’s ability to tend toward the best position in the population history. Through this approach, the particle swarm algorithm leverages information sharing among particles to perform optimized searches within the problem-solving space, thereby enhancing the efficiency and accuracy of problem-solving and ultimately obtaining the optimal solution. Optimizing the FAHP model The Analytic Hierarchy Process (AHP) was proposed by American scholar Saaty29 in 1973. Its basic idea is to establish a hierarchical decision-making framework to convert qualitative judgments on complex issues into quantifiable analytical results. With its excellent systematicity, flexibility, and scalability, AHP has become one of the most widely used classic methods in multi-criteria complex decision-making. The key lies in constructing a consistent judgment matrix. However, traditional AHP is often limited by cognitive fuzziness when expressing the subjective preferences of decision makers. To address this shortcoming, Van Laarhoven and Pedrycz30 were the trailblazers who first brought to light the concept of Triangular Fuzzy Numbers (TFN), thereby crafting the Fuzzy Analytic Hierarchy Process (FAHP). This innovative process has emerged as a powerful tool for grappling with uncertain information. Building on this foundation, subsequent studies have consistently sought to refine the methodology. Take, for instance, Chang,31, who introduced a novel weight calculation model grounded in the Extent Analysis Method (EAM), now known as geometric fuzzy AHP. On the other hand, achieves consistency testing through geometric mean closure operations on fuzzy numbers. Both of these operations require the design of complex fuzzy number multipliers. Therefore, considering the difficulty of combining PSO encoding in subsequent steps, this study will adopt the FAHP method based on fuzzy consistency matrices. Fuzzy Analytic Hierarchy Process (FAHP) has demonstrated significant application value in multiple fields. In the field of power safety, Sadeghi-Yarandi et al.,32 proposed the EISRI model, which is based on FAHP and establishes a three-dimensional quantitative assessment system for work risks covering individuals, the environment, and organizations. In urban transportation planning, Kramar and Sternad33 employed a triple integration method that combines NGT-FGs and FAHP to identify and prioritize sustainable transportation obstacles systematically. In the field of marine environmental management, Zhang et al.,34 combined FAHP with sensitivity analysis to provide a quantitative framework for prioritizing coastal and marine pollution sources, identifying chemical pollutants as the primary risk factors. At the same time, methodological innovations have driven the dynamic and intelligent development of FAHP. For example, Xie et al.,35 integrated AHP and FAHP to construct a quantitative assessment model for shield machine adaptability and verified its effectiveness through on-site construction. Mostafa et al.,36 combined FAHP with the Delphi method, significantly improving the accuracy of weighting economic criteria in forest management. Fan and Ma,37 proposed a framework for integrating FAHP with ecosystem service quantification, effectively coordinating the conflicting priorities of ecological protection and spatial development in the Shenyang metropolitan area. Despite significant progress, FAHP still has limitations, including a heavy reliance on expert judgment and a lack of dynamic adaptation mechanisms. Its static analysis characteristics are insufficient for dealing with the nonlinear evolution of complex systems. Therefore, this study combines FAHP with particle swarm optimization (PSO) to enhance its dynamic adaptability and computational stability by introducing intelligent optimization mechanisms, thereby constructing a more robust and objective product evaluation model. As a typical representative of swarm intelligence optimization methods, the particle swarm optimization algorithm demonstrates significant advantages when dealing with high-dimensional, nonlinear optimization problems. Compared with genetic algorithms (GA). Particle Swarm Optimization, known for its ease of parameter configuration, rapid convergence, and robust global search skills, was ingeniously integrated into the Fuzzy Analytic Hierarchy Process (FAHP) in this research. We developed a novel FAHP-PSO cooperative optimization scheme. This approach tackles the consistency adjustment issue in fuzzy judgment matrices by framing it as a constrained, nonlinear optimization challenge. To begin with, a fitness metric was crafted to measure the level of consistency within the matrix. Next, we leverage an iterative search strategy via particle swarm optimization to nudge us closer to the optimal outcome bit by bit. This approach adeptly transforms the consistency adjustment of the fuzzy judgment matrix and the hierarchical weight assignment in the FAHP into manageable nonlinear optimization challenges. As a result, the model churns out a perfectly consistent fuzzy judgment matrix and delivers a precise ranking of all evaluation factors. This not only enhances the system’s stability but also elevates its scientific credibility. (Subject recruitment for this study took place between 01/05/2025 and 30/06/2025. All participants signed written informed consent forms prior to participation, and no minors were involved. Both the research protocol and informed consent procedures received approval from the Ethics Committee of Quanzhou Normal University.) Approach overview This study conducts a systematic survey and data mining of the smartwatch market based on large-scale user-generated content (UGC) in the B2C e-commerce model. First, Python was used to crawl online reviews from multiple e-commerce platforms. After multi-level data cleaning and preprocessing, text segmentation and part-of-speech tagging were performed using Jieba, followed by topic modeling and visualization analysis using the pyLDAvis tool. This process identified the core concerns of consumers when purchasing smartwatches, such as health monitoring accuracy, battery life, and user experience. On this basis, a three-dimensional evaluation index system encompassing functional performance and human-computer interaction was constructed, drawing on relevant international standards, ergonomic parameters, professional literature, and Delphi expert opinions (see Fig. 3). To further enhance the scientific rigor and dynamic adaptability of the evaluation system, this study proposes a FAHP–PSO collaborative optimization framework. In this framework, the FAHP employs λ-cut aggregation constraints to correct the consistency of the judgment matrix. At the same time, the PSO algorithm utilizes a global search mechanism with a dynamic inertia factor to efficiently optimize in high-dimensional spaces, thereby obtaining the optimal weight distribution. This enables cross-modal mapping between LDA semantic features and engineering parameters. The model solution results reveal the multi-level weight relationships of key design factors in smartwatches, providing reliable quantitative support for product optimization, market positioning, and differentiated competitive strategies. Fig. 3Utilizing the LDA-FAHP-PSO analytical framework. Utilizing the LDA-FAHP-PSO analytical framework. Step-by-step structure of the whole checking mark system In order to build the evaluation framework, this study uses a step-by-step, top-down approach and divides it into three clearly defined levels (Fig. 4). The first layer is the objective layer, which serves as the core of the entire system and contains only one element (denoted as P) to represent the overall objective of the comprehensive evaluation. The second layer is the criterion layer (B), which further elaborates on the objective layer, encompassing the key dimensions and primary standards that must be followed to achieve the overall objective. As a third-level system, the secondary standard layer connects the past and separates the core principles into more refined components and measurement indicators, thereby ensuring that the evaluation system is logically rigorous and structurally systematic. Fig. 4Hierarchical framework of the comprehensive evaluation index system. Hierarchical framework of the comprehensive evaluation index system. Construction of the fuzzy complementary judgment matrix In the application of FAHP, quantifying qualitative factors is a critical step, and establishing an appropriate scaling system is a prerequisite for achieving this goal. After a comprehensive review of the existing literature, we found that current expansion methods can be roughly divided into two categories: inverse scaling or complementary scaling. Among these, the 0.1–0.9 scale, due to its simplicity and effectiveness in practical applications, is widely used in constructing fuzzy complementary judgment matrices and has gained widespread recognition in the academic community. This study uses the range of 0.1 to 0.9 detailed in Table 1 as the measurement scale. This move aims to assign quantitative values to various evaluation indicators to ensure that the model is both scientific and rigorous and logically self-consistent. Table 1The 0.1–0.9 rating way and its effects.ScaleDefinitionDescription0.5Pairwise comparison of elementsBoth parts are the same in importance0.6Pairwise comparison of elementsOne part is a little more important0.7Pairwise comparison of elementsOne part is somewhat more important0.8Pairwise comparison of elementsne part is much more important0.9Pairwise comparison of elementsOne part is very much more important than the other0.1–0.4Reverse comparisonIf element Pi is compared with Pj to obtain bij, then the reciprocal comparison of Pj with Pi is given by 1 − bij Table 1The 0.1–0.9 rating way and its effects.ScaleDefinitionDescription0.5Pairwise comparison of elementsBoth parts are the same in importance0.6Pairwise comparison of elementsOne part is a little more important0.7Pairwise comparison of elementsOne part is somewhat more important0.8Pairwise comparison of elementsne part is much more important0.9Pairwise comparison of elementsOne part is very much more important than the other0.1–0.4Reverse comparisonIf element Pi is compared with Pj to obtain bij, then the reciprocal comparison of Pj with Pi is given by 1 − bij The 0.1–0.9 rating way and its effects. If element Pi is compared with Pj to obtain bij, then the reciprocal comparison of Pj with Pi is given by 1 − bij To eliminate potential ambiguity, we clarify that this study adopts a single-value fuzzy complementary scale rather than triangular fuzzy numbers. Each expert’s pairwise comparison between indicators Pi and Pj is directly mapped to a scalar value bij ∈ [0.1,0.9], representing the degree to which Pi is preferred over Pj. The mapping rules follow the fuzzy complementary judgment framework: 0.5: equal importance.0.6: slightly more important.0.7: more important.0.8: much more important.0.9: extremely more important.0.1–0.4: reverse comparisons such that 0.5: equal importance. 0.5: equal importance. 0.6: slightly more important. 0.6: slightly more important. 0.7: more important. 0.7: more important. 0.8: much more important. 0.8: much more important. 0.9: extremely more important. 0.9: extremely more important. 0.1–0.4: reverse comparisons such that 0.1–0.4: reverse comparisons such that 5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${b_{ji}}=1 - {b_{ij}}$$\end{document} Based on this scale, the fuzzy complementary judgment matrix Y = [yij] is constructed as:6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${y_{ij}}={b_{ij}},\quad {y_{ji}}=1 - {b_{ij}},\quad {y_{ii}}=0.5$$\end{document} This formulation ensures that all matrices satisfy the axioms of fuzzy complementary relations and remain compatible with the subsequent FAHP–PSO consistency optimization. Since the scale is expressed using scalar values rather than triangular fuzzy numbers, no defuzzification step is required, and the consistency index can be computed directly using the Frobenius-norm–based measure defined in section “Consistency analysis and PSO-based correction for fuzzy judgment matrices”. Under this scaling rule, if an element is compared with itself, its importance value is 0.5. When \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P_{ij}} \in [0.1,0.5]$$\end{document}, it indicates that indicator bi is more important; conversely, when \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P_{ij}} \in [0.5,0.9]$$\end{document}, it indicates that indicator bi is more important than bj. By completing pairwise comparisons of indicators based on the above numerical intervals, an unclear matching decision table can finally be made:7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P={\left( {{P_{ij}}} \right)_{n \times n}}=\left[ {\begin{array}{*{20}{c}} {{P_{11}}}&{{P_{12}}}& \cdots &{{P_{1n}}} \\ {{P_{21}}}&{{P_{22}}}& \cdots &{{P_{2n}}} \\ \cdots & \cdots & \cdots & \cdots \\ {}&{}&{}&{} \\ {{P_{n1}}}&{{P_{n2}}}& \cdots &{{P_{nn}}} \end{array}} \right]$$\end{document} Consistency analysis and PSO-based correction for fuzzy judgment matrices Using the FAHP method, we can build a fuzzy judgment matrix that quantifies the relative importance of pairs of elements. If the matrix meets specific criteria, it can be classified as a fuzzy consensus judgment matrix. By definition, a matrix P can be said to be completely consistent only if it has the following characteristics: 8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${p_{ij}}+{p_{ji}}=1$$\end{document} Complementarity: 9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${p_{ii}}=0.5$$\end{document} Consistency condition: 10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${p_{ij}}={p_{ik}} - {p_{jk}}+0.5$$\end{document} Equation (10) is a sufficient condition for Eqs. (8) and (9), indicating that the relative relationships between different elements can be conveyed through quantitative forms. Under this condition, w represents the weight values of each element. This value reveals the difference in weight between rows indexed ‘i’ and rows indexed ‘j’in the matrix. The parameter ‘b’ is a secret one. It is specifically used to quantify the difference between any two components. When b is small, it means that decision makers are more sensitive to differences; when b is large, it means that differences have less influence. If the fuzzy matrix satisfies the following equation, it can be regarded as entirely consistent: 11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${p_{ij}}=0.5\left( {1+b({\omega _i} - {\omega _j})} \right)$$\end{document} However, when we put complex systems into practical applications, the judgment matrix is often difficult to perfect because the problems are inherently intertwined, and personal opinions inevitably vary. Therefore, FAHP generally only requires the matrix to have acceptable consistency. If the matrix P fails to meet the established criteria, it needs to be corrected. To this end, this study uses the Particle Swarm Optimization algorithm to evaluate and optimize the consistency of the fuzzy judgment matrix. The modified matrix is denoted Y. At this point, the weight vectors corresponding to each evaluation criterion are ωY, and they satisfy the following conditions: 12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$CI(Y) = \frac{1}{{n^{2} }}\mathop \sum \limits_{{i = 1}}^{n} \mathop \sum \limits_{{j = 1}}^{n} \left| {y_{{ij}} - 0.5(1 + b(\omega _{i} - \omega _{j} ))} \right|$$\end{document} Among these, CI is the consistency indicator function, which measures the deviation between the matrix and perfect consistency. By leveraging the global search capabilities of PSO, the matrix can be iteratively optimized in high-dimensional space, gradually approaching the optimal solution. When the consistency indicator value is less than 0.1, the matrix is considered to have achieved an acceptable level of consistency; if the global minimum is reached, it indicates that the matrix has achieved perfect consistency. Explicit formulation of consistency reconstruction and PSO-based correction To further improve the consistency of the fuzzy judgment matrix and reduce subjective bias from expert assessments, this study incorporates PSO into the FAHP consistency analysis. This subsection explicitly presents the reconstruction of a fully consistent matrix, the mathematical definition of the consistency index, and the encoding and constraint-handling mechanisms used in PSO, ensuring the reproducibility and transparency of the correction process. Reconstruction of a fully consistent matrix. Reconstruction of a fully consistent matrix. Reconstruction of a fully consistent matrix. Let the original fuzzy judgment matrix be:\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:Y=\left[{y}_{ij}\right],{\hspace{1em}}{y}_{ij}\in\:\left[\mathrm{0,1}\right].$$\end{document}According to the reciprocal-complementary scale used in FAHP, a fully consistent fuzzy judgment matrix can be reconstructed using the weight vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$w = (w_{1} , \ldots ,w_{n} ),\quad w_{i} \ge 0,\mathop \sum \limits_{i} w_{i} = 1$$\end{document} through the following proportional relationship:13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y_{{ij}}^{*}\left( w \right)=\frac{{{w_i}}}{{{w_i}+{w_j}}}$$\end{document} This formulation simultaneously satisfies the reciprocal condition \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y_{{ij}}^{*}+y_{{ji}}^{*}=1$$\end{document}and the transitivity condition, and therefore provides a theoretical basis for consistency correction. (2) Consistency Index (CI) (2) Consistency Index (CI) Consistency Index (CI) To quantify the deviation between the original judgment matrix and the reconstructed consistent matrix, a normalized Frobenius-based consistency index is adopted:14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$CI\left( {Y,w} \right)=\frac{{\parallel Y - {Y^*}\left( w \right){\parallel _F}}}{{\parallel Y{\parallel _F}}}=\frac{{\mathop \sum \nolimits_{{i,j}} {{\left( {{y_{ij}} - y_{{ij}}^{*}\left( w \right)} \right)}^2}}}{{\mathop \sum \nolimits_{{i,j}} y_{{ij}}^{2}}}$$\end{document} A judgment matrix is considered acceptably consistent when CI < 0.10. (3) PSO-based consistency correction (3) PSO-based consistency correction PSO-based consistency correction Based on the above reconstruction model, the optimization objective of PSO is explicitly defined as:15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathop {\hbox{min} }\limits_{w} CI\left( {Y,w} \right)$$\end{document} After each position update, the weight vector is projected back into the feasible domain to satisfy non-negativity and the unit-sum constraint:16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${w_i} \leftarrow \hbox{max} \left( {0,{w_i}} \right),{w_i} \leftarrow \frac{{{w_i}}}{{\mathop \sum \nolimits_{i} {w_i}}}$$\end{document} This projection-based approach ensures that the weight vector always remains valid during optimization, without the need for additional penalty terms. Optimization framework of the FAHP model based on PSO This section builds on the preceding discussion to construct an integrated FAHP-PSO optimization framework (Fig. 5). By introducing PSO to perform consistency correction and weight optimization on the fuzzy judgment matrix of FAHP, this framework not only solves the stability issues caused by expert subjective bias in traditional FAHP but also improves the scientific nature of weight allocation and the convergence performance of the model. This framework logically implements a closed-loop process, spanning from fuzzy decision modeling to consistency verification and weight optimization, ensuring that evaluation results remain highly reliable and interpretable, even in complex, multidimensional product contexts. Compared to traditional single-method approaches, the FAHP-PSO optimization framework provides more convincing quantitative support for subsequent empirical research in the context of multi-source information fusion. Fig. 5Optimization framework of the FAHP model based on PSO. Optimization framework of the FAHP model based on PSO. Test steps and data check This section takes market research on smartwatches in the B2C model as its research background. First, the sample data required for the study are determined. Subsequently, user evaluation information is collected, and the LDA topic model is used to perform in-depth analysis of the text data, identifying key features that users focus on when purchasing smartwatches, and to conduct cluster analysis. Based on this, combined with the data analysis results, a design evaluation index system for smartwatches is constructed to provide a scientific and reasonable basis for decision-making in product optimization and market positioning. Obtaining user needs based on text mining The survey results indicate that smartwatch sales are mainly conducted through three channels: O2O, B2C, and B2B. The current market offers a wide range of smartwatch products that differ in design, functionality, and market positioning, reflecting the heterogeneous needs of different consumer groups. Regarding online data collection, existing approaches generally fall into two categories: customised web crawlers and general-purpose scraping tools. To ensure flexibility, reproducibility, and scalability, this study employed a Python-based data collection and analysis framework. All online product and review data were collected exclusively from JD.com (Jingdong), one of the largest and most representative e-commerce platforms for consumer electronics in China. A search strategy centred on the core keyword “smartwatch” was adopted, and product popularity indicators provided by the platform (e.g., sales ranking and review volume) were used as screening criteria to identify representative products and capture mainstream market trends and user preferences. The data collection process was conducted over the period from [2025.04] to [2025.06], ensuring temporal consistency of product information and user reviews. Prior to data collection, key attributes were explicitly defined, including product name, price, sales volume, number of reviews, manufacturer information, and delivery regions. A fixed crawl depth strategy was applied, extracting product information from the top ten pages of search results on JD.com, as ranked by the platform’s default relevance and recency mechanisms. This approach balances data representativeness with computational efficiency. Following data standardisation and export to Excel format, Python scripts were further deployed to crawl user-generated review texts associated with the selected smartwatch products. After systematic data cleaning procedures—including the removal of duplicate entries, promotional content, and non-informative comments—and consistency validation, approximately 3,000 valid and high-quality user review records were retained for subsequent analysis. Owing to the platform’s privacy protection policies, detailed demographic attributes of reviewers (such as age, gender, or income level) were not publicly accessible, making explicit demographic stratification infeasible. Consequently, the analysis focused on experiential and functional content embedded in user reviews rather than demographic segmentation, a limitation that is acknowledged in this study. Data before work User-generated review data collected from e-commerce platforms often contain redundant or irrelevant information, such as system-generated default reviews or duplicated content resulting from repeated user submissions. To enhance the reliability and validity of the dataset, a series of preprocessing steps were applied to the raw text data. These steps included removing stop words, unifying text formats, and eliminating semantic redundancy. Through these procedures, noise within the review corpus was effectively reduced, ensuring that the remaining data accurately reflected users’genuine perceptions and usage experiences of smartwatches. The categories of stop words removed during text preprocessing are summarized in Table 2. Table 2Categories of stop words.Stop word categoryStop word typeCommon expressionsReceive, Purchase, Recommend, Experience, Delivery,Expect, Service, Feedback, Next time …Numerical characters0 1 2 3 4 5 6 7 8 9Punctuation & special marks? ~, ! ……Note: This table lists the main types of stop words excluded during preprocessing, including frequently used expressions, numerical characters, punctuation, and other special symbols. Table 2Categories of stop words.Stop word categoryStop word typeCommon expressionsReceive, Purchase, Recommend, Experience, Delivery,Expect, Service, Feedback, Next time …Numerical characters0 1 2 3 4 5 6 7 8 9Punctuation & special marks? ~, ! ……Note: This table lists the main types of stop words excluded during preprocessing, including frequently used expressions, numerical characters, punctuation, and other special symbols. Categories of stop words. Receive, Purchase, Recommend, Experience, Delivery, Expect, Service, Feedback, Next time … Note: This table lists the main types of stop words excluded during preprocessing, including frequently used expressions, numerical characters, punctuation, and other special symbols. Optimal topic selection and visualization in LDA modeling After completing text cleaning and standardization, this study employed the topic coherence metric to determine the appropriate number of topics for the LDA model. Topic coherence measures the semantic consistency and interpretability of topic partitions, with higher values indicating more semantically concentrated and distinguishable topic structures. As shown in Fig. 6, when the number of topics increases from 3 to 5, the coherence value rises markedly, indicating a substantial improvement in topic quality. Although further increases in the number of topics (K = 6–10) lead to minor fluctuations and a gradual upward trend, the overall growth rate of coherence becomes noticeably flatter, with the maximum value observed at K = 10 (approximately 0.46). It should be noted, however, that topic number selection in this study was not based on coherence alone. While larger K values may yield marginally higher coherence scores, an excessive number of topics tends to fragment semantically related content, resulting in redundant or overly fine-grained themes. Such semantic dispersion reduces interpretability and complicates the construction of a concise and stable evaluation hierarchy for subsequent multi-criteria decision-making analysis. Considering the trade-off between statistical coherence, semantic separability, and practical applicability to FAHP-based indicator construction, this study selected five topics as a balanced and methodologically appropriate solution. This choice preserves the core experiential structure of user evaluations while avoiding unnecessary analytical complexity, thereby better supporting the development and application of the smartwatch product evaluation indicator system. Fig. 6Topic coherence trend of the LDA model under different topic numbers. Topic coherence trend of the LDA model under different topic numbers. To further examine the semantic structure and distinctiveness of the extracted topics, this study employed LDAvis, an interactive topic visualization tool proposed by Sievert and Shirley38, which enables intuitive inspection of inter-topic relationships and term relevance. LDAvis integrates term frequency and term distinctiveness as key measures of word–topic relevance, thereby enhancing the interpretability of topic modeling results. In this study, the number of LDA topics was fixed at five. During visualization and interpretation, the relevance parameter λ was adjusted to balance frequency-based and exclusivity-based keyword representation (replacing the original parameter notation to avoid ambiguity). Specifically, when λ approaches 1, the visualization emphasizes high-frequency terms that may occur across multiple topics;when λ approaches 0, the visualization highlights terms that are highly specific to individual topics, even if their overall frequency is relatively low. when λ approaches 1, the visualization emphasizes high-frequency terms that may occur across multiple topics; when λ approaches 1, the visualization emphasizes high-frequency terms that may occur across multiple topics; when λ approaches 0, the visualization highlights terms that are highly specific to individual topics, even if their overall frequency is relatively low. when λ approaches 0, the visualization highlights terms that are highly specific to individual topics, even if their overall frequency is relatively low. To ensure clarity and thematic discriminability, a keyword extraction strategy was applied in the final modeling stage. For each topic, keywords were selected to summarize its core semantic focus, and the eight most representative terms with the highest relevance weights were retained. Although certain topics (e.g., Performance & Hardware and Connectivity & Features) are functionally related, they capture distinct layers of user perception—namely intrinsic device performance versus system-level interaction and ecosystem integration—and were therefore maintained as separate dimensions to support finer-grained evaluation and targeted weighting in the subsequent FAHP analysis. The resulting topic keywords and descriptions are presented in Table 3. LDA-based topic extraction of smartwatch user reviews To enhance the clarity and interpretability of topic representation, this study further refined a keyword extraction strategy based on the LDA modeling results. For each topic, the most representative keywords were selected by jointly considering term frequency and topic-specific weight values. Specifically, the top six high-weight words under each topic were retained as representative keywords, ensuring that each theme is characterized by a concise yet semantically informative vocabulary set. This approach highlights the core semantic focus of each topic while avoiding redundancy and excessive fragmentation. The final topic classification and corresponding keywords are presented in Table 3. Table 3LDA way subject finding results.ThemeWords and word countTotal countTopic 1: Performance & HardwareBattery life (100) Speed (72) Processor (60)Stability (50) Memory (45) Display (40)367Topic 2: Design & ComfortDesign (88) Comfort (70) Lightweight (60) Strap(50) Fashion (46) Material (40)354Topic 3: Health & Fitness TrackingHeart rate (95) Sleep monitor (82) Step count (70)Exercise tracking (65) Blood oxygen (58)Stress monitor (50)420Topic 4: Connectivity & FeaturesNotification (80) GPS (68) Bluetooth (58)Compatibility (55) App support (46) Payment (40)347Topic 5: Durability & ValueWaterproof (85) Price (72) Durability (62)Warranty (55) Reliability (50) Cost-effective (45)369 Table 3LDA way subject finding results.ThemeWords and word countTotal countTopic 1: Performance & HardwareBattery life (100) Speed (72) Processor (60)Stability (50) Memory (45) Display (40)367Topic 2: Design & ComfortDesign (88) Comfort (70) Lightweight (60) Strap(50) Fashion (46) Material (40)354Topic 3: Health & Fitness TrackingHeart rate (95) Sleep monitor (82) Step count (70)Exercise tracking (65) Blood oxygen (58)Stress monitor (50)420Topic 4: Connectivity & FeaturesNotification (80) GPS (68) Bluetooth (58)Compatibility (55) App support (46) Payment (40)347Topic 5: Durability & ValueWaterproof (85) Price (72) Durability (62)Warranty (55) Reliability (50) Cost-effective (45)369 LDA way subject finding results. Battery life (100) Speed (72) Processor (60) Stability (50) Memory (45) Display (40) Design (88) Comfort (70) Lightweight (60) Strap (50) Fashion (46) Material (40) Heart rate (95) Sleep monitor (82) Step count (70) Exercise tracking (65) Blood oxygen (58) Stress monitor (50) Notification (80) GPS (68) Bluetooth (58) Compatibility (55) App support (46) Payment (40) Waterproof (85) Price (72) Durability (62) Warranty (55) Reliability (50) Cost-effective (45) As shown in Table 3, the LDA model identifies five major themes that capture distinct layers of smartwatch user experience. Topic 1 (Performance & Hardware) centers on intrinsic device attributes such as battery life, processing capability, and display performance, reflecting users’ concern for operational stability and core technical quality. Topic 2 (Design & Comfort) emphasizes external form, material selection, and wearing comfort, highlighting experiential and aesthetic considerations that extend beyond pure functionality. Topic 3 (Health & Fitness Tracking) focuses on physiological monitoring and activity-related functions, including heart rate, sleep, and exercise tracking, underscoring the role of smartwatches as personal health management tools. Topic 4 (Connectivity & Features) addresses system-level interaction and ecosystem integration, such as notifications, connectivity stability, app compatibility, and payment functions, reflecting users’ expectations for seamless integration with digital services and smart environments. Topic 5 (Durability & Value) captures long-term usage considerations, including waterproof performance, reliability, warranty, and price sensitivity, indicating that cost-effectiveness and product longevity remain important factors in user decision-making. Although certain themes (e.g., Performance & Hardware and Connectivity & Features) are functionally related, they represent distinct perceptual and evaluative dimensions: the former reflects intrinsic device performance, whereas the latter emphasizes interaction quality and ecosystem compatibility. Maintaining these themes as separate categories enables finer-grained analysis and supports the construction of a structured, interpretable evaluation hierarchy for subsequent FAHP-based weighting and decision analysis. Collectively, the five extracted themes provide a coherent semantic foundation for modeling user experience and developing a scientifically grounded smartwatch evaluation indicator system. Establishing evaluation criteria for smartwatch products This study draws on international standards and industry norms in the field of smart wearable devices, combining LDA with in-depth analysis of user review data. Through systematic literature reviews, user data analysis, and expert questionnaire surveys, it comprehensively identifies and analyzes the multi-dimensional factors influencing smartwatch design and user experience13,39,40. Through scientific screening and synthesis, five core dimensions for evaluating smartwatch design were ultimately identified, encompassing 30 specific evaluation indicators that can comprehensively reflect the functional attributes of smartwatches and user demand characteristics. Based on this indicator system, the study further constructed a hierarchical structure model: the P layer represents the overall evaluation objective, the Bn layer represents the five core criteria layers, and the B1n layer represents the 30 specific evaluation indicators. The hierarchical structure is shown in Fig. 7. Fig. 7Smartwatch design evaluation index system. Smartwatch design evaluation index system. At the criteria level, this study defined five core dimensions: Performance & Hardware, Design & Comfort, Health & Fitness Tracking, Connectivity & Features, and Durability & Value. These dimensions collectively form the foundational framework of the smartwatch design evaluation system. Subsequently, we constructed a fuzzy complementary judgment matrix according to the established criteria. See Table 4 for the specific data. This matrix quantifies the relative importance and interrelationships between different dimensions through pairwise comparison methods, thereby effectively characterizing their intrinsic coupling mechanisms. This process not only ensures the scientific rigor and logical consistency of the evaluation indicator system but also lays a solid quantitative foundation for subsequent weight calculation and optimization analysis. Table 4Initial criterion layer fuzzy reciprocal matrix.APerformance & Hardware B1Design & Comfort B2Health & Fitness Tracking B3Connectivity & Features B4Durability & Value B5Performance & Hardware B10.5RRRRDesign & Comfort B21/R0.5RRRHealth & Fitness Tracking B31/R1/R0.5RRConnectivity & Features B41/R1/R1/R0.5RDurability & Value B51/R1/R1/R1/R0.5 Table 4Initial criterion layer fuzzy reciprocal matrix.APerformance & Hardware B1Design & Comfort B2Health & Fitness Tracking B3Connectivity & Features B4Durability & Value B5Performance & Hardware B10.5RRRRDesign & Comfort B21/R0.5RRRHealth & Fitness Tracking B31/R1/R0.5RRConnectivity & Features B41/R1/R1/R0.5RDurability & Value B51/R1/R1/R1/R0.5 Initial criterion layer fuzzy reciprocal matrix. In the second-level indicator layer, five second-level judgment matrices were established according to the aforementioned method. Due to their consistent structural arrangement, they are not shown individually here. When constructing a fuzzy complementary judgment matrix, each value R implies the weight trade-off of element m relative to element n under the overall criterion. What is particularly critical is that the values in the upper right corner of the matrix are directly filled in by the evaluator based on his professional attainments or insights into data analysis. They indicate the extent to which one element affects another element. The corresponding lower left part is not manually entered, but is calculated using algorithms such as fuzzy complementary relationships to ensure the consistency and logical rigor of the matrix. Formulation and deployment of the fuzzy complementary matrix questionnaire To ensure methodological rigor and robustness of the evaluation results, this study followed three principles in expert sample selection: multi-role coverage, professional experience stratification, and sample stability assessment. A total of 35 cross-disciplinary participants were invited to form the evaluation panel, including 8 smartwatch device experts, 6 health monitoring engineers, 9 industrial designers, 5 product managers, and 7 representatives of long-term smartwatch users. The adopted panel size is consistent with commonly recommended ranges in fuzzy AHP, fuzzy comprehensive evaluation (FCE), and MCDM studies (typically 20–40 experts), and was further supported by a convergence-oriented stability assessment. It should be noted that among the 35 panel members, seven participants were categorised as long-term smartwatch users. These individuals were not engaged as technical experts but rather as experience-oriented contributors, aiming to capture sustained user perceptions—perceptions that are often underestimated in purely expert-led evaluations. All user representatives met predefined inclusion criteria, including continuous smartwatch usage exceeding two years and experience with at least two distinct brands or models. In the judgment aggregation process, no explicit a priori weighting was assigned based on expert type. Instead, all judgments were integrated using the FAHP framework with PSO-based consistency correction, which reduces the influence of inconsistent or extreme inputs during matrix optimization. Additional inter-group consistency checks confirmed that the inclusion of long-term users did not introduce systematic bias into the weight estimation results. To examine the effect of expert sample size on weight estimation stability, a simulation-based convergence analysis was conducted by progressively increasing the number of experts and tracking the resulting variations in indicator membership degrees and weights. The analysis indicates that when the sample size approaches approximately 30 experts, the fluctuations in key indicator weights decrease markedly and enter a plateau region, with average relative changes falling below 2%. Further increases in the number of experts produce only marginal adjustments to the overall weighting structure. Based on this observed stabilization trend, a 35-member expert panel was adopted to balance diversity of perspectives with robustness of the aggregated weights. To further assess the rationality of the expert composition, inter-group scoring consistency was examined using variance comparison and Kendall’s W coefficient. The results demonstrate a high level of agreement across different expert groups, with no evidence of systematic structural bias. Overall, the interdisciplinary panel of 35 experts provides sufficient stability, representativeness, and reliability for constructing the fuzzy complementary judgment matrix, while effectively mitigating the influence of individual subjectivity on the final evaluation outcomes. The questionnaire comprised two sections. The first collected respondents’ background information to ensure data traceability and reliability, while the second focused on core smartwatch evaluation criteria. Participants were required to conduct pairwise comparisons and assign scores based on their professional knowledge and usage experience. To reduce respondent burden and improve data processing efficiency, experts were asked to complete only the upper triangular part of the fuzzy complementary judgment matrix, with the lower triangular elements automatically generated according to the complementary reciprocity rules. Expert sample convergence and stability validation To empirically justify the adequacy of the expert panel size, a convergence analysis of the FAHP-derived weight vector was conducted with respect to the number of participating experts. Let w(35) = (w1(35),…,wm(35)) denote the reference weight vector obtained by aggregating judgments from the full panel of 35 experts, where m is the number of evaluation criteria. For each sample size n ∈ {10,15,20,25,30,35}, n experts were randomly selected without replacement, and the corresponding weight vector w(n) was recomputed using the identical FAHP aggregation procedure and PSO-based consistency correction settings. To account for sampling variability, this resampling process was repeated 500 times for each n. Weight stability was quantified using the mean absolute percentage change (MAPC), defined as:17\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{MAPC}}\left( n \right)=\frac{1}{m}\mathop \sum \limits_{{j=1}}^{m} \left| {\frac{{w_{j}^{{\left( n \right)}} - w_{j}^{{\left( {35} \right)}}}}{{w_{j}^{{\left( {35} \right)}}}}} \right| \times 100\%$$\end{document} This metric captures the average relative deviation of criterion weights estimated from a subsample of n experts with respect to the full-sample reference, with smaller MAPC values indicating greater numerical stability of the weight estimation. Complementarily, rank-order robustness was assessed using Kendall’s τ coefficient to evaluate the consistency between the criterion importance rankings derived from w(n) and w(35), where higher values of τ reflect stronger agreement in relative ordering. As summarized in Table 5, both the magnitude of weight deviations and ranking instability decrease markedly as the number of experts increases and converge toward a stable plateau once n ≥ 30. Beyond this threshold, further increases in sample size yield only marginal reductions in numerical deviation and negligible changes in rank consistency. These results indicate that the FAHP-derived weights exhibit satisfactory convergence and robustness when the expert panel size approaches 30, thereby providing methodological support for adopting a 35-member expert panel in this study. For n = 35, MAPC values are trivially zero by definition, as the full-sample weights are used as the reference; therefore, stability assessment focuses on the plateau behavior observed for n ≥ 30. Table 5MAPC and Kendall’s τ across n.n (experts)MAPC mean (%)MAPC median (%)MAPC IQR (%)MAPC 95th (%)Kendall τ meanτ medianτ IQRτ 5th107.6597.6161.4409.4850.9270.9260.0230.899155.5985.5891.0836.9650.9480.9470.0150.926204.1484.1710.7705.0890.9610.9630.0140.940253.0863.0830.6193.8250.9700.9720.0140.954301.9591.9600.3852.4630.9810.9820.0090.96835~ 0.000~ 0.000~ 0.000~ 0.0001.0001.0000.0001.000 Table 5MAPC and Kendall’s τ across n.n (experts)MAPC mean (%)MAPC median (%)MAPC IQR (%)MAPC 95th (%)Kendall τ meanτ medianτ IQRτ 5th107.6597.6161.4409.4850.9270.9260.0230.899155.5985.5891.0836.9650.9480.9470.0150.926204.1484.1710.7705.0890.9610.9630.0140.940253.0863.0830.6193.8250.9700.9720.0140.954301.9591.9600.3852.4630.9810.9820.0090.96835~ 0.000~ 0.000~ 0.000~ 0.0001.0001.0000.0001.000 MAPC and Kendall’s τ across n. Applying PSO for nonlinear optimization of evaluation weights In the process of applying PSO for the consistency correction of fuzzy complementary judgment matrices, this study formulates the consistency improvement task as a nonlinear optimization problem defined over the 10 independent elements of the matrix. For a 5 × 5 fuzzy complementary matrix, the upper-triangular non-diagonal entries contain n(n-1)/2 = 10 free variables. Accordingly, the decision space of PSO is 10-dimensional. The lower-triangular entries are automatically generated by the reciprocal–complement relation rji = 1-rij, and the diagonal entries are fixed at 0.5, ensuring that the matrix structure remains valid throughout the optimization process. Initialization and fitness evaluation: Each particle is represented as a 10-dimensional vector corresponding to the ten optimizable elements of the judgment matrix. Particle positions are initialized uniformly within the interval [0.1, 0.9], and velocities are drawn randomly from the range [-vmax, vmax]. After initialization, the fitness of each particle is computed using the consistency index CI (Y, w), which quantifies the level of inconsistency in the corresponding reconstructed matrix. Iterative search and update: During optimization, particles iteratively update their positions and velocities according to the standard PSO update rules. If a particle’s current solution outperforms its historical best, the personal best is updated accordingly. Likewise, the global best is updated when a particle surpasses the best performance observed in the entire swarm. After each update, boundary repair and reciprocal-complement enforcement are applied to maintain structural validity of the judgment matrix. Stopping criterion and weight derivation: The search terminates when the maximum number of iterations (500) is reached or when convergence criteria are satisfied. The consistency-corrected fuzzy complementary matrix obtained from PSO is then used as input to FAHP. The final weight vector of the evaluation indicators is derived using the geometric mean method, thereby enabling accurate quantification of the hierarchical criteria. The corresponding visualization results are shown in Fig. 8. Fig. 8Convergence diagram of PSO iteration times. Convergence diagram of PSO iteration times. To ensure transparency in the particle swarm optimization process and reproducibility of experimental results, this study meticulously defined the key hyperparameters of the PSO algorithm. These parameter selections were based on common practices within the algorithmic field and validated through preliminary experiments to achieve an optimal balance between convergence speed and solution quality. Table 6 lists the PSO hyperparameter settings employed in this study for consistency correction of the FAHP fuzzy judgment matrix. Table 6PSO algorithm hyperparameter settings.Parameter categoryParameter value / descriptionExplanation / rationaleDecision Variable EncodingUpper triangular elements of the fuzzy judgmentmatrix (excluding the diagonal)For a 5th-order matrix, the number of optimizedvariables is 10Variable Dimension (D)10Determined by the matrix order (n×(n-1)/2)Swarm Size (N)100Balances search efficiency and computational costNumber of Iterations500Verified to be sufficient for algorithm convergenceLearning Factors (c₁, c₂)c1 = 2.0, c2 = 2.0Standard settings to balance individual andsocial learningInertia Weight (ω)Linearly decreasing from 0.9 to 0.4Time-varying: promotes global exploration earlyand local exploitation laterVelocity ClampingMaximum velocity set to 20% of the variablerange (0.2)Prevents particles from overshooting thesearch spaceInitializationParticle positions generated randomly anduniformly within [0.1, 0.9]Ensures diversity in the initial populationNumber of Runs30 independent runsMitigates the impact of randomness; the best-performing result is selectedRandom SeedFixed at 42 (for reproducibility testing)Ensures the experimental results are preciselyreproducible Table 6PSO algorithm hyperparameter settings.Parameter categoryParameter value / descriptionExplanation / rationaleDecision Variable EncodingUpper triangular elements of the fuzzy judgmentmatrix (excluding the diagonal)For a 5th-order matrix, the number of optimizedvariables is 10Variable Dimension (D)10Determined by the matrix order (n×(n-1)/2)Swarm Size (N)100Balances search efficiency and computational costNumber of Iterations500Verified to be sufficient for algorithm convergenceLearning Factors (c₁, c₂)c1 = 2.0, c2 = 2.0Standard settings to balance individual andsocial learningInertia Weight (ω)Linearly decreasing from 0.9 to 0.4Time-varying: promotes global exploration earlyand local exploitation laterVelocity ClampingMaximum velocity set to 20% of the variablerange (0.2)Prevents particles from overshooting thesearch spaceInitializationParticle positions generated randomly anduniformly within [0.1, 0.9]Ensures diversity in the initial populationNumber of Runs30 independent runsMitigates the impact of randomness; the best-performing result is selectedRandom SeedFixed at 42 (for reproducibility testing)Ensures the experimental results are preciselyreproducible PSO algorithm hyperparameter settings. Upper triangular elements of the fuzzy judgment matrix (excluding the diagonal) For a 5th-order matrix, the number of optimized variables is 10 Standard settings to balance individual and social learning Time-varying: promotes global exploration early and local exploitation later Maximum velocity set to 20% of the variable range (0.2) Prevents particles from overshooting the search space Particle positions generated randomly and uniformly within [0.1, 0.9] Mitigates the impact of randomness; the best -performing result is selected Ensures the experimental results are precisely reproducible Specifically, the decision variables for the optimization problem are defined as the upper triangular elements (excluding the diagonal) of the fuzzy complementary judgment matrix. For a 5th-order judgment matrix, the variable dimension D is 10. The particle swarm size is set to 100, a commonly used scale for solving such medium-dimensional problems, sufficient to maintain population diversity. The learning factors c1 and c2 both adopt the standard value of 2.0 to balance the particles’ reliance on their own experience and the group’s social information. The inertia weight ω employs a linear decay strategy, gradually decreasing from 0.9 to 0.4. This allows the algorithm to focus on global exploration during the early iterations and shift toward local refinement in the later stages. Additionally, velocity clamping (capping maximum velocity at 20% of the search space range) was implemented to control particle movement stride, preventing particles from skipping over optimal regions. In the experimental implementation, the position of each particle was randomly initialized within the variable domain [0.1, 0.9]. To mitigate the randomness inherent in the optimization algorithm, we conducted 30 independent runs and selected the solution with the optimal fitness value (i.e., the lowest consistency index CI) as the final result. To avoid ambiguity, it is clarified that the objective function minimized by PSO is the consistency index CI (Y, w) defined in section “Consistency analysis and PSO-based correction for fuzzy judgment matrices”, where the reconstructed consistent matrix Y\(w) is computed from the weight vector obtained via the geometric mean method. Because PSO optimizes the ten free upper-triangular elements rather than the weights directly, no normalization constraint such as ∑wi = 1 is imposed within the optimization loop; the weight vector produced by the geometric mean method naturally satisfies this condition. In addition to the maximum iteration limit of 500, a secondary convergence criterion was applied: optimization stops early if the improvement in the global best fitness falls below 10− 6 for 50 consecutive iterations. To ensure robustness against stochastic effects, the optimization procedure was repeated for 30 independent runs. Statistical descriptors (mean, standard deviation, and best-run performance) were recorded to assess the stability of the algorithm before selecting the best solution for the final FAHP weight computation. The importance values of the parts in each level were worked out, as shown in Table 7. Table 7Calculation results for the weight values of each layer factor.Goal levelStandard levelPriority valuesMeasure levelPriority valuesOverall prioritySmartwatch DesignEvaluation IndexSystem PPerformance &Hardware0.220B11 Battery life0.1680.0370B12 Speed0.1540.0339B13 Processor0.1470.0323B14 Stability0.1420.0312B15 Memory0.1300.0286B16 Display0.1190.0262Design &Comfort0.190B21 Design0.1680.0319B22 Comfort0.1560.0296B23 Lightweight0.1510.0287B24 Strap0.1400.0266B25 Fashion0.1300.0247B26 Material0.1260.0239Health & FitnessTracking0.240B31 Heart rate0.1750.0420B32 Sleep monitor0.1670.0401B33 Step count0.1600.0384B34 Exercise track0.1550.0372B35 Blood oxygen0.1700.0408B36 Stress monitor0.1730.0415Connectivity &Features0.165B41 Notification0.1740.0287B42 GPS0.1650.0272B43 Bluetooth0.1600.0264B44 Compatibility0.1550.0256B45 App support0.1730.0285B46 Payment0.1730.0285Durability & Value0.185B51 Waterproof0.1760.0326B52 Price0.1680.0311B53 Durability0.1640.0303B54 Warranty0.1580.0292B55 Reliability0.1670.0309B56 Cost-effective0.1670.0309 Table 7Calculation results for the weight values of each layer factor.Goal levelStandard levelPriority valuesMeasure levelPriority valuesOverall prioritySmartwatch DesignEvaluation IndexSystem PPerformance &Hardware0.220B11 Battery life0.1680.0370B12 Speed0.1540.0339B13 Processor0.1470.0323B14 Stability0.1420.0312B15 Memory0.1300.0286B16 Display0.1190.0262Design &Comfort0.190B21 Design0.1680.0319B22 Comfort0.1560.0296B23 Lightweight0.1510.0287B24 Strap0.1400.0266B25 Fashion0.1300.0247B26 Material0.1260.0239Health & FitnessTracking0.240B31 Heart rate0.1750.0420B32 Sleep monitor0.1670.0401B33 Step count0.1600.0384B34 Exercise track0.1550.0372B35 Blood oxygen0.1700.0408B36 Stress monitor0.1730.0415Connectivity &Features0.165B41 Notification0.1740.0287B42 GPS0.1650.0272B43 Bluetooth0.1600.0264B44 Compatibility0.1550.0256B45 App support0.1730.0285B46 Payment0.1730.0285Durability & Value0.185B51 Waterproof0.1760.0326B52 Price0.1680.0311B53 Durability0.1640.0303B54 Warranty0.1580.0292B55 Reliability0.1670.0309B56 Cost-effective0.1670.0309 Calculation results for the weight values of each layer factor. Smartwatch Design Evaluation Index System P Performance & Hardware Design & Comfort Health & Fitness Tracking Connectivity & Features When determining the weight distribution of the smartwatch design evaluation index system, this study used user evaluation data obtained from LDA topic modeling and further refined the five core dimensions (Performance & Hardware, Design & Comfort, Health & Fitness Tracking, Connectivity & Features, Durability & Value) into 30 specific evaluation indicators. Table 7 shows the weight distribution results for each dimension and its secondary indicators. In the weight calculation process, the relative weights of the five criterion layers are first obtained based on the normalized results of the frequency of each theme word. Subsequently, the six high-frequency keywords within each theme are normalized to obtain the indicator layer weights. Finally, the comprehensive weight is calculated as “criterion layer weight × indicator layer weight” to reflect the relative importance of each indicator in the overall evaluation system. From the results, Health & Fitness Tracking (weight 0.240) has the highest proportion among the five core dimensions, indicating that users are most concerned about the health monitoring functions of smartwatches; followed by Performance & Hardware (both with a weight of 0.220), highlighting the important role of hardware performance and durability in user experience. In the indicator layer, metrics such as “Heart rate,” “Battery life,” and “Waterproof” rank highly in terms of composite weight, indicating their central role in user evaluations and demands. Sensitivity of the FAHP–PSO weighting process to expert input To assess the robustness of the FAHP-PSO weighting process against potential variations in expert judgment, we conducted a series of sensitivity analyses. First, a leave-one-expert-out (LOEO) analysis was performed. For each evaluation indicator, one expert was removed in turn, and the fuzzy complementary judgment matrix was reconstructed using the remaining experts’ data. The entire FAHP-PSO pipeline was then rerun to generate a new weight vector. Across all LOEO trials, the maximum deviation observed for any indicator’s weight was below 4.3%, and the relative ranking of all indicators remained unchanged. This demonstrates that the weighting process is not overly dependent on the judgment of any single expert. The results of this analysis are summarized in Fig. 9. Fig. 9Weighted bias bar chart. Weighted bias bar chart. As illustrated in Fig. 9, the maximum percentage change for each indicator’s weight is well below the 4.3% threshold (dashed line). The most sensitive indicator shows a deviation of only 3.8%, providing visual confirmation that the results are not unduly influenced by any individual expert, thereby demonstrating the robustness of the FAHP-PSO process. Second, to evaluate robustness under minor judgment inconsistencies, random noise drawn from a uniform distribution U (-0.05, + 0.05) was added to each expert’s original pairwise comparison entry. The perturbed values were then re-normalized to preserve the fuzzy complementary constraints. This experiment was repeated 50 times. The resulting weight distributions showed low dispersion, with a mean standard deviation of 0.018, indicating strong resilience to small perturbations. The distribution of weights for two core indicators across these trials is shown in Fig. 10. Fig. 10Weighted distribution box plot. Weighted distribution box plot. Figure 10 presents the distribution of weights for two core indicators, ‘Battery life (B11)’ and ‘Heart rate monitoring (B31)’, across the 50 noise-perturbation trials. The compact interquartile ranges (the boxes) and the stability of the median values (the central lines) indicate low variability. The presence of longer whiskers and a single outlier for B10 suggests that while the weight for this indicator can be more sensitive to specific perturbations, the overall impact remains within acceptable limits, as corroborated by the LOEO analysis in Fig. 9. Robustness of LDA topic modeling The robustness of the LDA topic modeling component was rigorously evaluated against variations in its key parameters and preprocessing choices. First, the sensitivity to the number of topics (K) was assessed. LDA models were trained with different topic numbers K ∈ {8, 10, 12, 14}, and their topic coherence scores (Cv metric) were computed. As shown in Fig. 11, the coherence curve exhibits a clear maximum at K = 10, providing quantitative justification for the model configuration adopted in this study. Fig. 11Parameter sensitivity of LDA topic models. Parameter sensitivity of LDA topic models. The distinct peak at K = 10 in Fig. 11 indicates that the topics extracted with this parameter are the most semantically interpretable and coherent. The subsequent decline in coherence for K > 10 suggests the emergence of redundant or overlapping topics. This graph provides data-driven validation for our parameter choice. Furthermore, the core semantic themes—such as battery life, health-monitoring reliability, UI/UX experience, and material quality—were consistently identified across all tested K values, with an average Jaccard similarity of 0.76 for the top-10 keywords per topic, indicating semantic stability. Second, the impact of text preprocessing choices was tested using three configurations: (1) a baseline stop-word list; (2) an expanded stop-word list including review-specific filler terms; and (3) synonym merging (e.g., mapping “battery duration” to “battery life”). Across these configurations, topic coherence varied within a narrow range (0.486–0.512), and the dominant topics remained semantically stable. This confirms that the LDA-derived indicators are not overly sensitive to preprocessing variations. Finally, the stability of the LDA model (with K = 10) was tested by training it with 10 different random seeds. The cosine similarity between the topic-word distributions from different runs was computed pairwise, yielding a mean similarity of 0.84. This indicates high stability and low sensitivity to the algorithm’s inherent randomness. Across all three robustness dimensions examined—expert perturbation, LDA parameter sensitivity, and algorithmic randomness—the variations in the final indicator weights and product evaluation scores were minor. Crucially, no changes occurred in the ranking of the smartwatches. These results collectively demonstrate that the proposed LDA-FAHP-PSO evaluation system is stable and reliable, even under reasonable variations in expert inputs or topic modeling configurations. Tests and findings Based on the analysis of weight distributions across different smartwatch products, this study identified four representative smartwatch models that are currently prevalent in the market. These selected models were used as case studies to validate the effectiveness of the proposed evaluation framework in supporting product performance assessment and improvement. Smartwatch make and check case Within the product portfolio of smartwatches, they can be broadly categorized into health monitoring, fitness and sports, all-scenario comprehensive, and specialized models for specific demographics based on their functional characteristics. Given that the evaluation framework of this study primarily focuses on dimensions such as performance, health tracking, interactive functions, and user experience, and considering that current market demand places the highest emphasis on health and fitness scenarios, the research subject selected is a fitness-oriented smartwatch that is representative in terms of functional integration, market applicability, and technological maturity. To ensure the scientific rigor and representativeness of the experimental subjects, this study adheres to the following selection criteria: Compliant with mainstream certification standards: All sample products have passed CE or 3 C certification, ensuring compliance with safety and reliability standards.Covering mainstream price ranges: Select mid-to-high-end models priced between 1,000 and 1,500 yuan to avoid evaluation biases caused by overly low or high price points;Complete functional features: Equipped with core health monitoring functions such as heart rate, blood oxygen, sleep, and activity tracking, while also supporting basic connectivity features like Bluetooth, GPS, and mobile payments;High market recognition: Leading in both sales volume and user reviews, representing the mainstream development direction of fitness-oriented smartwatches. Compliant with mainstream certification standards: All sample products have passed CE or 3 C certification, ensuring compliance with safety and reliability standards. Compliant with mainstream certification standards: All sample products have passed CE or 3 C certification, ensuring compliance with safety and reliability standards. Covering mainstream price ranges: Select mid-to-high-end models priced between 1,000 and 1,500 yuan to avoid evaluation biases caused by overly low or high price points; Covering mainstream price ranges: Select mid-to-high-end models priced between 1,000 and 1,500 yuan to avoid evaluation biases caused by overly low or high price points; Complete functional features: Equipped with core health monitoring functions such as heart rate, blood oxygen, sleep, and activity tracking, while also supporting basic connectivity features like Bluetooth, GPS, and mobile payments; Complete functional features: Equipped with core health monitoring functions such as heart rate, blood oxygen, sleep, and activity tracking, while also supporting basic connectivity features like Bluetooth, GPS, and mobile payments; High market recognition: Leading in both sales volume and user reviews, representing the mainstream development direction of fitness-oriented smartwatches. High market recognition: Leading in both sales volume and user reviews, representing the mainstream development direction of fitness-oriented smartwatches. Sample design for smartwatches This study selected four mainstream sports and health smartwatches on the market, weighted and scored them, and named them W1, W2, W3, and W4 (Fig. 12). Fig. 12Smartwatch case selection. Smartwatch case selection. The W1 is the Huawei WATCH GT 5, offering up to 14 days of usage in typical scenarios. Its health monitoring features cover comprehensive data, including heart rate, blood oxygen, stress, and sleep, combined with high-precision GPS to track exercise trajectories and assess workout effectiveness accurately. The 1.43-inch AMOLED display offers high brightness, while the lightweight design (approximately 36 g) and breathable strap material ensure comfortable wearability. The WATCH GT 5 also supports Bluetooth calling, NFC payments, and local music playback, and features a 5ATM water resistance rating and durable design for enhanced longevity. The Xiaomi Watch S4 Sport is the W2, which is known for its high cost-effectiveness and practical sports features. It is equipped with a mainstream chip and a large-capacity battery, offering up to 12 days of battery life under regular use and supporting fast magnetic charging. Health features include sleep, blood oxygen, and stress monitoring, meeting daily health management needs. For sports, it has built-in GPS and multiple sports modes, providing users with exercise feedback. 5ATM water resistance and a reinforced body structure enhance durability. The W3 is designed for the Apple Watch Series 10, featuring the latest generation S-series chip. It offers a battery life of approximately 18–24 h and supports fast charging and wireless charging. Health features include temperature sensing, blood oxygen monitoring, fall detection, and heart rate alerts. For fitness, it supports GPS and multiple training modes, and integrates deeply with the Apple Fitness + and fitness app ecosystem. The design features an always-on display, a slim and stylish body, and a variety of interchangeable straps. It is water-resistant up to 50 m, ensuring safety and reliability in daily and athletic settings. The W4 is the Samsung Galaxy Watch 7, which emphasizes medical-grade health monitoring and smart ecosystem integration. It is equipped with an Exynos chip and the latest sensors, supporting ECG, blood pressure monitoring, blood oxygen tracking, and professional sleep analysis, offering comprehensive and in-depth health features. It has a battery life of up to 2–3 days and supports wireless charging. For fitness, it includes built-in GPS and multiple workout modes. The design features a metal frame and a rotating crown, offering both premium build quality and excellent usability, with highly customizable watch faces. It also boasts 5ATM water resistance and MIL-STD-810G military-grade certification, ensuring reliability in challenging environments. To ensure the scientific and objective nature of the evaluation results, this study selected four sports and health-oriented smartwatches with market prices ranging from 1,500 to 3000yuan, high market share, and representative characteristics as experimental samples. These products encompass the mainstream technical approaches and design features of various brands, providing a comprehensive reflection of the current development level of sports and health-oriented smartwatches. Rating the watch design check marks Looking at the stats on evaluator scores, we’ve determined the level to which each smartwatch case evaluation metric corresponds to the different quality grades. You can see the specific breakdown for case W1 in Table 8. As for W2 through W4, you’ll find all the details in the Appendix. Table 8Results of evaluation of W1.StandardlevelMeasure levelCheck resultsDesign check and levels(V)Very highHighMediumLowVery lowPerformance &HardwareB11 Battery life913940B12 Speed7111160B13 Processor6101360B14 Stability7121150B15 Memory6111260B16 Display8121050Design &ComfortB21 Design7121150B22 Comfort6111620B23 Lightweight7121150B24 Strap6101360B25 Fashion5101370B26 Material7111250Health &Fitness TrackingB31 Heart rate8121050B32 Sleep monitor7111250B33 Step count8121050B34 Exercise track7111250B35 Blood oxygen6111260B36 Stress monitor6101360Connectivity &FeaturesB41 Notification7111250B42 GPS7111250B43 Bluetooth8121050B44 Compatibility6111260B45 App support5101820B46 Payment591470Durability &ValueB51 Waterproof7121150B52 Price8121050B53 Durability7111250B54 Warranty6111530B55 Reliability6101360B56 Cost-effective7121150 Table 8Results of evaluation of W1.StandardlevelMeasure levelCheck resultsDesign check and levels(V)Very highHighMediumLowVery lowPerformance &HardwareB11 Battery life913940B12 Speed7111160B13 Processor6101360B14 Stability7121150B15 Memory6111260B16 Display8121050Design &ComfortB21 Design7121150B22 Comfort6111620B23 Lightweight7121150B24 Strap6101360B25 Fashion5101370B26 Material7111250Health &Fitness TrackingB31 Heart rate8121050B32 Sleep monitor7111250B33 Step count8121050B34 Exercise track7111250B35 Blood oxygen6111260B36 Stress monitor6101360Connectivity &FeaturesB41 Notification7111250B42 GPS7111250B43 Bluetooth8121050B44 Compatibility6111260B45 App support5101820B46 Payment591470Durability &ValueB51 Waterproof7121150B52 Price8121050B53 Durability7111250B54 Warranty6111530B55 Reliability6101360B56 Cost-effective7121150 Results of evaluation of W1. Performance & Hardware Design & Comfort Health & Fitness Tracking Connectivity & Features Durability & Value Based on the scores given by the evaluators, a fuzzy membership evaluation matrix can be further constructed for each design scheme. Given that the calculation steps are consistent, to avoid repetition, only case W1 is used as an example to show its fuzzy membership evaluation matrix. W1’s fuzzy membership assessment matrix: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${U_{{{_{{W1}}}_{ - 1}}}}=\left\{ {\begin{array}{*{20}{l}} {0.257}&{0.371}&{0.257}&{0.114}&{0.000} \\ {0.200}&{0.314}&{0.314}&{0.171}&{0.000} \\ {0.171}&{0.286}&{0.371}&{0.171}&{0.000} \\ {0.200}&{0.343}&{0.314}&{0.143}&{0.000} \\ {0.171}&{0.314}&{\begin{array}{*{20}{r}} {0.343} \end{array}}&{\begin{array}{*{20}{r}} {0.171} \end{array}}&{0.000} \\ {0.229}&{0.343}&{0.286}&{0.143}&{0.000} \end{array}} \right.$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${U_{{{_{{W1}}}_{ - 2}}}}=\left\{ {\begin{array}{*{20}{l}} {0.200}&{0.343}&{0.314}&{0.143}&{0.000} \\ {0.171}&{0.314}&{0.457}&{0.057}&{0.000} \\ {0.200}&{0.343}&{0.314}&{0.143}&{0.000} \\ {0.171}&{0.286}&{0.371}&{0.171}&{0.000} \\ {0.143}&{0.286}&{\begin{array}{*{20}{r}} {0.371} \end{array}}&{\begin{array}{*{20}{r}} {0.200} \end{array}}&{0.000} \\ {0.200}&{0.314}&{0.343}&{0.143}&{0.000} \end{array}} \right.$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${U_{{{_{{W1}}}_{ - 3}}}}=\left\{ {\begin{array}{*{20}{l}} {0.229}&{0.343}&{0.286}&{0.143}&{0.000} \\ {0.200}&{0.314}&{0.343}&{0.143}&{0.000} \\ {0.229}&{0.343}&{0.286}&{0.143}&{0.000} \\ {0.200}&{0.314}&{0.343}&{0.143}&{0.000} \\ {0.171}&{0.314}&{\begin{array}{*{20}{r}} {0.343} \end{array}}&{\begin{array}{*{20}{r}} {0.171} \end{array}}&{0.000} \\ {0.171}&{0.286}&{0.371}&{0.171}&{0.000} \end{array}} \right.$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${U_{{{_{{W1}}}_{ - 4}}}}=\left\{ {\begin{array}{*{20}{l}} {0.200}&{0.314}&{0.343}&{0.143}&{0.000} \\ {0.200}&{0.314}&{0.343}&{0.143}&{0.000} \\ {0.229}&{0.343}&{0.286}&{0.143}&{0.000} \\ {0.171}&{0.314}&{0.343}&{0.171}&{0.000} \\ {0.143}&{0.286}&{\begin{array}{*{20}{r}} {0.514} \end{array}}&{\begin{array}{*{20}{r}} {0.057} \end{array}}&{0.000} \\ {0.143}&{0.257}&{0.400}&{0.200}&{0.000} \end{array}} \right.$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${U_{{{_{{W1}}}_{ - 5}}}}=\left\{ {\begin{array}{*{20}{l}} {0.200}&{0.343}&{0.314}&{0.143}&{0.000} \\ {0.229}&{0.343}&{0.286}&{0.143}&{0.000} \\ {0.200}&{0.314}&{0.343}&{0.143}&{0.000} \\ {0.171}&{0.314}&{0.429}&{0.086}&{0.000} \\ {0.171}&{0.286}&{\begin{array}{*{20}{r}} {0.371} \end{array}}&{\begin{array}{*{20}{r}} {0.171} \end{array}}&{0.000} \\ {0.200}&{0.343}&{0.314}&{0.143}&{0.000} \end{array}} \right.$$\end{document} Fuzzy comprehensive evaluation Based on the indicator weights determined by the optimized FAHP model, this study performs fuzzy matrix operations on the evaluation data according to formula (18), thereby achieving an organic combination of indicator weights and evaluation results.18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q=U \times C'=({q_1},{q_2},{q_3},{q_4},{q_5})$$\end{document} If Q = 12, adjustment needs to be done so that 19\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$q_{\nu }^{\prime } = \frac{{q_{\nu } }}{{\mathop \sum \limits_{{\nu = I}}^{5} q_{\nu } }}$$\end{document} End-to-end computing example: case study W1 To clearly demonstrate the integrity of the fuzzy membership degree construction and aggregation path, this study uses the smartwatch case W1 (Huawei WATCH GT 5) as an example to step-by-step illustrate the entire process of calculating its comprehensive evaluation score. This process encompasses three core steps: constructing the fuzzy membership degree matrix, applying hierarchical aggregation operators, and calculating the final crisp value. Step 1: Constructing the fuzzy membership matrix Step 1: Constructing the fuzzy membership matrix Step 1: Constructing the fuzzy membership matrix First, based on the evaluation team’s scoring results (see Table 8), the qualitative evaluation for each assessment indicator is converted into a fuzzy membership vector. The specific method involves dividing the number of votes for each rating level (e.g., “Very High”) by the total valid votes (35 in this case). For example, for indicator B11 (Battery Life), with votes [14, 21, 14, 6, 0], its membership vector is calculated as follows: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${R_{B11}}=\left[ {\frac{9}{{35}},\frac{{13}}{{35}},\frac{9}{{35}},\frac{4}{{35}},\frac{0}{{35}}} \right]=\left[ {0.257,0.371,0.257,0.114,0.000} \right]$$\end{document} Based on this, the fuzzy relationship matrix RB1 for all six indicators under the “Performance & Hardware (B1)” guideline layer can be constructed as follows: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${R_{_{{B1}}}}=\left\{ {\begin{array}{*{20}{l}} {0.257}&{0.371}&{0.257}&{0.114}&{0.000} \\ {0.200}&{0.314}&{0.314}&{0.171}&{0.000} \\ {0.171}&{0.286}&{0.371}&{0.171}&{0.000} \\ {0.200}&{0.343}&{0.314}&{0.143}&{0.000} \\ {0.171}&{0.314}&{\begin{array}{*{20}{r}} {0.343} \end{array}}&{\begin{array}{*{20}{r}} {0.171} \end{array}}&{0.000} \\ {0.229}&{0.343}&{0.286}&{0.143}&{0.000} \end{array}} \right.$$\end{document} Each row of this matrix represents the degree to which an evaluation indicator falls within one of five evaluation levels. (2)Step 2: Hierarchical aggregation and operator application (2)Step 2: Hierarchical aggregation and operator application Step 2: Hierarchical aggregation and operator application Next, hierarchical aggregation is performed using the weighted average operator. This operator comprehensively reflects the influence of all factors while offering simplicity in computation and ease of interpretation, making it suitable for the comprehensive evaluation scenario in this study. First, using the weight vector WB1 = [0.168,0.154,0.147,0.142,0.130,0.119] determined by the FAHP-PSO model (see Table 6), perform fuzzy synthesis operations on criterion layer B1: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Q_{B1}}={W_{B1}} \cdot {R_{B1}}=\left[ {0.1770,0.2831,0.2693,0.1302,0.0000} \right]$$\end{document} Vector QB1 represents the membership degree of case W1 across five evaluation levels at the “Performance & Hardware” criterion layer. Repeating this process yields the membership vector QW1 for all five criterion layers relative to the overall objective layer (see original results). Finally, applying the weight vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W=\left[ {0.220,0.190,0.240,0.165,0.185} \right]$$\end{document} (criterion layer weights, see Table 6) for top-level aggregation produces the comprehensive membership vector: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_{W1}}=W \cdot {Q_{W1}}=\left[ {0.1829,0.3012,0.3228,0.1377,0.0000} \right]$$\end{document} (3)Step 3: Calculation of final uncertainty values and discussion of uncertainty (3)Step 3: Calculation of final uncertainty values and discussion of uncertainty Step 3: Calculation of final uncertainty values and discussion of uncertainty To obtain a comparable composite score, fuzzy membership degree vectors were converted into crisp values through defuzzification. This study employed a weighted averaging method, performing a dot product calculation between the composite membership degree vector SW1 and the score grade vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C={\left[ {90,80,70,60,50} \right]^T}$$\end{document} to derive the final score for case W1: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Z_{W1}}={S_{W1}} \cdot {C^T}=\left( {0.1829 \times 90} \right)+\left( {0.3012 \times 80} \right)+\left( {0.3228 \times 70} \right)+\left( {0.1377 \times 60} \right)+\left( {0.0000 \times 50} \right)=71.415$$\end{document} This score quantifies the overall performance of W1. It should be noted that this assessment is based on deterministic expert scoring and has not yet quantified the uncertainty arising from differences in evaluator reliability. Future research may incorporate fuzzy random models or confidence interval analysis to more comprehensively reflect the variability in assessment results. By following the steps above, the final numerical scores for cases W1 through W4 can be fully calculated. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Q_{W1}}=\left[ {\begin{array}{*{20}{c}} {{Q_{W1 - 1}}} \\ {{Q_{W1 - 2}}} \\ {{Q_{W1 - 3}}} \\ {{Q_{W1 - 4}}} \\ {{Q_{W1 - 5}}} \end{array}} \right]=\left[ {\begin{array}{*{20}{c}} {0.1770}&{0.2831}&{0.2693}&{0.1302}&{0.0000} \\ {0.1582}&{0.2752}&{0.3148}&{0.1225}&{0.0000} \\ {0.1998}&{0.3189}&{0.3287}&{0.1526}&{0.0000} \\ {0.1804}&{0.3039}&{0.3733}&{0.1423}&{0.0000} \\ {0.1954}&{0.3241}&{0.3417}&{0.1387}&{0.0000} \end{array}} \right]$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Q_{W2}}=\left[ {\begin{array}{*{20}{c}} {{Q_{W2 - 1}}} \\ {{Q_{W2 - 2}}} \\ {{Q_{W2 - 3}}} \\ {{Q_{W2 - 4}}} \\ {{Q_{W2 - 5}}} \end{array}} \right]=\left[ {\begin{array}{*{20}{c}} {0.2508}&{0.3233}&{0.2136}&{0.0725}&{0.0000} \\ {0.2450}&{0.3074}&{0.2395}&{0.0789}&{0.0000} \\ {0.2854}&{0.3615}&{0.2709}&{0.0865}&{0.0000} \\ {0.2663}&{0.3520}&{0.2916}&{0.0902}&{0.0000} \\ {0.2958}&{0.3668}&{0.2563}&{0.0812}&{0.0000} \end{array}} \right]$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Q_{W3}}=\left[ {\begin{array}{*{20}{c}} {{Q_{W3 - 1}}} \\ {{Q_{W3 - 2}}} \\ {{Q_{W3 - 3}}} \\ {{Q_{W3 - 4}}} \\ {{Q_{W3 - 5}}} \end{array}} \right]=\left[ {\begin{array}{*{20}{c}} {0.3928}&{0.3320}&{0.1254}&{0.0096}&{0.0000} \\ {0.3792}&{0.3317}&{0.1601}&{0.0000}&{0.0000} \\ {0.4432}&{0.3904}&{0.1662}&{0.0000}&{0.0000} \\ {0.4577}&{0.3760}&{0.1659}&{0.0000}&{0.0000} \\ {0.4140}&{0.3761}&{0.2094}&{0.0000}&{0.0000} \end{array}} \right]$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Q_{W4}}=\left[ {\begin{array}{*{20}{c}} {{Q_{W4 - 1}}} \\ {{Q_{W4 - 2}}} \\ {{Q_{W4 - 3}}} \\ {{Q_{W4 - 4}}} \\ {{Q_{W4 - 5}}} \end{array}} \right]=\left[ {\begin{array}{*{20}{c}} {0.3113}&{0.3071}&{0.2201}&{0.0215}&{0.0000} \\ {0.3035}&{0.3109}&{0.2353}&{0.0216}&{0.0000} \\ {0.3524}&{0.3526}&{0.2709}&{0.0245}&{0.0000} \\ {0.3521}&{0.3522}&{0.2718}&{0.0242}&{0.0000} \\ {0.3382}&{0.3525}&{0.2903}&{0.0193}&{0.0000} \end{array}} \right]$$\end{document} The results of the evaluation matrix for each design case are as follows. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_{{W_1}}}=C \times {Q_{{W_1}}}=\left[ {\begin{array}{*{20}{c}} {0.1829}&{0.3012}&{0.3228}&{0.1377}&{0.0000} \end{array}} \right]$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_{{W_2}}}=C \times {Q_{{W_2}}}=\left[ {\begin{array}{*{20}{c}} {0.2689}&{0.3422}&{0.2531}&{0.0816}&{0.0000} \end{array}} \right]$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_{{W_3}}}=C \times {Q_{{W_3}}}=\left[ {\begin{array}{*{20}{c}} {0.4169}&{0.3614}&{0.1640}&{0.0021}&{0.0000} \end{array}} \right]$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_{{W_4}}}=C \times {Q_{{W_4}}}=\left[ {\begin{array}{*{20}{c}} {0.3314}&{0.3346}&{0.2567}&{0.0223}&{0.0000} \end{array}} \right]$$\end{document} For design cases W1–W4, this study quantified the comprehensive evaluation scores based on the optimized weight distribution and further normalized them to a percentage system to achieve comparability between different cases and to enhance the intuitiveness of the evaluation results. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathrm{Z}}_{{W_1}}}=(\begin{array}{*{20}{c}} {0.1829}&{0.3012}&{0.3228}&{0.1377}&{0.0000} \end{array}){(9080706050)^{\mathrm{T}}}=71.415$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathrm{Z}}_{{W_2}}}=(\begin{array}{*{20}{c}} {0.2689}&{0.3422}&{0.2531}&{0.0816}&{0.0000} \end{array}){(9080706050)^{\mathrm{T}}}=74.190$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathrm{Z}}_{{W_3}}}=(\begin{array}{*{20}{c}} {0.4169}&{0.3614}&{0.1640}&{0.0021}&{0.0000} \end{array}){(9080706050)^{\mathrm{T}}}=78.039$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathrm{Z}}_{{W_4}}}=(\begin{array}{*{20}{c}} {0.3314}&{0.3346}&{0.2567}&{0.0223}&{0.0000} \end{array}){(9080706050)^{\mathrm{T}}}=75.901$$\end{document} In the comparative analysis of smartwatch case studies, W3 (Apple Watch Series 10) achieved the highest overall score among the evaluated products, establishing itself as the benchmark case for validating the assessment framework’s efficacy. The final standardised scores and ranking outcomes for W1 to W4 clearly illustrate each product’s relative performance across all evaluation dimensions. W3 demonstrated a clear advantage in the highest-weighted dimension, “Health and Fitness Tracking” (weight = 0.240). Specifically, its heart rate monitoring (B31) and blood oxygen detection (B35) indicators received “extremely high” ratings from more than 40% of experts, which substantially contributed to its overall score. In addition, W3 performed consistently well in performance hardware (e.g., processing speed and display quality) and design craftsmanship, resulting in a balanced and robust multi-dimensional performance profile. From the user review perspective, health-related features were mentioned in approximately 65% of positive evaluations, indicating a qualitative consistency between the evaluation outcomes and user-perceived value. By contrast, the remaining products exhibited strengths in specific dimensions but showed limitations in addressing multiple high-priority criteria simultaneously. W4 (Samsung Galaxy Watch7) achieved relatively high scores in health monitoring accuracy and connectivity; however, its lower performance in user experience and ecosystem integration constrained its overall ranking. W2 (Xiaomi Watch S4 Sport) performed well in cost-effectiveness and battery endurance, yet these indicators were assigned relatively low weights (0.0309 and 0.0370, respectively), resulting in a limited contribution to the composite score. W1 (Huawei WATCH GT5) showed a clear advantage in battery life but lagged behind in multidimensional integration, particularly in software ecosystem compatibility and interactive experience, which led to a comparatively lower overall score. Taken together, the ranking results reflect the current value orientation of the smartwatch market, where health monitoring capabilities and user experience-related attributes play a dominant role in shaping overall product competitiveness. Although the alignment between the evaluation rankings and market popularity or user ratings is primarily discussed at a qualitative level due to the lack of directly comparable external numerical indicators, the observed consistency in rank order provides preliminary support for the validity of the proposed framework. Future research could further strengthen this validation by incorporating quantitative correlation analysis using standardized sales or rating datasets. Discussion and outlook Methodological contributions of the LDA-FAHP-PSO framework This study employs the LDA topic model to effectively identify latent demand themes within user-generated content, overcoming the limitations of traditional methods that rely on expert experience and prior knowledge to define metrics. Leveraging big data semantic mining, this evaluation model dynamically tracks user preferences, broadens the research perspective on product evaluation, and provides reliable data support and theoretical foundations for innovative design. This facilitates the translation of evaluation theory into design practice. At the methodological level, this study employs multiple mature techniques from various fields to propose a comprehensive evaluation model based on the synergistic operation of LDA, FAHP, and PSO. FAHP plays a pivotal role in integrating expert knowledge and addressing uncertainties within the evaluation process, while PSO compensates for potential consistency biases inherent in traditional subjective judgments through global search and parameter optimization. This integration not only enhances the scientific rigor and stability of indicator weight allocation but also enables more efficient solutions to complex multi-objective evaluation problems, forming a data-driven, intelligent evaluation model. The theoretical contribution primarily lies in the cross-disciplinary integration and systematic innovation of methodologies. This study organically combines semantic data mining, fuzzy analytic hierarchy process, and swarm intelligence optimization to form a closed-loop pathway from requirement information extraction and weight determination to optimization solution. The LDA model reveals the latent thematic structure within textual data, providing a robust data foundation for indicator construction. The coupled application of FAHP and PSO enhances the system’s accuracy and robustness when confronting complex, multidimensional decision-making scenarios. This innovative integration not only deepens the research into product evaluation but also provides quantitative, systematic tools for designing and optimizing smart wearable products. It offers a viable solution for the industry’s transition from experience-driven to data-driven approaches. Evaluation framework extensibility and cross-scenario value This study successfully validated its predefined research objectives by establishing a product evaluation framework. Concurrently, it optimised the systematic assessment pathway for smart wearable devices: firstly, by translating product selection and evaluation processes into computable knowledge models; secondly, by constructing a comprehensive workflow encompassing requirement identification, solution screening, and design optimisation, underpinned by an experiment-data-driven validation mechanism. The LDA-FAHP-PSO integrated evaluation system was validated in a B2C scenario in this study. By performing semantic mining on user-generated content from e-commerce platforms, the LDA model identifies latent demand themes. Combined with FAHP to construct a multidimensional decision matrix, the PSO algorithm achieves global optimization of indicator weights. This enables quantitative mapping and analysis of product attributes against user preferences. The evaluation model also demonstrates application potential in O2O and B2B contexts. In O2O scenarios, evaluations extend beyond online reviews to encompass multiple factors such as offline service experiences, store environments, and logistics delivery. This necessitates expansion in data integration and dynamic feedback mechanisms. At this stage, LDA remains effective in identifying latent needs within social media and review data, while FAHP and PSO can adjust weight distributions by incorporating offline metrics and real-time data to ensure results align with actual operational environments. In B2B scenarios, evaluations place greater emphasis on technical performance, cost-effectiveness, supply chain reliability, and strategic alignment. Unlike B2C user experience data, B2B data sources rely more heavily on standardized corporate procurement records and industry research reports. By expanding the metric system and adjusting algorithm inputs, LDA can still uncover shifts in industry demand. Meanwhile, FAHP and PSO enable optimized decision-making based on expert knowledge and structured data, making evaluation outcomes better aligned with the practical needs of corporate collaboration and long-term strategic planning. In summary, the LDA-FAHP-PSO integrated evaluation system not only enables efficient consolidation and quantitative analysis of user feedback in B2C environments but also extends its applicability to O2O and B2B models through multi-source data fusion and dynamic optimization mechanisms. This framework provides a scientific, transferable support tool for product design, marketing promotion, and strategic decision-making across diverse sales models, demonstrating strong adaptability and practical value. Limitations and future prospects Although the LDA–FAHP–PSO integrated evaluation framework demonstrates strong innovation and adaptability in both methodology and application, several limitations remain. First, the empirical analysis focuses on a case study of smartwatches, whose sample size and scope are relatively limited. This may affect the model’s external validity when applied to larger-scale, multi-category product environments. Secondly, FAHP remains heavily reliant on expert input when constructing fuzzy judgment matrices. Even with consistency optimization via PSO, it cannot entirely eliminate the inherent subjectivity in expert selection bias and pairwise comparison judgments. Furthermore, traditional LDA exhibits limited topic extraction capabilities in scenarios involving short texts and semantic sparsity, potentially resulting in unclear topic boundaries. Finally, in large-scale or real-time evaluation scenarios, LDA topic modeling combined with multi-round PSO optimization incurs high computational costs, imposing certain limitations on system scalability and online application capabilities. Future research may explore the deep integration of advanced machine learning models with ordinal multi-criteria decision-making (MCDM) frameworks to enhance decision-making capabilities for complex systems. In recent years, relevant studies have demonstrated that deep learning exhibits significant advantages in handling high-dimensional feature interactions, temporal dependencies, and highly nonlinear relationships41,42. Consequently, integrating neural network architectures such as Transformers, GNNs, BiLSTMs, and convolutional self-attention into ordinal MCDM holds promise for enhancing automated feature extraction, criterion relationship modeling, and preference learning. This approach opens new possibilities for application in dynamic, multimodal decision-making scenarios. Furthermore, future research can expand the application scope by extending this evaluation framework to comprehensive assessment tasks involving multiple product types, diverse data sources across platforms, and cross-cultural contexts. This will systematically validate the model’s generalizability, adaptability, and robustness. Supplementary Information Below is the link to the electronic supplementary material. Supplementary Material 1 Supplementary Material 1