An intelligent drug supply chain management and recommendation framework using blockchain and TRPO-driven multi-agent learning Authors: Shahrzad Bastani Alahabadi Published: 2026 ABSTRACT Pharmaceutical companies increasingly face difficulties in tracking products across the supply chain, enabling counterfeiters to introduce fake medicines that cause substantial economic losses and serious health risks. A mechanism capable of tracing and monitoring drug movement at every stage is therefore essential. Blockchain offers a promising foundation for secure and transparent supply chain tracking. This paper introduces a two-module framework. It integrates a blockchain-based drug supply chain management (DSCM) system with a multi-agent recommendation model driven by trust region policy optimization (TRPO). The first module employs a customized blockchain to continuously record, monitor, and verify drug movement within a simulated smart pharmaceutical environment. The second module is a sentiment analysis (SA) that operates with two TRPO agents in a blockchain-secured setting. To enhance policy performance, the TRPO agents incorporate entropy regularization. This setup specifically addresses key SA challenges, including handling unlabeled data, feature selection, and class imbalance mitigation. The first agent applies semi-supervised learning (SSL) with pseudo-labels on high-confidence unlabeled samples to expand the training set. The second agent performs SA, applies Shapley additive explanations (SHAP) for feature ranking, and uses reward mechanisms to improve performance on underrepresented classes. The framework was evaluated on two large real-world drug review datasets, Drugs.com and Druglib.com. For Drugs.com, the blockchain module achieved 3.015-second latency and 172.322 tps throughput, while the SA model reached 93.250% accuracy and 94.329% F-measure. For Druglib.com, latency was 2.930 s, throughput was 189.538 tps, accuracy was 95.192%, and F-measure was 96.257%. These results demonstrate the effectiveness of the framework in analyzing patient reviews. It successfully provides secure supply chain recording and sentiment-based insights within controlled experimental conditions. Introduction Counterfeit medications pose a severe threat to global health and economic stability, particularly in developing regions, where they account for nearly 30% of the market1. According to the World Health Organization (WHO), substandard or falsified drugs cause over 200,000 preventable deaths annually, while the pharmaceutical industry faces economic losses exceeding $200 billion2. Although security investments account for nearly 25% of healthcare supply chain expenditures, traditional detection systems still fail to address the inherent complexity and opacity of global distribution networks3. This persistent vulnerability necessitates a transition to secure, transparent, and intelligent pharmaceutical supply chains that ensure drug authenticity and patient safety in real time. Beyond securing the physical distribution of pharmaceuticals, ensuring that medications align with individual patient needs is equally vital for therapeutic success4. In this context, SA-driven drug recommendation systems (DRSs) serve as a strategic bridge between supply chain integrity and clinical outcomes. By integrating patient feedback into supply chain analytics, these systems facilitate a feedback-driven architecture where real-world efficacy and safety data directly inform demand forecasting. This integration enables the detection of underperforming products or counterfeit anomalies at the consumer level, thereby validating both the pharmaceutical authenticity and the clinical utility of the medication5. To address these supply chain vulnerabilities, this study integrates blockchain technology as the foundational layer for a secure and intelligent DSCM framework. Unlike centralized databases, the decentralized and immutable nature of blockchain ensures end-to-end traceability and data integrity throughout the pharmaceutical distribution process6. Through peer-to-peer validation and consensus protocols, the proposed system eliminates data silos and prevents fraudulent record entry, thereby establishing a unified, immutable ledger for all stakeholders. Beyond enhancing transparency, this decentralized architecture provides the fault tolerance and systemic resilience required to maintain operational continuity within multi-actor logistics environments7. For the recommendation framework, various neural architectures, including convolutional neural networks (CNNs), long short-term memory networks (LSTMs), and transformers such as generative pretrained transformer (GPT), have been extensively explored for SA8. However, their performance in specialized clinical domains is often constrained by a reliance on large-scale labeled datasets, which are both costly and scarce9. While SSL leverages unlabeled data through techniques such as pseudo-labeling and consistency training, most existing frameworks employ static sampling strategies10. These fixed criteria fail to adapt to the evolving state of the model, limiting the selection of the most informative samples11. To overcome these limitations, this study introduces a dynamic reinforcement learning (RL) approach, based on TRPO, to guide the SSL sample selection process. Integrating entropy regularization into the TRPO framework stabilizes policy updates, promotes exploration, and addresses class imbalance, ultimately yielding more robust sentiment classification for drug recommendations. Effective feature selection remains a cornerstone of high-performance SA, yet traditional strategies often struggle with the high dimensionality and non-linear complexity of clinical text12. Conventional approaches, ranging from wrapper-based techniques such as recursive feature elimination (RFE) to embedded models such as L1-penalized logistic regression (Lasso), often rely on static evaluation metrics that fail to adapt to evolving data patterns during training13. While attention-based deep learning models offer improvements, they often struggle to distinguish between genuine feature importance and mere correlation14. A primary limitation of these methods is their rigidity; they lack a dynamic feedback loop to refine feature prioritization as the model matures15. To address this, our framework embeds SHAP-based feature ranking within an entropy-regularized TRPO loop. This integration facilitates a dynamic, interpretable selection process that adapts to evolving feature relevance, enhancing both engine accuracy and transparency. Addressing class imbalance is critical for sentiment-driven recommendation systems, as models trained on skewed data often marginalize minority yet significant user sentiments, leading to biased predictions16. While traditional data-level solutions like synthetic minority over-sampling technique (SMOTE) and NearMiss aim to balance distributions, their static nature often leads to overfitting or the loss of crucial context-specific patterns17. Similarly, algorithmic strategies, such as cost-sensitive learning and threshold tuning, rely on fixed penalty structures that lack the flexibility to adapt during training. Over-reliance on these rigid rules often results in a trade-off where gains in minority class recognition compromise overall model stability18. To overcome these challenges, our framework employs entropy-regularized TRPO coupled with a custom reward function. This mechanism adaptively allocates higher rewards and penalties to the minority class, ensuring a balanced optimization that preserves the nuances of rare sentiments without sacrificing global accuracy. The aim of this paper is to propose two modules for medical systems: a DSCM module and a DR module. The DSCM module employs a custom blockchain system to ensure traceability and secure transactions across the supply chain. The second module uses a dual-agent TRPO structure to address drug recommendation through sentiment analysis. The dual-agent TRPO structure is designed to reduce reliance on labeled data, improve feature interpretability, and improve performance for underrepresented sentiment classes. The novelty of this work lies in the synergistic interaction of these modules: blockchain, entropy-regularized TRPO training, SSL, and SHAP-based feature selection. Together, these components form an integrated framework that has not, to the best of our knowledge, been used in prior research. The research questions (RQ) and hypotheses (H) of this paper are defined as follows: RQ1: Can a blockchain-based DSCM system reliably detect and prevent counterfeit medications by enabling full traceability across the PSC?RQ2: Can dual TRPO agents improve the accuracy of RSs under data constraints? The first agent uses SSL to expand labeled data, while the second agent focuses on interpretability and class balance.H1: A two-module framework combining blockchain and TRPO-enhanced SA will outperform traditional systems in both supply chain traceability and recommendation reliability. The blockchain component provides secure monitoring of drug movement. The TRPO agents improve classification performance by managing class imbalance and highlighting key predictive features using SHAP. RQ1: Can a blockchain-based DSCM system reliably detect and prevent counterfeit medications by enabling full traceability across the PSC? RQ1: Can a blockchain-based DSCM system reliably detect and prevent counterfeit medications by enabling full traceability across the PSC? RQ2: Can dual TRPO agents improve the accuracy of RSs under data constraints? The first agent uses SSL to expand labeled data, while the second agent focuses on interpretability and class balance. RQ2: Can dual TRPO agents improve the accuracy of RSs under data constraints? The first agent uses SSL to expand labeled data, while the second agent focuses on interpretability and class balance. H1: A two-module framework combining blockchain and TRPO-enhanced SA will outperform traditional systems in both supply chain traceability and recommendation reliability. The blockchain component provides secure monitoring of drug movement. The TRPO agents improve classification performance by managing class imbalance and highlighting key predictive features using SHAP. H1: A two-module framework combining blockchain and TRPO-enhanced SA will outperform traditional systems in both supply chain traceability and recommendation reliability. The blockchain component provides secure monitoring of drug movement. The TRPO agents improve classification performance by managing class imbalance and highlighting key predictive features using SHAP. The core contributions introduced in this study include: Customized blockchain architecture for DSCM: The proposed model introduces a tailored blockchain infrastructure specifically designed for PSCs. This module provides real-time, immutable tracking of drug movement across distribution stages, significantly reducing the risk of counterfeit infiltration. This blockchain component addresses persistent issues in centralized, opaque drug logistics by ensuring data integrity and transparency.Dual-agent TRPO-based SA system: This study presents a novel SA framework powered by two coordinated TRPO agents, each targeting a specific learning bottleneck. The dual-agent setup addresses key issues in sentiment analysis, including uneven class distributions, missing labeled data, and unclear feature importance. The first agent leverages SSL to expand the training dataset with pseudo-labeled high-confidence samples, addressing data scarcity. The second agent enhances interpretability and robustness by using SHAP for feature ranking and introducing reward mechanisms that prioritize underrepresented sentiment classes.End-to-End integration for smart pharmaceutical applications: What makes this framework distinct is its combination of secure logistics via blockchain with patient feedback analysis using TRPO-SA. This integrated method connects logistics operations with tailored healthcare, supporting systems that can both track drugs and improve recommendations.Comprehensive evaluation and empirical validation: The model is evaluated using two widely used sentiment datasets, Drugs.com and Druglib.com, which contain large collections of patient drug reviews. The performance metrics validate the dual functionality of the framework, confirming high efficiency in both supply chain monitoring and sentiment-based recommendations under experimental conditions. The blockchain component achieves high scalability and fast execution, achieving a throughput of up to 189.5 tps and a latency as low as 2.9 s, indicating the potential feasibility of the framework for healthcare data management. Meanwhile, the SA module delivers strong performance, achieving an accuracy of up to 95.2% and a balanced F-measure of up to 96.3%. These results highlight the ability of the model to analyze patient sentiment while securely managing transaction records. This performance underscores its potential for future research in integrated healthcare data analytics. Customized blockchain architecture for DSCM: The proposed model introduces a tailored blockchain infrastructure specifically designed for PSCs. This module provides real-time, immutable tracking of drug movement across distribution stages, significantly reducing the risk of counterfeit infiltration. This blockchain component addresses persistent issues in centralized, opaque drug logistics by ensuring data integrity and transparency. Customized blockchain architecture for DSCM: The proposed model introduces a tailored blockchain infrastructure specifically designed for PSCs. This module provides real-time, immutable tracking of drug movement across distribution stages, significantly reducing the risk of counterfeit infiltration. This blockchain component addresses persistent issues in centralized, opaque drug logistics by ensuring data integrity and transparency. Dual-agent TRPO-based SA system: This study presents a novel SA framework powered by two coordinated TRPO agents, each targeting a specific learning bottleneck. The dual-agent setup addresses key issues in sentiment analysis, including uneven class distributions, missing labeled data, and unclear feature importance. The first agent leverages SSL to expand the training dataset with pseudo-labeled high-confidence samples, addressing data scarcity. The second agent enhances interpretability and robustness by using SHAP for feature ranking and introducing reward mechanisms that prioritize underrepresented sentiment classes. Dual-agent TRPO-based SA system: This study presents a novel SA framework powered by two coordinated TRPO agents, each targeting a specific learning bottleneck. The dual-agent setup addresses key issues in sentiment analysis, including uneven class distributions, missing labeled data, and unclear feature importance. The first agent leverages SSL to expand the training dataset with pseudo-labeled high-confidence samples, addressing data scarcity. The second agent enhances interpretability and robustness by using SHAP for feature ranking and introducing reward mechanisms that prioritize underrepresented sentiment classes. End-to-End integration for smart pharmaceutical applications: What makes this framework distinct is its combination of secure logistics via blockchain with patient feedback analysis using TRPO-SA. This integrated method connects logistics operations with tailored healthcare, supporting systems that can both track drugs and improve recommendations. End-to-End integration for smart pharmaceutical applications: What makes this framework distinct is its combination of secure logistics via blockchain with patient feedback analysis using TRPO-SA. This integrated method connects logistics operations with tailored healthcare, supporting systems that can both track drugs and improve recommendations. Comprehensive evaluation and empirical validation: The model is evaluated using two widely used sentiment datasets, Drugs.com and Druglib.com, which contain large collections of patient drug reviews. The performance metrics validate the dual functionality of the framework, confirming high efficiency in both supply chain monitoring and sentiment-based recommendations under experimental conditions. The blockchain component achieves high scalability and fast execution, achieving a throughput of up to 189.5 tps and a latency as low as 2.9 s, indicating the potential feasibility of the framework for healthcare data management. Meanwhile, the SA module delivers strong performance, achieving an accuracy of up to 95.2% and a balanced F-measure of up to 96.3%. These results highlight the ability of the model to analyze patient sentiment while securely managing transaction records. This performance underscores its potential for future research in integrated healthcare data analytics. Comprehensive evaluation and empirical validation: The model is evaluated using two widely used sentiment datasets, Drugs.com and Druglib.com, which contain large collections of patient drug reviews. The performance metrics validate the dual functionality of the framework, confirming high efficiency in both supply chain monitoring and sentiment-based recommendations under experimental conditions. The blockchain component achieves high scalability and fast execution, achieving a throughput of up to 189.5 tps and a latency as low as 2.9 s, indicating the potential feasibility of the framework for healthcare data management. Meanwhile, the SA module delivers strong performance, achieving an accuracy of up to 95.2% and a balanced F-measure of up to 96.3%. These results highlight the ability of the model to analyze patient sentiment while securely managing transaction records. This performance underscores its potential for future research in integrated healthcare data analytics. The paper is divided into five primary components. Section 2 provides an in-depth overview of earlier studies on DSCM and DRS, emphasizing both progress and existing research gaps. Section 3 introduces the proposed model and explains its structure, key algorithms, and the computational methods used to improve DSCM. Section 4 presents the experimental evaluation, showcasing how the framework performs under different operational conditions. Lastly, Sect. 5 concludes the work by summarizing the main contributions and offering directions for further research and future enhancements. Related works This section reviews the literature in two main categories: (1) approaches focused on DSCM, and (2) techniques developed for intelligent DRS. DSCM This section groups DSCM strategies into three categories: traditional systems, AI-driven models, and blockchain-based solutions. This classification helps clarify how different approaches contribute to the evolution of the field. Traditional methods Traditional approaches in DSCM relied heavily on operations research (OR), manual planning, and scenario-based modeling. Techniques such as bi-objective optimization, system dynamics modeling, and risk mapping formed the foundation of resilience planning before the emergence of AI-driven or blockchain-enabled systems. Goswami et al.19 introduced a framework for PSCs that combines supply chain mapping with disruption modeling. The approach incorporates input from subject matter experts (SMEs) to support resilience strategy analysis. However, framework complexity may limit real-time scalability in the face of diverse disruptions. Rajabi et al.20 proposed a bi-objective model for resilient PSC design during pandemics. The approach uses scenario-based discrete programming and backup inventory allocation to minimize both drug shortages and operational costs. Yet, scenario explosion increases model runtime and the burden of parameter calibration. Hong et al.21 used system dynamics (SD) modeling to simulate the resilience of the PSC. The model integrates a digital control tower (DCT) to improve coordination, visibility, and adaptive responses to disruptions. However, the SD model lacks granularity for discrete logistics node behavior. AI methods Traditional approaches in DSCM suffer from poor traceability, limited responsiveness, and low resilience to disruptions or the entry of counterfeit drugs. AI techniques address these limitations by enabling real-time data processing and predictive analytics for delivery patterns. They also support dynamic inventory management and automated quality control. AI-based models enhance risk-aware decision-making and anomaly detection throughout the supply chain. These developments have encouraged extensive research into data-driven frameworks that improve adaptability and efficiency in pharmaceutical logistics. Recent advances in AI have introduced a wide array of techniques to optimize various aspects of DSCM. Broadly, AI-driven approaches in this field can be categorized into two thematic groups based on their methodological focus and target application. The first group applies machine learning (ML), DL, and hybrid optimization techniques to predict deliveries, classify supply patterns, and improve decision-making. These models collectively build adaptive and resilient data-driven logistics networks. Hezam et al.22 used three ML models, including gradient boosting (GB), histogram-based gradient boosting (HGB), and cat boosting (CB), to classify delivery status in healthcare SCM (HSCM). They applied feature selection methods and data preprocessing to improve model input. Interpretability remains limited due to the ensemble structure. Anand et al.23 developed a hybrid DL architecture that combines DenseNet 121, a deep CNN (DCNN), with a one-dimensional CNN (1D CNN) to analyze sequential data in HSCM. The combined complexity increases both training and inference time. Shen et al.24 introduced an AI plus vendor-managed inventory (AI + VMI) framework that integrates enterprise resource planning (ERP) and supply processing and distribution (SPD) systems. This setup enables real-time decision-making in hospital drug logistics. However, its adaptation to low-digital settings is still unresolved. Li25 proposed a dual-objective model for vaccine delivery that incorporates analysis of fuel consumption and delay penalties. The model uses a multi-population evolutionary algorithm with simulated annealing and a custom crossover operator. Simultaneous convergence for both objectives remains computationally intensive. Camur et al.26 tested several regression methods, including linear regression, lasso, ridge, elastic net, random forests (RF), gradient boosting machines (GBM), and neural networks, to predict delivery availability using shipment features. Model accuracy is sensitive to missing data and imbalanced inputs under real-time conditions. The second group applies deep RL (DRL) frameworks to manage inventory policies that must adapt to uncertain demand and supply conditions. These studies often model the supply chain as a Markov decision process (MDP), where agents learn optimal restocking actions through interaction with a simulated supply environment. Kaur et al.27 used DRL to optimize pharmaceutical inventory replenishment modeled as an MDP. The system applied real-time variables and a continuous action space to minimize shortages and expirations. However, model generalization may suffer as supply chain dynamics shift. Mohamadi et al.28 applied a DRL framework using the advantage actor-critic (A2C) algorithm to allocate perishable inventory in a two-echelon VMI system affected by random demand and supply changes. Yet, learning stability declines when retail node behavior becomes highly volatile. Blockchain-based methods Centralized drug supply chain systems typically do not ensure transparency or shared oversight. This allows powerful parties to modify records without alerting other involved stakeholders. In contrast, blockchain-based models preserve data integrity through cryptographic techniques, consensus protocols, and tamper-resistant ledgers. These systems enhance visibility, traceability, and secure information exchange among all participants. An online survey found that most users had a positive attitude toward adopting blockchain technology for pharmaceutical logistics. Recent advancements in blockchain applications for DSCM can be categorized into three thematic groups. Each reflects a distinct technical focus, architectural design, and use case. The first group involves blockchain–Internet of things (IoT) integration for real-time tracking, environmental monitoring (e.g., temperature), and secure data logging to enhance traceability, cold chain management, and fraud prevention. In 2023, Hu et al.29 designed an intelligent vaccine supply chain (VSC) by combining blockchain for tamper-proof records, IoT sensors for real-time tracking, and ML for demand prediction. Yet, synchronizing these technologies in real time introduces integration complexity and risks data inconsistency. Nanda et al.6 formulated an architecture linking IoT sensors with blockchain ledgers to enable secure authentication and transparent tracking throughout the medical supply chain. Still, network jitter challenges real-time ledger updates, reducing operational stability. Abdallah et al.30 built a decentralized pharmaceutical sales system using Ethereum smart contracts and IoT monitoring to manage delivery status, ensure compliance, and automate refunds. A drawback is the decline in contract execution reliability under high transaction volumes. Chen et al.7 combined systematic review and expert interviews to develop a blockchain–IoT PSC model focused on post-pandemic resilience. Yet, hardware diversity across hospitals and logistics systems limits interoperability. Turki et al.31 introduced a decentralized drug traceability framework that leverages IoT devices, smart contracts, off-chain storage, and non-fungible tokens (NFTs) to ensure data integrity. Nonetheless, constrained IoT nodes face bandwidth and processing challenges related to NFT integration. Tahmasbzadeh et al.32 developed a traceable supply chain system using Hyperledger Besu and the interplanetary file system (IPFS), in which smart contracts and off-chain hash storage ensure data authenticity. Yet enabling privacy-preserving interoperability across siloed systems remains unresolved. In 2024, Mangala33 presented a layered architecture incorporating IoT, cloud computing, and blockchain via the IoT–mist–edge–fog–cloud (IMEFC) model to enhance responsiveness and security. Despite this, reliance on Filecoin may delay data retrieval under constrained bandwidth. Rizzardi et al.34 implemented a blockchain–IoT framework based on Hyperledger Fabric (HLF) to automate transactions, ensure secure onboarding, and maintain tamper-resistant logs. A limitation remains in scaling secure device registration across large networks. In 2025, Nagargoje et al.35 proposed a blockchain-based framework that integrates fifth-generation (5G) networks and IoT devices. The framework aims to enhance communication reliability, automate transactions using smart contracts, and ensure data integrity, confidentiality, and traceability within healthcare supply chains. However, scalability and interoperability remain challenging for large-scale deployment. Liu et al.36 developed a healthcare supply chain (HSC) framework enabled by IoT devices. The framework combines edge computing, cloud computing, AI-based predictive analytics, and blockchain technology to improve traceability, security, and operational resilience. However, energy efficiency and protocol interoperability still pose implementation challenges. The second group centers on blockchain-only architectures that automate verification and enforce drug provenance through smart contracts and cryptographic protocols, without relying on IoT sensors. In 2023, Bandhu et al.37 created a decentralized HSC on Ethereum, where smart contracts and cryptographic hash functions ensured data integrity while eliminating intermediaries. A key issue is the proof-of-work mechanism of Ethereum, which consumes significant energy during peak loads. Gomasta et al.38 developed a HLF-based platform that uses smart contracts, the elliptic curve digital signature algorithm (ECDSA), and hash-based encryption to secure drug provenance. Yet, integrating with legacy systems presents interoperability challenges. In 2024, Panda et al.39 built a blockchain-based medical supply chain (MSC) system using smart contracts and Web3.js to ensure traceability and dual-party verification, excluding third-party intermediaries. Still, the absence of IoT restricts automated real-time traceability. Datta et al.40 introduced a blockchain-enabled SCM model leveraging the interplanetary file system (IPFS) for off-chain data and smart contracts, with validation via proof-of-work (PoW). PoW-based consensus mechanisms remain computationally expensive. Sharma et al.41 designed a multi-layer authentication architecture on HLF, incorporating smart contracts and quick response (QR) code watermarking to combat counterfeit drugs. Nonetheless, combining cryptographic QR checks with high-throughput supply chain operations adds complexity. Mishra et al.42 implemented a traceable PSC framework using smart contracts, parallel search strategies, and data chunk replication. Formal security analysis confirms tamper resistance, though synchronizing distributed nodes may hinder timely access to data. In 2025, Famous et al.43 developed a blockchain-based framework that uses polymorphic encryption and cloud storage to secure pharmaceutical logistics. The system integrates a decentralized supply chain application (SCMapp) on the Ethereum blockchain for encrypted data management and transparent supplier authentication. However, implementation complexity may hinder scalability across global networks. Kutybayeva et al.44 created the efficient blockchain-enhanced transparent PSC management (EBETPSCM) model. This model integrates blockchain and big data analytics to improve traceability, security, and transparency. It uses HLF to support decentralized consensus and cryptographic validation. However, regulatory alignment and interoperability still require further optimization. The final category incorporates ML or AI into blockchain-based DSCM systems to detect anomalies, predict delays, and improve security. In 2023, Gruchmann et al.45 applied transaction cost economics (TCE) and conducted 25 semi-structured interviews to explore blockchain integration within the Egyptian pharmaceutical ecosystem. Resistance from traditional stakeholders and a limited understanding of blockchain concepts slowed adoption. Yazdinejad46 developed a cyber-physical system (CPS) driven by ML, using bidirectional encoder representations from transformers (BERT) and security analytics to identify malicious activity. A challenge remains in the computational demands and frequent retraining required by transformer-based models. Shah et al.47 designed a hybrid DSCM framework by integrating blockchain with role-based permissions and predictive ML analytics to enable proactive logistics management. Synchronization across distributed nodes and inconsistent data standards limit its scalability. Tanvir et al.48 created a dual-blockchain setup using HLF for confidential data and Quorum for public checks, linked through burn-to-claim and governed via stakeholder reputation. Yet, latency and mismatched governance policies across chains complicate coordination. In 2024, Trautmann et al.49 conducted expert interviews and developed a combined blockchain–IoT–AI architecture to improve traceability and coordination across PSC. Nonetheless, varying levels of digitization and incompatible data formats hinder seamless adoption. In 2025, Rupa et al.50 proposed an Ethereum-based blockchain system for DSCM. The system integrates a decentralized application (DApp) to ensure authorization, confidentiality, and immutability. It uses the Sepolia test network and Truffle Suite for secure transaction tracking. However, deployment scalability and interoperability across heterogeneous systems remain challenging. Mirdad51 developed a blockchain-based pharmaceutical management framework that integrates ML models for demand forecasting and reducing drug waste. It uses a decentralized distributed ledger to provide traceability and transparency. However, optimizing model training and blockchain synchronization remains a challenge. DRS Early models in DRS used classical ML methods such as decision trees (DTs), RFs, and support vector machines (SVMs)52–55. This offers fast, interpretable solutions but struggles with feature engineering and generalization. Hybrid techniques later combined collaborative filtering and content-based filtering56,57 to reduce cold-start issues and personalize suggestions, though they often lacked contextual depth. Graph-based and causality-aware models58–61 capture relationships among diseases, drugs, and patient states. These models improve safety and interpretability but often struggle with scalability and high computational requirements. DL models such as recurrent neural networks (RNNs), LSTMs, graph neural networks (GNNs), and graph convolutional networks (GCNs)62–65 have become prominent with the availability of electronic health records. These models handle sequential data and drug-drug interactions but are computationally intensive and complex to train. Recent methods combine natural language processing (NLP) and SA66–68 to process patient feedback and reviews. While they improve personalization, they are sensitive to noisy inputs and limited supervision. These traditional models have laid foundational groundwork, but often fall short in adaptability, interpretability, or robustness, limitations that newer RL-based methods aim to overcome. Transformer-based models have gained significant traction in DRS for their ability to capture semantic context and model complex relationships in clinical text. Early works used BERT-based models for SA and prescription classification from patient reviews69,70. These models improved contextual understanding, but had limited interpretability and required significant computational resources. More advanced systems utilize multimodal inputs, including text, audio, visuals, and structured clinical data, by employing complex architectures such as Bayesian functional with Gaussian tanh-based deep bidirectional LSTM (BFGT-DBi-LSTM)71 and vision transformers72. While these approaches significantly enhance adaptability, they also result in a substantial increase in overall model complexity. Later hybrid designs combined entity recognition transformers (ERT), large language models (LLMs), and mixtures of experts (MOE)73,74. These models enabled personalization using multimodal patient data, though they struggled with integration and scalability. Recent efforts have integrated GPT-4 and similar LLMs with medical knowledge graphs to enrich RSs75–78. These models use contextual embeddings and structured knowledge to improve drug suggestions and clinical trial recommendations. However, they require curated data and face regulatory barriers when deployed in real-world settings. Positioning of the proposed model within existing research While recent blockchain-based frameworks in DSCM have significantly enhanced data integrity and counterfeit detection, they remain primarily focused on the logistics of authenticity. These systems lack the analytical intelligence required for real-time decision-making and personalized DRS. In modern pharmaceutical ecosystems, ensuring a drug is genuine is only half the challenge; ensuring it is the right fit for the evolving feedback of the patient is equally critical. Our work addresses this gap by combining immutable blockchain records with adaptive AI-based recommendations. This creates a unified system for both supply chain security and clinical intelligence. Furthermore, existing DRS models often operate in a logistical vacuum, ignoring the complexities of the pharmaceutical supply chain, such as drug authenticity and delivery reliability. Technically, many deep learning and transformer-based models struggle with sparse, unlabeled, and imbalanced clinical datasets. As a result, they often produce black-box predictions with limited interpretability. These limitations hinder their adoption in industry and clinical settings, where transparency and evidence-based reasoning are mandatory. By integrating a dual-agent TRPO architecture within a secure blockchain environment, this study shifts the focus from isolated optimization to an integrated ecosystem. Unlike prior studies that treat traceability and SA as disparate domains, our framework positions them as interdependent layers. This allows the system to use supply chain transparency to validate the recommendation context. At the same time, advanced RL addresses data scarcity, feature selection, and class imbalance. Consequently, this research provides a holistic solution that aligns pharmaceutical supply integrity with high-precision, interpretable patient-centric feedback. Fig. 1Overview of the proposed dual module framework integrating blockchain-based DSCM with a multi-agent driven DRS. The blockchain module ensures transparency, traceability, and authenticity across all supply participants, while the dual agent recommendation module operates within this secure environment. Overview of the proposed dual module framework integrating blockchain-based DSCM with a multi-agent driven DRS. The blockchain module ensures transparency, traceability, and authenticity across all supply participants, while the dual agent recommendation module operates within this secure environment. Materials and methods The architecture of the proposed model is illustrated in Fig. 1. It consists of two main modules. The first module is a blockchain-enabled DSCM that provides secure, immutable, and traceable recording of pharmaceutical transactions. The second module is a multi-agent based on TRPO that performs DRS. Blockchain-enabled DSCM infrastructure The system uses a decentralized blockchain network to secure data, improve traceability, and maintain transaction integrity across the PSC. Figure 1 shows the architecture of the DSC framework, in which each participant interacts with the blockchain via a dedicated client interface. This interface supports real-time tracking and management of supply activities. The blockchain records information from all major participants, suppliers, manufacturers, distributors, pharmacies, hospitals, and patients, ensuring transparency and protection from data tampering. Core system assets include medicines, raw materials, purchase orders, and transaction logs. Users access lightweight applications that allow them to initiate operations, verify deliveries, and track product movements. A connected off-chain data repository supports large-scale analysis and feeds the recommendation module. The ML engine processes this data to generate customized drug suggestions, trained on historical sentiment and feedback patterns. Users can also access detailed product information such as cost, shelf life, manufacturing date, and ingredients, helping maintain accuracy and informed decision-making throughout the supply process. To maintain synchronization of the distributed ledger, peer nodes run the consensus protocol. The process begins when the main supplier sends raw materials to the pharmaceutical company. Each participant interacts with the system through a dedicated interface that allows them to log in, start transactions, and track their activities in real time across the blockchain network. For example, when a manufacturer places an order for raw materials, any peer node can validate the request. Once confirmed, the supplier receives the order. When the materials are ready for shipment, the supplier triggers a confirmation event in the system. When a patient or regulator requests detailed information about a drug, the built-in permissions of the blockchain determine which data can be viewed. Likewise, suppose a doctor places an order with a manufacturer. In that case, the workflow follows a consistent sequence: the doctor is authenticated, a transaction request is submitted, the manufacturer verifies it, and a confirmation message is issued once the transaction is approved and logged on the ledger. Multi-agent RL for DRS Figure 2 shows the structure of the proposed DRS. The proposed DRS addresses three primary challenges: data scarcity, class imbalance, and low interpretability of models. The first agent employs a semi-supervised protocol, assigning pseudo-labels to high-confidence unlabeled reviews to expand the effective training dataset. The second agent performs sentiment classification and utilizes SHAP values to rank influential features for clinical interpretation. To mitigate class imbalance, this agent uses a weighted reward mechanism integrated into the TRPO loop that prioritizes underrepresented categories, such as adverse drug reactions. Both agents are optimized using entropy-regularized TRPO. Fig. 2Overview of the proposed DRS framework. It employs a TRPO-based dual-agent system. The first agent leverages SSL via pseudo-labeling, and the second performs an SA model based on interpretability signals (e.g., SHAP). A custom reward mechanism addresses class imbalance. Overview of the proposed DRS framework. It employs a TRPO-based dual-agent system. The first agent leverages SSL via pseudo-labeling, and the second performs an SA model based on interpretability signals (e.g., SHAP). A custom reward mechanism addresses class imbalance. The SSL agent This study utilizes a dataset \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:X={\left\{{x}_{i}\right\}}_{i=1}^{N}$$\end{document}, where each \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{i}$$\end{document} denotes the i-th textual input and N is the total number of samples. Each sample \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{i}$$\end{document} carries an associated sentiment tag \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{c}_{i}$$\end{document}, selected from a finite label space \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:c={\left\{{c}_{i}\right\}}_{i=1}^{C}$$\end{document}, where C specifies the number of sentiment categories. The labeled subset is indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}_{L}={\left\{\right({x}_{{L}_{i}},{y}_{{L}_{i}}\left)\right\}}_{i=1}^{{N}_{L}}$$\end{document}, with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{{L}_{i}}$$\end{document} representing the i-th annotated input and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{y}_{{L}_{i}}$$\end{document} its corresponding label. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{N}_{L}$$\end{document} is the total count of labeled items. Conversely, the unlabeled split is captured as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}_{U}={\left\{{x}_{{U}_{i}}\right\}}_{i=1}^{{N}_{U}}$$\end{document}, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{{U}_{i}}$$\end{document} denotes the i-th unannotated entry and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{N}_{U}$$\end{document} is the number of such examples. These unlabeled samples serve as the basis for pseudo-label generation in the semi-supervised training workflow. The following explains the components of the TRPO-based decision module at the core of the system: State: At each iteration, the agent captures its surroundings through a state descriptor composed of key contextual signals. This descriptor merges two main sources of information: (1) the sentence-level feature representation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f}_{{U}_{t}}$$\end{document}, extracted via the GPT-LSTM network of the second agent, and (2) the sentiment class of the sentence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{p}_{cls}\left({x}_{{U}_{t}}\right)$$\end{document}, also derived from the second agent. At time step t, these two features form a time-varying sequence of environment states, represented as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{s}_{t}=({f}_{{U}_{t}},{p}_{cls}({x}_{{U}_{t}}\left)\right)$$\end{document}.Action: At every decision point, the agent chooses between two possible actions, expressed as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}\:\in\:\:\{0,\:1\}$$\end{document}. If \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}=1$$\end{document}, the agent treats the current input \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{i}^{t}$$\end{document} as reliable according to the model confidence and assigns it a pseudo-label. The sample is then incorporated into the labeled dataset for subsequent training. Conversely, if \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}\:=\:0$$\end{document}, the sample is set aside and excluded from the current learning cycle. This decision-making process operates autonomously using the internal confidence values of the model, eliminating the need for labeling.Reward: The agent receives performance feedback based on the prediction uncertainty of the model, which is calculated using an entropy metric. This feedback guides the learning process by rewarding or penalizing actions based on the confidence of the generated outputs. The corresponding reward formulation is presented in Eq. 1: 1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:R\left({s}_{t},{a}_{t}\right)=\left\{\begin{array}{c}0\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:if\:{a}_{t}=0\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\\\:+\lambda\:,\:\:\:{if\:{a}_{t}=1,\:\:L}_{\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{p}\mathrm{y}}\:\left({x}_{{U}_{i}}\right)<\alpha\:,\:and\:M\:\left({x}_{{U}_{i}}\right)>\beta\:\\\:-1,\:\:\:\:\:\:\:\:if\:{a}_{t}=1\:and\:L\:\left({x}_{{U}_{i}},\:c\right)>\alpha\:,\:and\:M\:\left({x}_{{U}_{i}}\right)<\beta\:\end{array}\right.\:$$\end{document} State: At each iteration, the agent captures its surroundings through a state descriptor composed of key contextual signals. This descriptor merges two main sources of information: (1) the sentence-level feature representation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f}_{{U}_{t}}$$\end{document}, extracted via the GPT-LSTM network of the second agent, and (2) the sentiment class of the sentence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{p}_{cls}\left({x}_{{U}_{t}}\right)$$\end{document}, also derived from the second agent. At time step t, these two features form a time-varying sequence of environment states, represented as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{s}_{t}=({f}_{{U}_{t}},{p}_{cls}({x}_{{U}_{t}}\left)\right)$$\end{document}. State: At each iteration, the agent captures its surroundings through a state descriptor composed of key contextual signals. This descriptor merges two main sources of information: (1) the sentence-level feature representation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f}_{{U}_{t}}$$\end{document}, extracted via the GPT-LSTM network of the second agent, and (2) the sentiment class of the sentence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{p}_{cls}\left({x}_{{U}_{t}}\right)$$\end{document}, also derived from the second agent. At time step t, these two features form a time-varying sequence of environment states, represented as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{s}_{t}=({f}_{{U}_{t}},{p}_{cls}({x}_{{U}_{t}}\left)\right)$$\end{document}. Action: At every decision point, the agent chooses between two possible actions, expressed as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}\:\in\:\:\{0,\:1\}$$\end{document}. If \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}=1$$\end{document}, the agent treats the current input \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{i}^{t}$$\end{document} as reliable according to the model confidence and assigns it a pseudo-label. The sample is then incorporated into the labeled dataset for subsequent training. Conversely, if \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}\:=\:0$$\end{document}, the sample is set aside and excluded from the current learning cycle. This decision-making process operates autonomously using the internal confidence values of the model, eliminating the need for labeling. Action: At every decision point, the agent chooses between two possible actions, expressed as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}\:\in\:\:\{0,\:1\}$$\end{document}. If \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}=1$$\end{document}, the agent treats the current input \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{i}^{t}$$\end{document} as reliable according to the model confidence and assigns it a pseudo-label. The sample is then incorporated into the labeled dataset for subsequent training. Conversely, if \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}\:=\:0$$\end{document}, the sample is set aside and excluded from the current learning cycle. This decision-making process operates autonomously using the internal confidence values of the model, eliminating the need for labeling. Reward: The agent receives performance feedback based on the prediction uncertainty of the model, which is calculated using an entropy metric. This feedback guides the learning process by rewarding or penalizing actions based on the confidence of the generated outputs. The corresponding reward formulation is presented in Eq. 1: Reward: The agent receives performance feedback based on the prediction uncertainty of the model, which is calculated using an entropy metric. This feedback guides the learning process by rewarding or penalizing actions based on the confidence of the generated outputs. The corresponding reward formulation is presented in Eq. 1: where\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\:L}_{\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{p}\mathrm{y}}\:\left({x}_{{U}_{t}}\right)$$\end{document} is computed as Eq. 2:2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{L}_{\mathrm{entropy\:}}\left({x}_{{U}_{t}}\right)=-\sum\:_{j=1}^{C}\:{p}_{j}\mathrm{l}\mathrm{o}\mathrm{g}{p}_{j}\:$$\end{document} Within this framework, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\alpha\:$$\end{document} serves as a dynamic confidence boundary that progressively adjusts as training progresses. The variable \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{p}_{j}$$\end{document} denotes the model-assigned probability for class j. When the entropy of a sample falls below this boundary, the outcome is considered reliable and eligible for pseudo-label generation. In such cases, the agent receives a positive incentive of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:+\lambda\:$$\end{document}. Conversely, if the entropy surpasses \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\alpha\:$$\end{document}, the prediction is considered uncertain, and a negative reward of − 1 is applied to discourage the selection of low-confidence examples. The function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:M\:\left({x}_{{U}_{t}}\right)$$\end{document}quantifies the margin separating the two largest class probability estimates, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{p}_{{top}_{1}}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{p}_{{top}_{2}}$$\end{document}. This margin indicates how distinct the most likely class is from the runner-up prediction, serving as a measure of model confidence. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:M\:\left({x}_{{U}_{t}}\right)$$\end{document} is computed based on Eq. 3:3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:M\:\left({x}_{{U}_{t}}\right)={p}_{{top}_{1}}-{p}_{{top}_{2}}\:$$\end{document} This probability gap helps identify unlabeled samples whose predictions are both confident (low entropy) and well separated from competing categories, thereby minimizing classification ambiguity. A wider gap implies sharper decision boundaries, suggesting that the assigned pseudo-label is more dependable. To regulate the integration of pseudo-labeled data during learning, the confidence boundary \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\alpha\:$$\end{document} is designed to evolve dynamically rather than stay static. At the beginning of training, predictions tend to be unstable; therefore, using a relatively high \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\alpha\:$$\end{document} ensures that only highly confident samples are selected. As the model stabilizes, this boundary is gradually lowered, allowing moderately confident examples to participate in training. This schedule balances exploration and exploitation: it filters noisy pseudo-labels early and widens data use later. Instead of a fixed uncertainty limit, the method uses time-based annealing where α decreases exponentially over training epochs, as shown in Eq. 4:4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\alpha\:}_{t}={\alpha\:}_{\mathrm{init\:}}\times\:\mathrm{e}\mathrm{x}\mathrm{p}(-\mu\:t)\:$$\end{document} In this expression, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\alpha\:}_{\mathrm{init\:}}$$\end{document} represents the initial confidence value, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mu\:$$\end{document} determines the exponential reduction factor, and t denotes the current training epoch. This adaptive schedule updates confidence estimates over training, improves pseudo-label quality, and boosts semi-supervised performance. Fig. 3Visualization of entropy-guided pseudo-label generation across 20 unlabeled samples, regulated by an adaptive confidence boundary (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\delta\:\:=\:0.5$$\end{document}). Visualization of entropy-guided pseudo-label generation across 20 unlabeled samples, regulated by an adaptive confidence boundary (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\delta\:\:=\:0.5$$\end{document}). Figure 3 shows how pseudo-labels are assigned to 20 unlabeled samples. This method uses entropy as a selection signal, guided by a flexible threshold δ that starts at 0.5. Each column in the figure represents an unlabeled data point, while the vertical axis shows the entropy value obtained from model predictions. Bars shown in yellow indicate instances whose entropy values fall below \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\delta\:$$\end{document}, leading to selection and assignment of a positive reward (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:+\lambda\:$$\end{document}). In contrast, brown bars identify samples exceeding \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\delta\:$$\end{document}, which are discarded and assigned a negative reward (− 1). The dashed yellow line marks the current confidence limit of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\delta\:\:=\:0.5$$\end{document}. Samples demonstrating lower entropy are interpreted as more trustworthy and are thus included for pseudo-labeling, whereas high-entropy cases are removed to reduce label noise. This screening method directs the model to concentrate on samples with stronger confidence during the initial learning phase. Over time, δ gradually declines, allowing a wider variety of instances to be included. The visualization emphasizes how the entropy-based feedback mechanism enables the model to rank and select data according to certainty levels. In the semi-supervised setup, the model uses two fully connected layers that output scores, each linked to one possible action. When the chosen action is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}=\:$$\end{document}0, the current instance is ignored and excluded from pseudo-label generation. Conversely, if \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}=\:1$$\end{document}, the agent forwards the instance to the secondary module (SA) responsible for assigning a pseudo-label. The decision is made by selecting the action with the higher predicted value. Throughout training, every interaction is logged as a transition tuple \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:({s}_{t},{a}_{t},{r}_{t},{s}_{t+1})$$\end{document}. In this notation, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{s}_{t}$$\end{document} denotes the current observation, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}$$\end{document} specifies the selected operation, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{r}_{t}$$\end{document} represents the reward computed from entropy and class margin, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{s}_{t+1}$$\end{document} defines the subsequent observation after the action. This setup helps the agent improve its action strategy over time by skipping uncertain samples and favoring confident ones. Algorithm 1Pseudocode for pseudo-labeling with TRPO-based SSL. Pseudocode for pseudo-labeling with TRPO-based SSL. Algorithm 1 outlines the proposed RL-guided pseudo-labeling strategy for SSL using TRPO. The process begins with a small labeled set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}_{L}$$\end{document} and a larger unlabeled pool \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}_{U}$$\end{document}. A base model is first trained solely on \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}_{L}$$\end{document}. Then, during each iteration, the algorithm randomly permutes \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}_{U}$$\end{document} to avoid selection bias. For each unlabeled sample, the algorithm first extracts a feature embedding using the GPT-LSTM, then produces a sentiment class label using the SA agent. These two vectors form the current state, which is passed to a policy network \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\pi\:$$\end{document}. Based on the output action \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{a}_{t}\in\:\:$$\end{document}{0,1}, the sample is either selected for pseudo-labeling or ignored. Accepted samples are added to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{X}_{L}$$\end{document}, and a reward is assigned based on prediction confidence. The policy is updated using the proposed TRPO to optimize action selection over time. The loop continues until convergence or the unlabeled pool is exhausted. The SA agent The second component functions as an SA engine. It classifies each input sentence into one of several sentiment categories using the labeled dataset \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{D}_{L}$$\end{document}. During each training iteration, a sample \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{{L}_{t}}$$\end{document} is randomly drawn from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{D}_{L}$$\end{document}. As shown in Fig. 2, each sentence is passed through GPT and LSTM modules to extract semantic and contextual features. The encoded representation is then passed to a multilayer perceptron (MLP). The MLP performs two key tasks: determining the relative importance of each word and predicting the sentiment label of the sequence. The model outputs a composite vector of size n+m, where n corresponds to the number of words in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{{L}_{t}}$$\end{document} and captures word importance, while m represents sentiment category probabilities. These attention weights improve the interpretability of model predictions. Under the TRPO framework, the agent operates in a defined environment comprising states, actions, and rewards to guide semi-supervised RL. The definitions of state, action, and reward are outlined as follows. State: The state \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{s}_{t}$$\end{document} represents the observable condition of the model at time step t and consists of three key elements: (1) The GPT-LSTM encoded representation of the input sentence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{{L}_{t}}$$\end{document}, (2) The token-level importance scores produced by the MLP, and (3) The set of currently selected tokens \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{y}_{active}$$\end{document}, which tracks words already identified as sentiment-relevant. Together, these components provide the agent with a complete view of the environment, enabling sequential and non-redundant decision-making for both feature selection and sentiment classification.Action and reward: The action space of the agent, A, is structured into two distinct roles: (1) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document}: responsible for choosing individual word-level features to improve model transparency, (2) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{c}$$\end{document}: focused on assigning the sentiment class of the input sequence. When a selection from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document} is made, the corresponding word feature is added to the current feature set. This enriched set is then used in future prediction steps. The reward for this action is calculated based on the SHAP-derived importance score of the selected word, as detailed in Eq. 5: 5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:r\left(\left({x}_{{L}_{t}},{y}_{{L}_{t}}\right),a\in\:{A}_{f}\right)=\:\mu\:\times\:\mathrm{S}\mathrm{H}\mathrm{A}\mathrm{P}\:\left({w}_{best}\right)$$\end{document} State: The state \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{s}_{t}$$\end{document} represents the observable condition of the model at time step t and consists of three key elements: (1) The GPT-LSTM encoded representation of the input sentence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{{L}_{t}}$$\end{document}, (2) The token-level importance scores produced by the MLP, and (3) The set of currently selected tokens \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{y}_{active}$$\end{document}, which tracks words already identified as sentiment-relevant. Together, these components provide the agent with a complete view of the environment, enabling sequential and non-redundant decision-making for both feature selection and sentiment classification. State: The state \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{s}_{t}$$\end{document} represents the observable condition of the model at time step t and consists of three key elements: (1) The GPT-LSTM encoded representation of the input sentence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{x}_{{L}_{t}}$$\end{document}, (2) The token-level importance scores produced by the MLP, and (3) The set of currently selected tokens \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{y}_{active}$$\end{document}, which tracks words already identified as sentiment-relevant. Together, these components provide the agent with a complete view of the environment, enabling sequential and non-redundant decision-making for both feature selection and sentiment classification. Action and reward: The action space of the agent, A, is structured into two distinct roles: (1) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document}: responsible for choosing individual word-level features to improve model transparency, (2) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{c}$$\end{document}: focused on assigning the sentiment class of the input sequence. When a selection from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document} is made, the corresponding word feature is added to the current feature set. This enriched set is then used in future prediction steps. The reward for this action is calculated based on the SHAP-derived importance score of the selected word, as detailed in Eq. 5: Action and reward: The action space of the agent, A, is structured into two distinct roles: (1) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document}: responsible for choosing individual word-level features to improve model transparency, (2) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{c}$$\end{document}: focused on assigning the sentiment class of the input sequence. When a selection from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document} is made, the corresponding word feature is added to the current feature set. This enriched set is then used in future prediction steps. The reward for this action is calculated based on the SHAP-derived importance score of the selected word, as detailed in Eq. 5: In this context, the parameter \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mu\:$$\end{document} acts as a regularization factor that imposes a penalty to control model complexity. The term \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{w}_{best}$$\end{document} identifies the word that the MLP assigns the highest significance during prediction. This reward strategy prioritizes accurate keyword identification by incorporating interpretability signals from SHAP at the word level. As a result, the agent is encouraged to concentrate on terms that have a stronger influence on sentiment classification. The TRPO approach gradually updates the policy of the agent using these feedback signals. This iterative adaptation allows the agent to more effectively recognize meaningful linguistic patterns and propagate them across relevant words. Altogether, this learning framework improves both the accuracy of SA and the transparency of the reasoning process of the model in the context of recommendation tasks. If an action from the set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{c}$$\end{document} is executed, the feedback provided depends on whether the predicted class label is correct. The reward corresponding to the i-th input sequence is calculated according to Eq. 6:6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:r(\left({x}_{{L}_{t}},{y}_{{L}_{t}}\right),a\in\:{A}_{c})=\left\{\begin{array}{c}+1\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:if\:{y}_{{L}_{t}}\in\:{D}_{O},{\widehat{y}}_{{L}_{t}}=\:{y}_{{L}_{t}}\:\:\:\:\:\:\:\:\:\:\:\:\\\:-1\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:if\:{y}_{{L}_{t}}\in\:{D}_{O},{\widehat{y}}_{{L}_{t}}\ne\:\:{y}_{{L}_{t}}\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\\\:+\gamma\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:if\:{y}_{{L}_{t}}\in\:{D}_{s},{\widehat{y}}_{{L}_{t}}=\:{y}_{{L}_{t}}\:\:\:\:\:\:\:\:\:\:\:\:\:\\\:-\gamma\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:if\:{y}_{{L}_{t}}\in\:{D}_{s},{\widehat{y}}_{{L}_{t}}\ne\:\:{y}_{{L}_{t}}\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\end{array}\right.\:\:\:\:\:\:\:\:\:$$\end{document} In this setting, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\widehat{y}}_{{L}_{t}}$$\end{document} represents the predicted sentiment label for the i-th input sentence. The subsets \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{D}_{O}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{D}_{s}$$\end{document} correspond to the underrepresented and dominant sentiment groups, respectively. The underrepresented set contains the least frequently observed sentiment category, while the dominant set contains the most frequently observed sentiment. The parameter \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\gamma\:$$\end{document}, defined within the range \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:0<\gamma\:<10$$\end{document}, controls the reward for correct predictions of the dominant class. This reward formulation is intentionally designed to favor accurate predictions of the underrepresented sentiment by assigning them a higher value, thereby reducing class imbalance. Through this scheme, the model is incentivized to detect subtle and less frequent emotional tones rather than defaulting to the majority class. As training progresses, the TRPO-driven policy updates in response to these signals, gradually learning to extract features linked to minority sentiment. This results in more equitable performance across sentiment categories. In the SA, the transition mechanism, formally expressed as a mapping \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:t:\:S\:\times\:\:A\:\to\:\:S\:\cup\:\:T$$\end{document}, is detailed in Eq. 7:7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:t\left(\left(x,y\right),a\right)=\:\left\{\begin{array}{c}T\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:if\:a\in\:{A}_{c}\\\:\left(x,y\cup\:a\right)\:\:\:\:\:\:if\:a\in\:{A}_{f}\end{array}\right.$$\end{document} Here, T denotes the end of an episode. If the selected action belongs to the feature-selection group (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:a\in\:{A}_{f}$$\end{document}), the corresponding word is added to the active set of important words. In contrast, if the agent selects an action from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{c}$$\end{document}, it assigns a sentiment label to the sentence and completes the episode for that input. The training then proceeds to the next input sample. At each step t, the agent selects one word from the input and evaluates its relevance using the SAHP (SA highlighting protocol) mechanism. Gradually, the agent builds a set of sentiment-relevant words, which are then used to determine the final classification. Traditional models generate predictions in a single step. In contrast, this method separates the process into multiple rounds of interpretation and decision-making. This sequential process enhances interpretability and encourages the model to justify its sentiment predictions using word-level evidence, a crucial property for recommendation or feedback systems. Algorithm 2 outlines the training steps used for the proposed SA model. This model uses TRPO to improve its policy through repeated updates. At each training epoch, the model processes a batch of labeled samples. For each sentence, it first extracts contextual embeddings using a hybrid GPT-LSTM encoder. Then, an MLP generates token-level importance scores. The agent runs through a sequence of decisions. At each step, it observes the current state, which includes the embeddings, token scores, and the active word set. Based on this, it selects one of two actions: either add a token to the active word list or assign a sentiment label. Rewards are calculated accordingly. Equation 5 handles token selection, and Eq. 6 handles classification. All interaction records are stored. TRPO then updates the policy by maximizing the expected return while enforcing a constraint to stabilize updates. Theoretical basis of SHAP values SHAP metrics originate from the Shapley framework in collaborative game theory. The expression used to calculate the SHAP attribution for a specific input feature iii, denoted by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{{\varnothing}}_{i}$$\end{document}, is as Eq. 8:8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\varnothing\:}_{i}=\sum\:_{S\subseteq\:F\backslash\:\left\{i\right\}}\frac{\left|S\right|!\left(\left|F\right|-\left|S\right|-1\right)!}{\left|F\right|!}\left(v\right(S\cup\:\left\{i\right\})-v(S\left)\right)$$\end{document} In this formulation, F is the complete set of features, and S is any subset that excludes feature i. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left|S\right|$$\end{document} shows the size of subset S, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left|F\right|$$\end{document} is the total number of features. The model output when only features in S are used is denoted by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:v\left(S\right)$$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:v(S\cup\:\left\{i\right\})$$\end{document} gives the output when feature i is added. SHAP computes the average change in the prediction of the model when feature i is added across all subsets, capturing its overall contribution. Figure 4 shows how SHAP values are calculated using the feature set {A, B, C}. It explains how each feature affects the model prediction. The process begins by selecting the features of interest, A, B, and C, and forming all possible subsets. To measure the impact of a feature like A, the model compares predictions with and without A across different subsets such as {}, {B}, {C}, and {B, C}. This shows how much A contributes in different situations. These contributions are averaged across all possible feature orders to ensure a fair result. This prevents any feature from being unfairly favored or penalized due to its position. SHAP then assigns a share of the prediction to each feature based on these averages. A positive SHAP value indicates that the feature increases the prediction, while a negative value indicates that it decreases it. This method clearly breaks down the prediction and shows how much each feature matters. Algorithm 2Pseudocode for the TRPO-based SA model. Pseudocode for the TRPO-based SA model. Fig. 4Illustration of the SHAP value calculation process for feature A. Illustration of the SHAP value calculation process for feature A. Training This section begins with a summary of the foundational TRPO framework and then details the modifications introduced by integrating an entropy-based regularization component. Original TRPO The main objective in RL is to optimize the cumulative reward obtained starting from the initial state. This objective is mathematically represented by the policy evaluation function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:\left(\pi\:\right)={\mathrm{E}}_{\mathrm{s}0,\mathrm{a}0,\dots\:.}\left[{R}_{0}\right]$$\end{document}, which expresses the expected return under policy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\pi\:$$\end{document}. A step-by-step explanation of this formulation is provided as Eq. 9:9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:\left(\pi\:\right)=E{s}_{0},{a}_{0},\dots\:\left[\sum\:_{t=0}^{{\infty\:}}\:{\gamma\:}^{t}r\left({s}_{t},{a}_{kt}\right)\right]$$\end{document} where10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{s}_{0}\sim\:{\rho\:}_{0},{a}_{t}\sim\:\pi\:\left(.,{s}_{t}\right),{s}_{t+1}\sim\:P\left(.,{s}_{t},{a}_{t}\right)$$\end{document} The expected return under an alternative policy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\stackrel{\sim}{\pi\:}$$\end{document} is computed by summing the advantage values across time steps, compared to the baseline policy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\pi\:$$\end{document}, as shown in Eq. 11:11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\begin{array}{c}\eta\:\left(\stackrel{\sim}{\pi\:}\right)=\eta\:\left(\pi\:\right)+{E}_{{s}_{0},{a}_{0},\dots\:\sim\:\stackrel{\sim}{\pi\:}}\left[\sum\:_{t=0}^{{\infty\:}}\:{\gamma\:}^{t}{A}_{\pi\:}\left({s}_{t},{a}_{t}\right)\right]=\\\:=\eta\:\left(\pi\:\right)+\sum\:_{s}\:{\rho\:}_{\stackrel{\sim}{\pi\:}}\left(s\right)\sum\:_{a}\:\stackrel{\sim}{\pi\:}(a\mid\:s){A}_{\pi\:}(s,a)\end{array}$$\end{document} Moving from policy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\pi\:$$\end{document} to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\stackrel{\sim}{\pi\:}$$\end{document} improves performance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:$$\end{document} when the condition \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\sum\:_{s}\:{\rho\:}_{\stackrel{\sim}{\pi\:}}\left(s\right)\sum\:_{a}\:\stackrel{\sim}{\pi\:}(a,s){A}_{\pi\:}(s,a)\ge\:0$$\end{document} holds. The main difficulty is that the state visitation frequency \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\rho\:}_{\stackrel{\sim}{\pi\:}}\left(s\right)$$\end{document} depends on the updated policy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\stackrel{\sim}{\pi\:}$$\end{document}, which complicates optimization. To simplify this dependency, a localized approximation of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:\left(\stackrel{\sim}{\pi\:}\right)$$\end{document} is used by replacing \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\rho\:}_{\stackrel{\sim}{\pi\:}}$$\end{document} with the distribution of the current policy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\rho\:}_{\pi\:}$$\end{document}, as shown in Eq. 12:12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{L}_{\pi\:}\left(\stackrel{\sim}{\pi\:}\right)=\eta\:\left(\pi\:\right)+\sum\:_{s}\:{\rho\:}_{\pi\:}\left(s\right)\sum\:_{a}\:\stackrel{\sim}{\pi\:}(a\mid\:s){A}_{\pi\:}(s,a)$$\end{document} Adjusting the policy from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\pi\:$$\end{document} to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\stackrel{\sim}{\pi\:}$$\end{document} typically increases \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{L}_{\pi\:}$$\end{document} and, in turn, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:$$\end{document}. However, the degree of improvement in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:$$\end{document} is limited by the size of the policy update. To address this constraint, the TRPO algorithm was introduced, building on the ideas behind conservative policy improvement. TRPO defines an alternative objective function based on a rigorously proven lower bound of the performance of the original policy. By optimizing this surrogate objective, the algorithm guarantees stable, consistent policy improvements. The mathematical form of this lower performance bound is expressed in Eq. 13:13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:\left({\pi\:}_{\mathrm{new\:}}\right)\ge\:{L}_{{\pi\:}_{old}}\left({\pi\:}_{\mathrm{new\:}}\right)-\frac{4 \epsilon \gamma\:}{(1-\gamma\:{)}^{2}}{D}_{KL}^{max}\left({\pi\:}_{\mathrm{old\:}},{\pi\:}_{\mathrm{new}\text{}}\right)$$\end{document} In this setting, the parameter \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\: \epsilon$$\end{document} represents the maximum observed advantage across every state-action combination and is defined as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\: \epsilon =\underset{s,a}{max}\:\left|{A}_{{\pi\:}_{old}}(s,a)\right|$$\end{document}. The policies \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\pi\:}_{\mathrm{old\:}}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\pi\:}_{\mathrm{new}\text{}}$$\end{document} refer to the previous and current policy versions, respectively. The term \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{D}_{kl}^{max}\left({\pi\:}_{\mathrm{old\:}},{\pi\:}_{\mathrm{new\:}}\right)$$\end{document} quantifies the largest Kullback–Leibler (KL) divergence between the two policy distributions and is defined mathematically as Eq. 14:14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{D}_{kl}^{max}\left({\pi\:}_{\mathrm{old\:}},{\pi\:}_{\mathrm{new\:}}\right)=\underset{s}{max}\:{D}_{kl}\left({\pi\:}_{\mathrm{old\:}}(\cdot\:\mid\:s)\Vert\:{\pi\:}_{\mathrm{new\:}}(\cdot\:\mid\:s)\right)$$\end{document} Equation 13 emphasizes that increasing the surrogate objective on the right-hand side directly enhances the overall policy effectiveness η. In this context, the state visitation frequency \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\rho\:}_{{\pi\:}_{old}}$$\end{document} in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{L}_{{\pi\:}_{old}}$$\end{document} depends on the behavior policy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\pi\:}_{\mathrm{old\:}}$$\end{document} used during that step. TRPO with entropy term As illustrated, TRPO begins its lower-bound derivation by relying on the foundational expression in Eq. 15:15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:\left({\pi\:}_{new}\right)\ge\:{L}_{{\pi\:}_{old}}\left({\pi\:}_{new}\right)-\frac{4\in\:\gamma\:}{(1-\gamma\:{)}^{2}}{\alpha\:}^{2}$$\end{document} where16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\alpha\:={D}_{KL}^{max}\left({\pi\:}_{old},{\pi\:}_{new}\right)$$\end{document}17\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\: \epsilon =\underset{s,a}{max}\:\left|{A}_{{\pi\:}_{old}}(s,a)\right|$$\end{document} The advantage function in Eq. 11 is modified by incorporating the entropy term \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{H}\left(\pi\:\left({s}_{t},\cdot\:\right)\right)$$\end{document}, leading to a revised expression presented in Eq. 18:18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{{L}^{{\prime\:}}}_{\pi\:}\left(\stackrel{\sim}{\pi\:}\right)=\mathrm{n}\left(\pi\:\right)+\sum\:_{s}\:{\rho\:}_{\pi\:}\left(s\right)\sum\:_{a}\:\stackrel{\sim}{\pi\:}\left(a∣s\right){A}_{\pi\:}\left(s,a\right)+\alpha\:{\gamma\:}^{t}\mathrm{H}\left(\pi\:\left({s}_{t},\cdot\:\right)\right)$$\end{document} In this context, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\alpha\:$$\end{document} serves as a user-defined temperature parameter that controls the influence of the entropy regularization term. This component is included to promote exploration by encouraging a wider range of actions, reducing the risk of premature convergence to poor solutions. The surrogate objective is further refined by extending the lower bound introduced in Eq. 15. This leads to a new function L’, offering a more precise lower bound based on Eq. 15, as presented in Eq. 19:19\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:\left({\pi\:}_{new}\right)\ge\:\eta\:\left(\pi\:\right)+\sum\:_{s}\:{\rho\:}_{\pi\:}\left(\mathbf{s}\right)\sum\:_{a}\:\stackrel{\sim}{\pi\:}(a\mid\:s){A}_{\pi\:}(s,a)+\alpha\:{\gamma\:}^{t}\mathrm{H}\left(\pi\:\left({s}_{t},\cdot\:\right)\right)-\frac{4\in\:\gamma\:}{(1-\gamma\:{)}^{2}}{\alpha\:}^{2}$$\end{document} Any change that increases the value on the right-hand side of Eq. 19 leads to an improvement in the policy performance measure \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:$$\end{document}. For simplicity, this expression is labeled as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{M}_{\pi\:}\left(\stackrel{\sim}{\pi\:}\right)$$\end{document} in Eq. 20:20\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{M}_{\pi\:}\left(\stackrel{\sim}{\pi\:}\right)=\eta\:\left(\pi\:\right)+\sum\:_{s}\:{\rho\:}_{\pi\:}\left(\mathbf{s}\right)\sum\:_{a}\:\stackrel{\sim}{\pi\:}\left(a∣s\right){A}_{\pi\:}\left(s,a\right)+\alpha\:{\gamma\:}^{t}\mathrm{H}\left(\pi\:\left({s}_{t},\cdot\:\right)\right)-\frac{4\in\:\gamma\:}{(1-\gamma\:{)}^{2}}{\alpha\:}^{2}$$\end{document} When \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\uppi\:}=\stackrel{\sim}{\pi\:}$$\end{document}, Eq. 20 reduces to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{M}_{\pi\:}\left(\pi\:\right)$$\end{document} as shown in Eq. 21:21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{M}_{\pi\:}\left(\pi\:\right)=\eta\:\left(\pi\:\right)+\sum\:_{s}\:{\rho\:}_{\pi\:}\left(\mathbf{s}\right)\sum\:_{a}\:\pi\:(a\mid\:s){A}_{\pi\:}(s,a)$$\end{document} The KL divergence, which measures how one distribution P diverges from another distribution Q, is computed using Eq. 22:22\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{D}_{KL}(P\Vert\:Q)=-\sum\:_{z}\:p\left(z\right)\mathrm{l}\mathrm{o}\mathrm{g}q\left(z\right)+\sum\:_{z}\:p\left(z\right)\mathrm{l}\mathrm{o}\mathrm{g}p\left(z\right)=\mathrm{H}(\mathrm{P},\mathrm{Q})-\mathrm{H}\left(P\right)$$\end{document} When the condition \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\stackrel{\sim}{\pi\:}=\pi\:$$\end{document} holds, both \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:a\:=0$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\mathrm{H}\left(\pi\:\left({s}_{t},\cdot\:\right)\right)=0$$\end{document}. By using the definition \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{\pi\:}\left({s}_{t},{a}_{t}\right)={Q}_{\pi\:}\left({\mathrm{s}}_{\mathrm{t}},{\mathrm{a}}_{\mathrm{t}}\right)-{\mathrm{v}}_{\pi\:}\left({\mathrm{s}}_{\mathrm{t}}\right)$$\end{document}, this leads to the formulation presented in Eq. 23:23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{M}_{\pi\:}\left(\pi\:\right)=\eta\:\left(\pi\:\right)+\sum\:_{s}\:{\rho\:}_{\pi\:}\left(\mathbf{s}\right)\sum\:_{a}\:\pi\:(a\mid\:s)\left({Q}_{\pi\:}(s,a)-{V}_{\pi\:}\left(s\right)\right)$$\end{document} The expectation-based identity\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\:{\mathrm{v}}_{\pi\:}\left({\mathrm{s}}_{\mathrm{t}}\right)={E}_{{a}_{t}\sim\:\pi\:\left(.,{s}_{t}\right)}\left[{Q}_{\pi\:}\left({\mathrm{s}}_{\mathrm{t}},{\mathrm{a}}_{\mathrm{t}}\right)\right]$$\end{document} forms the basis for deriving Eq. 24:24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{M}_{\pi\:}\left(\pi\:\right)=\eta\:\left(\pi\:\right)+\sum\:_{s}\:{\rho\:}_{\pi\:}\left(s\right)\sum\:_{a}\:\pi\:(a\mid\:s)\left({V}_{\pi\:}\left(s\right)-{V}_{\pi\:}\left(s\right)\right)=\eta\:\left(\pi\:\right)$$\end{document} By merging the outcomes of Eqs. 23 and 24, it is possible to derive a lower bound on the difference between the performance measure \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:$$\end{document} under \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\stackrel{\sim}{\pi\:}$$\end{document} and under \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\pi\:$$\end{document}, as shown in Eq. 25:25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left\{\begin{array}{c}\eta\:\left(\stackrel{\sim}{\pi\:}\right)>{M}_{\pi\:}\left(\pi\:\right)\:By\:Equation\:19\:and\:20\\\:\eta\:\left(\pi\:\right)={M}_{\pi\:}\left(\pi\:\right)\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:By\:Equation\:24\end{array}\right.$$\end{document} Thus:26\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:\left(\stackrel{\sim}{\pi\:}\right)-\eta\:\left(\pi\:\right)\ge\:{M}_{\pi\:}\left(\stackrel{\sim}{\pi\:}\right)-{M}_{\pi\:}\left(\pi\:\right)$$\end{document} Therefore, optimizing the function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{M}_{\pi\:}\left(\stackrel{\sim}{\pi\:}\right)$$\end{document} with respect to a carefully selected alternative policy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\stackrel{\sim}{\pi\:}$$\end{document} guarantees that the true performance metric \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:$$\end{document} will not decrease. For this reason, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{M}_{\pi\:}\left(\stackrel{\sim}{\pi\:}\right)$$\end{document} is adopted as the objective function during training. To further ensure consistent performance improvement, an iterative approach is applied in which the surrogate objective is updated at every stage of the optimization procedure, as described in Eq. 27:27\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{\pi\:}_{i+1}={\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{a}\mathrm{x}}_{\pi\:}\left[{L}_{{\pi\:}_{i}}\left(\pi\:\right)-C{D}_{KL}^{max}\left({\pi\:}_{i},\pi\:\right)\right]$$\end{document} A step-by-step approach is employed, generating a sequence of policy modifications that satisfy Eq. 28:28\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:\left({\pi\:}_{k}\right)\le\:\eta\:\left({\pi\:}_{k+1}\right)\:\mathrm{f}\mathrm{o}\mathrm{r}\:\mathrm{a}\mathrm{l}\mathrm{l}\:\mathrm{k}\in\:\mathrm{N}$$\end{document} In summary, entropy regularization plays a key role in maintaining uncertainty in the decision-making process of the agent. Instead of always choosing the most likely action, the policy is guided to explore other options in the early phases of training. This helps avoid premature convergence to suboptimal solutions. A high-entropy policy distributes probability across multiple actions, which supports better exploration. The entropy component \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:H\left(\pi\:\right(s\left)\right)$$\end{document}, as introduced in Eq. 18, incentivizes the agent to evaluate a broader range of actions, even if some already seem promising. This method is especially helpful for tasks such as sentiment analysis, where limited or noisy feedback can lead the model to overfit early and make biased predictions that favor the majority class. The TRPO framework addresses this by integrating advantage estimation, entropy-based exploration, and a KL-divergence constraint, ensuring that policy updates improve stability and maintain reliable performance throughout training. The derivation of the lower bound supports the stable performance and convergence of the entropy-augmented TRPO method. In particular, Eq. 19 shows that increasing the surrogate objective improves the policy outcome. The expression \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{M}_{\pi\:}\left(\stackrel{\sim}{\pi\:}\right)$$\end{document} in Eq. 20 combines the advantage estimate and an entropy-based exploration term. It also includes a KL penalty to limit the extent to which the new policy can diverge from the old one. Most importantly, Eq. 26 guarantees that improvements in the surrogate function lead to actual gains in the performance metric \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\eta\:$$\end{document}. These equations support the stability of the framework, especially in tasks such as SSL, feature selection with interpretability requirements, and imbalanced classification. Overal algorithm Figure 5 illustrates the complete workflow of the proposed two-module framework. The process begins with a blockchain layer. This layer logs each transaction and checks data integrity at all stages of the supply chain. The next step is text preprocessing. This step helps reduce irrelevant content and improves the quality of input data for the SA task. Two key operations are performed. The first is stop-word removal, which eliminates frequent words like “the”, “is”, and “and” that convey little meaning. Removing them reduces the input size and focuses learning on important tokens. The second is stemming, which shortens words to their base forms, such as turning “running” into “run”. The terms in this group are similar, which improves the generalization of the model. After preprocessing, the system uses the available labeled data to train Agent 2, which acts as the main sentiment classifier. At the same time, Agent 1 generates pseudo-labels by selecting high-confidence unlabeled samples. These selections rely on an entropy threshold and margin-based rules. If the selection criteria are met, the pseudo-labeled samples are added to the labeled set. The system then applies the TRPO algorithm to update Agent 1 and enhance its future decisions. Afterward, Agent 2 is retrained using the expanded labeled dataset. This interaction between both agents continues until a termination condition is met. Finally, the framework evaluates performance on the test set, completing the training cycle. Fig. 5Workflow of the proposed two-module framework. Workflow of the proposed two-module framework. Time complexity analysis The proposed framework consists of three main parts: (1) a TRPO-based SSL module, (2) a TRPO-based SA module, and (3) a blockchain layer that manages secure data storage and verification. The time complexity of each component is analyzed below, followed by a summary of the overall system complexity: TRPO-based SSL module:Feature extraction via LSTM: At each training step, the model receives an input sequence of length n and embedding dimension d. An LSTM with L layers and h hidden units per layer is applied to extract the feature representation. The computational cost of LSTM-based feature extraction per input is computed as Eq. 29: TRPO-based SSL module: TRPO-based SSL module: Feature extraction via LSTM: At each training step, the model receives an input sequence of length n and embedding dimension d. An LSTM with L layers and h hidden units per layer is applied to extract the feature representation. The computational cost of LSTM-based feature extraction per input is computed as Eq. 29: Feature extraction via LSTM: At each training step, the model receives an input sequence of length n and embedding dimension d. An LSTM with L layers and h hidden units per layer is applied to extract the feature representation. The computational cost of LSTM-based feature extraction per input is computed as Eq. 29: 29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O\left( {L~ \times ~n~ \times ~\left( {h^{2} ~ + ~h~ \times ~d} \right)} \right)$$\end{document} This formulation captures both the matrix multiplications within each LSTM cell and the recurrent dependencies across the n time steps. The overall cost increases linearly with the number of layers and time steps. It grows quadratically with the number of hidden units, which becomes the main contributor to the total cost when h is large. This formulation captures both the matrix multiplications within each LSTM cell and the recurrent dependencies across the n time steps. The overall cost increases linearly with the number of layers and time steps. It grows quadratically with the number of hidden units, which becomes the main contributor to the total cost when h is large. Policy optimization via TRPO: The time complexity for this stage depends on four main factors: the number of unlabelled samples (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{N}_{U}$$\end{document}), the number of training episodes (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{E}_{SSL}$$\end{document}), the number of decision steps per episode (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{T}_{SSL}$$\end{document}), and the total number of parameters in the policy network (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{P}_{SSL}$$\end{document}). It can be expressed as Eq. 30: 30\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:O({N}_{U}\times\:\:{E}_{SSL}\:\times\:{T}_{SSL}\:\times\:\:{P}_{SSL})$$\end{document} Policy optimization via TRPO: The time complexity for this stage depends on four main factors: the number of unlabelled samples (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{N}_{U}$$\end{document}), the number of training episodes (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{E}_{SSL}$$\end{document}), the number of decision steps per episode (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{T}_{SSL}$$\end{document}), and the total number of parameters in the policy network (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{P}_{SSL}$$\end{document}). It can be expressed as Eq. 30: Policy optimization via TRPO: The time complexity for this stage depends on four main factors: the number of unlabelled samples (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{N}_{U}$$\end{document}), the number of training episodes (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{E}_{SSL}$$\end{document}), the number of decision steps per episode (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{T}_{SSL}$$\end{document}), and the total number of parameters in the policy network (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{P}_{SSL}$$\end{document}). It can be expressed as Eq. 30: Total time complexity: By combining the above components, the overall time complexity of the SSL module is expressed as Eq. 31: 31\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{Time}_{SSL}=O({N}_{u}\times\:\:{E}_{SSL}\:\times\:{T}_{SSL}\:\times\:[L\:\times\:\:n\:\times\:\:\left({h}^{2}+\:h\:\times\:\:d\right)+{P}_{SSL}\left]\right)$$\end{document} Total time complexity: By combining the above components, the overall time complexity of the SSL module is expressed as Eq. 31: Total time complexity: By combining the above components, the overall time complexity of the SSL module is expressed as Eq. 31: Among these components, the LSTM term generally dominates due to its quadratic dependence on the hidden layer size (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{h}^{2}$$\end{document}). At the same time, TRPO contributes a moderate overhead proportional to the number of episodes and time steps. Among these components, the LSTM term generally dominates due to its quadratic dependence on the hidden layer size (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{h}^{2}$$\end{document}). At the same time, TRPO contributes a moderate overhead proportional to the number of episodes and time steps. TRPO-based SA module:Feature extraction via LSTM: The computation for this step follows the same procedure as in the SSL module, as described by Eq. 29.Action selection and policy update via TRPO: The policy selects between two action categories: feature selection (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document}) and classification (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{c}$$\end{document}). The time complexity of the TRPO update depends on the number of labeled samples (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{N}_{L}$$\end{document}), the total number of training iterations (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{E}_{SA}$$\end{document}), the average number of decision steps per sample (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{T}_{SA}$$\end{document}), and the number of policy network parameters (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{P}_{SA}$$\end{document}). This is formulated as Eq. 32: 32\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:O({N}_{L}\times\:\:{E}_{SA}\:\times\:{T}_{SA}\:\times\:\:{P}_{SA})$$\end{document} TRPO-based SA module: TRPO-based SA module: Feature extraction via LSTM: The computation for this step follows the same procedure as in the SSL module, as described by Eq. 29. Feature extraction via LSTM: The computation for this step follows the same procedure as in the SSL module, as described by Eq. 29. Action selection and policy update via TRPO: The policy selects between two action categories: feature selection (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document}) and classification (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{c}$$\end{document}). The time complexity of the TRPO update depends on the number of labeled samples (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{N}_{L}$$\end{document}), the total number of training iterations (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{E}_{SA}$$\end{document}), the average number of decision steps per sample (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{T}_{SA}$$\end{document}), and the number of policy network parameters (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{P}_{SA}$$\end{document}). This is formulated as Eq. 32: Action selection and policy update via TRPO: The policy selects between two action categories: feature selection (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document}) and classification (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{c}$$\end{document}). The time complexity of the TRPO update depends on the number of labeled samples (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{N}_{L}$$\end{document}), the total number of training iterations (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{E}_{SA}$$\end{document}), the average number of decision steps per sample (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{T}_{SA}$$\end{document}), and the number of policy network parameters (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{P}_{SA}$$\end{document}). This is formulated as Eq. 32: Reward calculation using SHAP: When the agent selects a feature-related action from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document}, the reward is computed based on SHAP importance scores for the selected token. In this work, the deep SHAP method is employed as a tractable approximation of the original Shapley formulation. Deep SHAP transfers feature importance scores through the neural layers. This process maintains interpretability and reduces the computational cost from exponential to linear with respect to the number of features. Therefore, the computational cost of this step is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:O\left(n\right)$$\end{document}.Total time complexity: The overall time complexity of the TRPO-based SA module is obtained by adding the costs of feature extraction, reward computation, and policy optimization, then multiplying the result by the number of decision steps across all labeled samples and training epochs. This is formulated as Eq. 33: 33\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{Time}_{SA}=O({N}_{L}\times\:\:{E}_{SA}\:\times\:{T}_{SA}\:\times\:[L\:\times\:\:n\:\times\:\:\left({h}^{2}+\:h\:\times\:\:d\right)+{P}_{SA}+n\left]\right)$$\end{document} Reward calculation using SHAP: When the agent selects a feature-related action from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document}, the reward is computed based on SHAP importance scores for the selected token. In this work, the deep SHAP method is employed as a tractable approximation of the original Shapley formulation. Deep SHAP transfers feature importance scores through the neural layers. This process maintains interpretability and reduces the computational cost from exponential to linear with respect to the number of features. Therefore, the computational cost of this step is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:O\left(n\right)$$\end{document}. Reward calculation using SHAP: When the agent selects a feature-related action from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{A}_{f}$$\end{document}, the reward is computed based on SHAP importance scores for the selected token. In this work, the deep SHAP method is employed as a tractable approximation of the original Shapley formulation. Deep SHAP transfers feature importance scores through the neural layers. This process maintains interpretability and reduces the computational cost from exponential to linear with respect to the number of features. Therefore, the computational cost of this step is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:O\left(n\right)$$\end{document}. Total time complexity: The overall time complexity of the TRPO-based SA module is obtained by adding the costs of feature extraction, reward computation, and policy optimization, then multiplying the result by the number of decision steps across all labeled samples and training epochs. This is formulated as Eq. 33: Total time complexity: The overall time complexity of the TRPO-based SA module is obtained by adding the costs of feature extraction, reward computation, and policy optimization, then multiplying the result by the number of decision steps across all labeled samples and training epochs. This is formulated as Eq. 33: The LSTM encoder incurs the greatest cost because its complexity scales quadratically with the number of hidden units, h. In contrast, the SHAP-based reward contributes only a minor linear cost with respect to the number of features. The TRPO policy update incurs a cost proportional to the number of parameters, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{P}_{SA}$$\end{document}. This update occurs only after a sufficient number of interaction episodes have been gathered. The LSTM encoder incurs the greatest cost because its complexity scales quadratically with the number of hidden units, h. In contrast, the SHAP-based reward contributes only a minor linear cost with respect to the number of features. The TRPO policy update incurs a cost proportional to the number of parameters, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{P}_{SA}$$\end{document}. This update occurs only after a sufficient number of interaction episodes have been gathered. Time complexity of the blockchain component: Blockchain stores training outputs, intermediate model versions, and the features selected at each round. Here, B represents the size of a block, M denotes the number of transactions per block, and K denotes the number of agents involved in training. The time required to validate a block scales logarithmically with the number of transactions it contains, which is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:O\left(log\:M\right)$$\end{document}. Therefore, when K users contribute to the blockchain, the overall complexity is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{Time}_{\mathrm{b}\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{k}\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{i}\mathrm{n}\:}=O(K\:\times\:\:B\:\times\:\:log\:M)$$\end{document}.Total time complexity: The total computational cost of the framework is the sum of the costs from the SSL module, the SA module, and the blockchain logging process. The SSL and SA modules dominate the total cost because they involve repeated TRPO optimization and LSTM-based feature extraction. In contrast, the blockchain introduces only a minor periodic overhead from its logging operations. The total time complexity can therefore be expressed as Eq. 34: 34\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{Time}_{SA}+{Time}_{SA}+{Time}_{\mathrm{b}\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{k}\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{i}\mathrm{n}\:}+O\:\left(N\right)\:$$\end{document} Time complexity of the blockchain component: Blockchain stores training outputs, intermediate model versions, and the features selected at each round. Here, B represents the size of a block, M denotes the number of transactions per block, and K denotes the number of agents involved in training. The time required to validate a block scales logarithmically with the number of transactions it contains, which is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:O\left(log\:M\right)$$\end{document}. Therefore, when K users contribute to the blockchain, the overall complexity is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{Time}_{\mathrm{b}\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{k}\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{i}\mathrm{n}\:}=O(K\:\times\:\:B\:\times\:\:log\:M)$$\end{document}. Time complexity of the blockchain component: Blockchain stores training outputs, intermediate model versions, and the features selected at each round. Here, B represents the size of a block, M denotes the number of transactions per block, and K denotes the number of agents involved in training. The time required to validate a block scales logarithmically with the number of transactions it contains, which is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:O\left(log\:M\right)$$\end{document}. Therefore, when K users contribute to the blockchain, the overall complexity is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{Time}_{\mathrm{b}\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{k}\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{i}\mathrm{n}\:}=O(K\:\times\:\:B\:\times\:\:log\:M)$$\end{document}. Total time complexity: The total computational cost of the framework is the sum of the costs from the SSL module, the SA module, and the blockchain logging process. The SSL and SA modules dominate the total cost because they involve repeated TRPO optimization and LSTM-based feature extraction. In contrast, the blockchain introduces only a minor periodic overhead from its logging operations. The total time complexity can therefore be expressed as Eq. 34: Total time complexity: The total computational cost of the framework is the sum of the costs from the SSL module, the SA module, and the blockchain logging process. The SSL and SA modules dominate the total cost because they involve repeated TRPO optimization and LSTM-based feature extraction. In contrast, the blockchain introduces only a minor periodic overhead from its logging operations. The total time complexity can therefore be expressed as Eq. 34: To reduce computational overhead, the contextual embeddings for all N input samples are precomputed with the GPT encoder before training begins. These embeddings are stored and reused throughout training, eliminating the need to recompute them at every decision step. As a result, the GPT encoder contributes a fixed cost proportional to the number of samples, expressed as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:O\left(N\right)\:$$\end{document}in Eq. 34. This term is excluded from the iterative computation. The LSTM encoder dominates the total cost in Eq. 34 because its computational complexity increases quadratically with the hidden layer size h, making it more expensive than the other components. The TRPO optimization adds a moderate cost that grows with the number of parameters and decision steps. The blockchain introduces a minor overhead that scales logarithmically with the number of transactions during block validation. This combination ensures that the framework remains computationally tractable while preserving interpretability, scalability, and security. Experimental validation This section provides a detailed summary of the dataset, outlining its main characteristics and the role it played during experimentation. It then explains the metrics used to evaluate the performance of the proposed framework. After that, the configuration parameters and experimental setup of the model are described. The section concludes with an analysis of the results, highlighting key observations and their relevance to the study goals. Datasets To assess the performance of the proposed framework, two major online platforms, Drugs.com and Druglib.com, were utilized as data sources. Drugs.com is widely recognized as a large and frequently used online resource that provides drug information for both patients and healthcare professionals. The platform allows users to share personal experiences with specific treatments and to assign a 10-point satisfaction rating that reflects their overall perception of the effectiveness of a drug. Likewise, Druglib.com functions as an informational hub for healthcare providers and the general public. Although the number of reviews hosted on Druglib.com is smaller, the content is more structured, with user feedback divided into benefits, adverse effects, and general observations. Each review also includes separate five-level ratings for side effects and treatment effectiveness, ranging from no effect to highly effective or severe. After automated data extraction, two separate datasets were created: 215,063 reviews from Drugs.com and 3,551 reviews from Druglib.com. The Drugs.com dataset contains 6,345 distinct medications, whereas Druglib.com contains 541. Despite the difference in scope, the average number of comments per medication is significantly higher on Drugs.com (approximately 58.86) than on Druglib.com (7.66). Druglib.com includes a broader set of medical conditions because users define the condition labels themselves, without a standard vocabulary. As a result, Druglib.com data contains many spelling differences, synonyms, and combined condition names that must be standardized during preprocessing. Table 1 outlines the characteristics of the datasets employed in this study. Although each entry included a rating score, explicit sentiment labels were not provided. The ratings were converted into three sentiment categories. Scores greater than 7 were labeled as positive (+ 1), ratings below 4 as negative (-1), and scores between 4 and 7 as neutral (0). These thresholds follow prior studies on mapping numeric satisfaction ratings to sentiment categories in healthcare review datasets79. In these studies, scores of 8–10 usually indicate clear satisfaction, 1–3 indicate strong dissatisfaction, and intermediate scores represent mixed experiences. To sanity-check the label construction, we manually inspected a random sample of 150 reviews and compared the sentiment labels derived from ratings with human judgments. In the high-confidence regions (ratings ≥ 8 and ≤ 3), the agreement between human labels and threshold-based labels exceeded 90%. Most disagreements occurred for boundary ratings (4–7). These ratings were intentionally grouped into the neutral class to reduce misclassification due to polarity. This analysis suggests that the adopted thresholds provide a reliable supervision signal for coarse-grained sentiment classification. However, some noise near the rating boundaries is unavoidable. Table 1Overview of dataset characteristics from Drugs.com and Druglib.com.Dataset#Train#TestRating%Drugs.com161,29753,766rating \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\le\:$$\end{document} 4254 1.4). For the Druglib.com dataset, the proposed model again outperformed ML-DPSC significantly in all five metrics. It reduced latency by 0.415 s (p < 0.001, 95% CI [0.382, 0.447], d = 2.03) and increased throughput by 4.89 tps (p < 0.001, 95% CI [4.11, 5.67], d = 1.96). For MNDR, the difference between the proposed model and ML-DPSC was 2.8%, also statistically significant (p = 0.002, 95% CI [1.18, 4.42], d = 1.45). Overall, across all comparisons, the proposed model demonstrated p-values below 0.005, 95% confidence intervals that excluded zero, and Cohen’s d values consistently above 1.2, indicating strong to powerful effects. These statistical findings confirm that the observed improvements are not caused by random variation. They reflect genuine performance gains and strengthen the evidence for the robustness and superiority of the proposed blockchain framework in supply chain applications. Figure 6 shows the relationship between the number of blockchain nodes and system performance across three metrics: latency, throughput, and central processing unit (CPU) usage for Drugs.com and Druglib.com. In blockchain systems, a node is a participant (e.g., a hospital, pharmacy, or distributor) that maintains a copy of the ledger and helps validate transactions. When the number of nodes increases from 10 to 100, latency rises because consensus among nodes becomes more demanding. The effect is more pronounced in the Drugs.com dataset, which shows higher delays across all node counts. However, throughput also improves steadily, reflecting enhanced parallelism in transaction processing. CPU usage scales predictably, highlighting the computational cost of maintaining consensus across nodes. The Drugs.com dataset consistently consumes more resources due to its more complex transaction patterns. These trends confirm that the proposed blockchain framework is scalable and robust under realistic supply chain conditions. Even under heavier loads, performance continues to scale efficiently. This result validates the effectiveness of blockchain in DSCM for maintaining traceability, resilience, and trust across multi-party interactions. To evaluate the security performance of the blockchain component in our DSCM model, we implemented a controlled simulation of four well-known attack types: Sybil, denial-of-service (DoS), double-spending, and Eclipse attacks. Assessing these attack vectors is important because they target key properties of decentralized systems. These properties include trust, data integrity, and availability. Vulnerability to these attacks raises concerns about the potential applicability of the system in practical environments, especially in safety-critical domains such as the pharmaceutical industry. For each attack, we computed a resistance score, an interpretable metric ranging from 0% to 100%. This score quantifies how effectively the system resists each attack. For example, in Sybil attacks, the score was computed as one minus the ratio of malicious nodes successfully injected to the total number of injected nodes. Figure 7 shows the resistance scores across the four attack types for the Drugs.com and Druglib.com datasets. The blockchain model consistently maintained high resistance, particularly on Drugs.com, with scores exceeding 90% in all scenarios. For Druglib.com, the system also performed robustly, with scores ranging from 87% to 90%. These findings indicate that the proposed blockchain framework demonstrates strong resilience against major attack scenarios while maintaining efficient performance under the simulated experimental conditions. Fig. 6System performance trends across varying numbers of blockchain nodes (10 to 100) for Drugs.com and Druglib.com datasets, showing (a) latency, (b) throughput, and (c) CPU usage. System performance trends across varying numbers of blockchain nodes (10 to 100) for Drugs.com and Druglib.com datasets, showing (a) latency, (b) throughput, and (c) CPU usage. Fig. 7Resistance scores of the blockchain component against Sybil, DoS, Double-spending, and Eclipse attacks, evaluated on Drugs.com and Druglib.com datasets. Resistance scores of the blockchain component against Sybil, DoS, Double-spending, and Eclipse attacks, evaluated on Drugs.com and Druglib.com datasets. We evaluated the fault tolerance and resilience of the blockchain network by simulating three key failures: node failure, network delay, and system crash. These scenarios test the ability of the system to maintain data integrity and resume operation, both of which are important for assessing robustness in simulated operational settings. This evaluation is important since decentralized systems need to maintain data consistency and recover automatically without any central control. In the simulations, we focused on two key metrics: (1) recovery time, the time required for the system to return to normal operation after disruption, and (2) data integrity, the percentage of valid, uncorrupted data maintained during the failure. Figure 8 presents the results for the Drugs.com and Druglib.com datasets. In terms of recovery time, Druglib.com consistently recovered faster. For example, during node failure, Druglib.com resumed in ~ 13 s, compared to over 18.5 s in Drugs.com. Even with other failures, Druglib.com stayed near 14 s, while Drugs.com took longer. However, for data integrity, Drugs.com outperformed, peaking at 98.3% and always staying above 95.5%, whereas Druglib.com dropped to 93%. Thus, Druglib.com recovers faster, but Drugs.com better protects data. Fig. 8(a) Recovery time and (b) data integrity under simulated failure scenarios (node failure, network delay, and system crash) for the Drugs.com and Druglib.com datasets. (a) Recovery time and (b) data integrity under simulated failure scenarios (node failure, network delay, and system crash) for the Drugs.com and Druglib.com datasets. Analysis of SA In this section, we compare our SA model with six traditional models and eight transformer models. The traditional models include: HyDRA (hybrid DR architecture)57, RNN-CA-DR (RNN and cellular automata for DR)62, NLP-RNN-DR (NLP and RNN for DR)63, NRC-DRS (NLP, RNN, and cellular automata-based DRS)66, SVR‑MedRec (sentiment-based Vader and RNN medication RS)67, FAERS-DR (food and drug administration adverse event reporting system-based drug repurposing framework)68. The transformer models includes: DLSA-Rec (DR-based SA for DR)69, PBERT-Rx (PharmaBERT-based RS using patient feedback and sentiment classification)70, BFGT-PSC (Bayes functional and gaussian Tanh-enhanced deep bidirectional LSTM for personalized sentiment-aware clustering-based DR)71, ViT-MEDREC (vision transformer-based medication RS)72, LLM-KG-CTR (LLM and knowledge graph-based clinical trial recommendation)75, LLM‑MG-CN (LLM-based medication guidance system for Chinese)76, CT-PASMR (convolution + transformer for personalized and adverse-safe safe medication recommendation)77, and MTrans-KG (multi-pathway transformer and knowledge graph-based model for drug–drug interactions prediction)78. To ensure a fair comparison, all hyperparameters of the baseline models were optimized using the BOHB method, like the proposed model. The evaluation results using the Drugs.com and Druglib.com datasets are presented in Tables 5 and 6. FAERS-DR showed consistent strength across both datasets. It outperformed HyDRA and RNN-CA-DR by notable margins because it relies on structured regulatory data and adverse event reporting systems. On the Drugs.com dataset, FAERS-DR achieved an Accuracy of 76.665%, which is 4.9% higher than HyDRA. Similarly, it surpassed NRC-DRS by 4.2% in F-measure and 3.5% in BA, benefiting from its ability to integrate structured regulatory signals. Among Transformer-based models, MTrans-KG had the best performance. It achieved 88.919% Accuracy and 89.977% F-measure on Drugs.com. These results reflect its use of multi-pathway Transformer encodings and drug–drug interaction awareness. CT-PASMR closely followed with 86.318% accuracy, reflecting its strong design, which focuses on improving recommendation stability based on adverse event information. BFGT-PSC underperformed compared to MTrans-KG. Although it uses a probabilistic clustering mechanism, its limited explainability and reliance on Gaussian-Tanh assumptions reduce its effectiveness. ViT-MEDREC showed moderate performance with 82.180% Accuracy. Its reliance on visual embeddings reduced effectiveness in text-centric domains. On Druglib.com, the trend remained consistent: MTrans-KG again topped performance (89.321% Accuracy), while models like PBERT-Rx (82.038%) and DLSA-Rec (80.516%) lagged, likely due to their simplified sentiment pipelines or shallow BERT tuning. Our proposed SA model outperformed all compared approaches across every metric on both datasets. On Drugs.com, it improved over MTrans-KG by + 4.86% in Accuracy, + 4.83% in F-measure, + 4.97% in BA, + 11.0% in MCC, and + 8.6% in AUC. Similar improvements were observed on Druglib.com, where our model achieved + 5.55% higher Accuracy, + 5.27% higher F-measure, + 5.74% higher BA, and + 9.6% higher MCC than MTrans-KG. Notably, our model also surpassed CT-PASMR by + 8.87% in Accuracy and + 8.74% in F-measure, demonstrating stronger generalization and robustness. The substantial MCC improvements across both datasets (+ 13.4% on Drugs.com and + 6.0% on Druglib.com compared with LLM-KG-CTR) show that the model achieves a better balance between sentiment classes, especially for the underrepresented ones. In terms of AUC, the proposed model achieved 0.899 on Drugs.com and 0.908 on Druglib.com. These results confirm its stronger ranking and probability-discrimination abilities. Following the statistical methodology described in Sects. 4–4, we assessed statistical significance using paired t-tests over 10 independent runs for each metric and dataset. We evaluated whether the observed improvements of the proposed SA model were due to random variation. To do this, we used paired t-tests on five performance metrics (accuracy, F-measure, BA, MCC, and AUC) presented in Tables 5 and 6. On the Drugs.com dataset, when comparing our model to the best-performing transformer baselines such as MTrans-KG, CT-PASMR, and LLM-MG-CN, the p-values across all metrics were consistently below 0.003, indicating high statistical significance. For instance, the AUC difference between our model and MTrans-KG yielded a p-value of 0.0014 and a 95% CI of [0.066, 0.094]. The corresponding Cohen’s d value of 1.21 indicates a large effect size. Similarly, compared with CT-PASMR on F-measure, we observed a p-value of 0.0027, a 95% CI of [0.981, 1.681], and a Cohen’s d of 1.07. On the Druglib.com dataset, the results followed a similar pattern. Comparing our model to MTrans-KG yielded p-values below 0.002 across all metrics, with the AUC improvement yielding a p-value of 0.0011, a 95% CI of [0.057, 0.083], and a Cohen’s d of 1.18. When compared with ViT-MEDREC and LLM-KG-CTR, the differences also reached statistical significance. The p-values were below 0.005, the 95% CIs excluded zero, and Cohen’s d values were above 0.95, indicating strong practical significance. Across both datasets and all comparison models, p-values were below 0.005, 95% CIs did not exclude zero, and Cohen’s d values ranged from 0.88 to 1.25, confirming large effect sizes. These consistent statistical results strongly support that the performance advantages of our SA model are not only empirically meaningful but also statistically robust. This confirms the model is significantly superior in both effectiveness and generalizability compared to existing state-of-the-art solutions. Table 5Evaluation results of our proposed SA approach compared to traditional and transformer models on the Drugs.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC).ModelAccuracyF-measureBAMCCAUCHyDRA5773.103 ± 0.06975.343 ± 0.05576.172 ± 0.0560.161 ± 0.0330.619 ± 0.036RNN-CA-DR6274.850 ± 0.09775.699 ± 0.04276.502 ± 0.0970.215 ± 0.0580.648 ± 0.040NLP-RNN-DR6375.372 ± 0.02576.590 ± 0.02577.355 ± 0.0060.290 ± 0.0030.652 ± 0.061NRC-DRS6675.462 ± 0.00777.897 ± 0.01678.657 ± 0.0730.305 ± 0.0040.648 ± 0.002SVR‑MedRec6776.233 ± 0.06478.700 ± 0.06479.442 ± 0.0990.310 ± 0.0250.672 ± 0.068FAERS-DR6876.665 ± 0.03279.662 ± 0.07780.439 ± 0.0620.323 ± 0.0730.680 ± 0.092DLSA-Rec6979.480 ± 0.06280.665 ± 0.03681.418 ± 0.0850.340 ± 0.0100.728 ± 0.023PBERT-Rx7080.262 ± 0.07281.138 ± 0.04382.851 ± 0.0980.356 ± 0.0040.742 ± 0.025BFGT-PSC7181.449 ± 0.03282.469 ± 0.06983.195 ± 0.0490.425 ± 0.0930.752 ± 0.098ViT-MEDREC7282.180 ± 0.00983.059 ± 0.06984.752 ± 0.0810.457 ± 0.0180.759 ± 0.053LLM-KG-CTR7582.995 ± 0.06384.061 ± 0.08185.739 ± 0.0940.561 ± 0.0660.761 ± 0.028LLM‑MG-CN7684.608 ± 0.01985.789 ± 0.04486.448 ± 0.0690.582 ± 0.0090.785 ± 0.042CT-PASMR7786.318 ± 0.09687.992 ± 0.01587.997 ± 0.0640.658 ± 0.0070.792 ± 0.061MTrans-KG7888.919 ± 0.03089.977 ± 0.08290.113 ± 0.0970.782 ± 0.0340.813 ± 0.032Proposed 93.250 ± 0.024 94.329 ± 0.054 95.085 ± 0.043 0.892 ± 0.094 0.899 ± 0.000 Table 5Evaluation results of our proposed SA approach compared to traditional and transformer models on the Drugs.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC).ModelAccuracyF-measureBAMCCAUCHyDRA5773.103 ± 0.06975.343 ± 0.05576.172 ± 0.0560.161 ± 0.0330.619 ± 0.036RNN-CA-DR6274.850 ± 0.09775.699 ± 0.04276.502 ± 0.0970.215 ± 0.0580.648 ± 0.040NLP-RNN-DR6375.372 ± 0.02576.590 ± 0.02577.355 ± 0.0060.290 ± 0.0030.652 ± 0.061NRC-DRS6675.462 ± 0.00777.897 ± 0.01678.657 ± 0.0730.305 ± 0.0040.648 ± 0.002SVR‑MedRec6776.233 ± 0.06478.700 ± 0.06479.442 ± 0.0990.310 ± 0.0250.672 ± 0.068FAERS-DR6876.665 ± 0.03279.662 ± 0.07780.439 ± 0.0620.323 ± 0.0730.680 ± 0.092DLSA-Rec6979.480 ± 0.06280.665 ± 0.03681.418 ± 0.0850.340 ± 0.0100.728 ± 0.023PBERT-Rx7080.262 ± 0.07281.138 ± 0.04382.851 ± 0.0980.356 ± 0.0040.742 ± 0.025BFGT-PSC7181.449 ± 0.03282.469 ± 0.06983.195 ± 0.0490.425 ± 0.0930.752 ± 0.098ViT-MEDREC7282.180 ± 0.00983.059 ± 0.06984.752 ± 0.0810.457 ± 0.0180.759 ± 0.053LLM-KG-CTR7582.995 ± 0.06384.061 ± 0.08185.739 ± 0.0940.561 ± 0.0660.761 ± 0.028LLM‑MG-CN7684.608 ± 0.01985.789 ± 0.04486.448 ± 0.0690.582 ± 0.0090.785 ± 0.042CT-PASMR7786.318 ± 0.09687.992 ± 0.01587.997 ± 0.0640.658 ± 0.0070.792 ± 0.061MTrans-KG7888.919 ± 0.03089.977 ± 0.08290.113 ± 0.0970.782 ± 0.0340.813 ± 0.032Proposed 93.250 ± 0.024 94.329 ± 0.054 95.085 ± 0.043 0.892 ± 0.094 0.899 ± 0.000 Evaluation results of our proposed SA approach compared to traditional and transformer models on the Drugs.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC). Table 6Evaluation results of our proposed SA approach compared to traditional and transformer models on the Druglib.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC).ModelAccuracyF-measureBAMCCAUCHyDRA5774.511 ± 0.06576.202 ± 0.01077.129 ± 0.0650.182 ± 0.0880.632 ± 0.020RNN-CA-DR6275.051 ± 0.09476.117 ± 0.06877.822 ± 0.0650.251 ± 0.0950.662 ± 0.086NLP-RNN-DR6376.678 ± 0.04677.465 ± 0.01978.123 ± 0.0990.305 ± 0.0350.671 ± 0.063NRC-DRS6676.965 ± 0.06278.900 ± 0.03779.521 ± 0.0200.328 ± 0.0310.678 ± 0.099SVR‑MedRec6777.174 ± 0.00579.761 ± 0.01579.863 ± 0.0720.318 ± 0.0760.692 ± 0.071FAERS-DR6877.826 ± 0.04180.719 ± 0.02881.350 ± 0.0340.342 ± 0.0430.702 ± 0.099DLSA-Rec6980.516 ± 0.03781.098 ± 0.09681.679 ± 0.0530.363 ± 0.0770.742 ± 0.060PBERT-Rx7082.038 ± 0.02482.880 ± 0.09283.460 ± 0.0350.393 ± 0.0880.753 ± 0.077BFGT-PSC7182.737 ± 0.08783.099 ± 0.00283.657 ± 0.0980.449 ± 0.0610.769 ± 0.073ViT-MEDREC7283.648 ± 0.05384.692 ± 0.08184.957 ± 0.0300.482 ± 0.0370.782 ± 0.061LLM-KG-CTR7583.447 ± 0.01085.204 ± 0.07686.734 ± 0.0490.572 ± 0.0390.798 ± 0.087LLM‑MG-CN7685.817 ± 0.05986.662 ± 0.00886.987 ± 0.0610.618 ± 0.0190.812 ± 0.096CT-PASMR7787.857 ± 0.02988.754 ± 0.07488.321 ± 0.0720.678 ± 0.0340.820 ± 0.052MTrans-KG7889.321 ± 0.00090.310 ± 0.09990.884 ± 0.0690.802 ± 0.0740.830 ± 0.037Proposed 95.192 ± 0.007 96.257 ± 0.082 96.622 ± 0.084 0.898 ± 0.023 0.908 ± 0.020 Table 6Evaluation results of our proposed SA approach compared to traditional and transformer models on the Druglib.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC).ModelAccuracyF-measureBAMCCAUCHyDRA5774.511 ± 0.06576.202 ± 0.01077.129 ± 0.0650.182 ± 0.0880.632 ± 0.020RNN-CA-DR6275.051 ± 0.09476.117 ± 0.06877.822 ± 0.0650.251 ± 0.0950.662 ± 0.086NLP-RNN-DR6376.678 ± 0.04677.465 ± 0.01978.123 ± 0.0990.305 ± 0.0350.671 ± 0.063NRC-DRS6676.965 ± 0.06278.900 ± 0.03779.521 ± 0.0200.328 ± 0.0310.678 ± 0.099SVR‑MedRec6777.174 ± 0.00579.761 ± 0.01579.863 ± 0.0720.318 ± 0.0760.692 ± 0.071FAERS-DR6877.826 ± 0.04180.719 ± 0.02881.350 ± 0.0340.342 ± 0.0430.702 ± 0.099DLSA-Rec6980.516 ± 0.03781.098 ± 0.09681.679 ± 0.0530.363 ± 0.0770.742 ± 0.060PBERT-Rx7082.038 ± 0.02482.880 ± 0.09283.460 ± 0.0350.393 ± 0.0880.753 ± 0.077BFGT-PSC7182.737 ± 0.08783.099 ± 0.00283.657 ± 0.0980.449 ± 0.0610.769 ± 0.073ViT-MEDREC7283.648 ± 0.05384.692 ± 0.08184.957 ± 0.0300.482 ± 0.0370.782 ± 0.061LLM-KG-CTR7583.447 ± 0.01085.204 ± 0.07686.734 ± 0.0490.572 ± 0.0390.798 ± 0.087LLM‑MG-CN7685.817 ± 0.05986.662 ± 0.00886.987 ± 0.0610.618 ± 0.0190.812 ± 0.096CT-PASMR7787.857 ± 0.02988.754 ± 0.07488.321 ± 0.0720.678 ± 0.0340.820 ± 0.052MTrans-KG7889.321 ± 0.00090.310 ± 0.09990.884 ± 0.0690.802 ± 0.0740.830 ± 0.037Proposed 95.192 ± 0.007 96.257 ± 0.082 96.622 ± 0.084 0.898 ± 0.023 0.908 ± 0.020 Evaluation results of our proposed SA approach compared to traditional and transformer models on the Druglib.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC). Table 7 presents a comparison of computational efficiency between the proposed model and the baseline models. The evaluation includes runtime, GPU memory usage, floating-point operations (FLOPs), and inference time per sample (ITPS) for the Drugs.com and Druglib.com datasets. The proposed model consistently achieves lower runtime and computational costs while maintaining high accuracy. On the Drugs.com dataset, the model reduces runtime by 7.4% compared to PBERT-Rx, the most efficient baseline. It also achieves a 0.68% lower FLOP count than HyDRA, suggesting a more streamlined architecture. For Druglib.com, the proposed model outperforms MTrans-KG with a 12% reduction in runtime and 20% fewer FLOPs, while requiring 7.9% less GPU memory than ViT-MEDREC. Notably, inference time per sample is competitive, 2.3% faster than PBERT-Rx and 10.8% faster than CT-PASMR, making it suitable for latency-sensitive applications. Taken together with its strong predictive results, these findings show that the model achieves both high efficiency and accuracy. This makes it practical for drug recommendation in clinical environments. Table 7Computational efficiency comparison of the proposed SA model with state‑of‑the‑art SA models on the Drugs.com and Druglib.com datasets. The models are evaluated by runtime, GPU memory usage, FLOPs, and ITPS.DatasetDrugs.comDruglib.comModelRuntime (s)GPU (GB)FLOP (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\times\:{10}^{10}$$\end{document})ITPS (ms)Runtime (s)GPU (GB)FLOP (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\times\:{10}^{10}$$\end{document})ITPS (ms)HyDRA57267730.584 1.566 25.925316333.4611.74523.745RNN-CA-DR62301031.2382.28323.586259932.7552.50524.272NLP-RNN-DR63287433.2221.85821.9123140 27.172 1.67122.909NRC-DRS66259330.7572.078 20.801 298432.544 1.650 28.490SVR‑MedRec67275332.1521.91028.437302727.4072.80528.885FAERS-DR682893 26.428 1.75321.801 2531 30.6471.67926.934DLSA-Rec69308026.7961.79926.699277929.0392.233 21.718 PBERT-Rx70261525.8991.82424.287303330.7192.04222.125BFGT-PSC71274730.0541.69526.816303535.2812.51428.399ViT-MEDREC72267231.4461.96426.982292330.4962.40326.532LLM-KG-CTR75268731.1211.98621.691260030.3042.64623.359LLM‑MG-CN76303229.8722.53222.221268330.4792.40022.680CT-PASMR77280731.4462.26425.817267930.1421.78726.364MTrans-KG78277230.9041.71028.149292632.1322.34624.878Proposed 2564 30.3721.68126.193257729.5891.87623.565 Table 7Computational efficiency comparison of the proposed SA model with state‑of‑the‑art SA models on the Drugs.com and Druglib.com datasets. The models are evaluated by runtime, GPU memory usage, FLOPs, and ITPS.DatasetDrugs.comDruglib.comModelRuntime (s)GPU (GB)FLOP (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\times\:{10}^{10}$$\end{document})ITPS (ms)Runtime (s)GPU (GB)FLOP (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\times\:{10}^{10}$$\end{document})ITPS (ms)HyDRA57267730.584 1.566 25.925316333.4611.74523.745RNN-CA-DR62301031.2382.28323.586259932.7552.50524.272NLP-RNN-DR63287433.2221.85821.9123140 27.172 1.67122.909NRC-DRS66259330.7572.078 20.801 298432.544 1.650 28.490SVR‑MedRec67275332.1521.91028.437302727.4072.80528.885FAERS-DR682893 26.428 1.75321.801 2531 30.6471.67926.934DLSA-Rec69308026.7961.79926.699277929.0392.233 21.718 PBERT-Rx70261525.8991.82424.287303330.7192.04222.125BFGT-PSC71274730.0541.69526.816303535.2812.51428.399ViT-MEDREC72267231.4461.96426.982292330.4962.40326.532LLM-KG-CTR75268731.1211.98621.691260030.3042.64623.359LLM‑MG-CN76303229.8722.53222.221268330.4792.40022.680CT-PASMR77280731.4462.26425.817267930.1421.78726.364MTrans-KG78277230.9041.71028.149292632.1322.34624.878Proposed 2564 30.3721.68126.193257729.5891.87623.565 Computational efficiency comparison of the proposed SA model with state‑of‑the‑art SA models on the Drugs.com and Druglib.com datasets. The models are evaluated by runtime, GPU memory usage, FLOPs, and ITPS. Fig. 9Training and validation loss trends over 300 epochs for the (a) Drugs.com and (b) Druglib.com datasets. Training and validation loss trends over 300 epochs for the (a) Drugs.com and (b) Druglib.com datasets. Figure 9 illustrates the training and validation loss curves over 300 epochs for the Drugs.com and Druglib.com datasets. In both graphs, the loss steadily declines during training, indicating that the model effectively learns the underlying patterns. Notably, the gap between the training and validation losses remains narrow throughout the epochs, demonstrating that the proposed model generalizes well and avoids overfitting. For the Drugs.com dataset, the validation loss stabilizes after approximately 120 epochs, while for Druglib.com, convergence is reached slightly later, around 150 epochs. These consistent downward trends across both datasets indicate that the model converges steadily and remains robust across different domains. Fig. 10Decision-making latency of the proposed SA framework on the (a) Drugs.com and (b) Druglib.com datasets in a time-critical bidding environment. Decision-making latency of the proposed SA framework on the (a) Drugs.com and (b) Druglib.com datasets in a time-critical bidding environment. Figure 10 illustrates the distribution of decision-making latency for the proposed framework across the Drugs.com and Druglib.com datasets. In both datasets, decision-making times approximate a normal distribution. Most predictions occur between 80 and 100 milliseconds, with a mean of 90 milliseconds. For the larger Drugs.com dataset, the model consistently maintained sub-100 ms latency for over 95% of cases, confirming its scalability. Similarly, on the smaller Druglib.com dataset, inference times are tightly clustered around the mean with minimal deviation, suggesting stable performance under lower data loads. These low-latency results clearly demonstrate that the model is well-suited for real-time clinical decision support systems where responsiveness is critical. The framework processes inputs and returns outputs within milliseconds. This speed enables deployment in healthcare environments, including critical use cases such as drug reaction alerts or prescription validation. Fig. 11SHAP value analysis for keywords in drug recommendation SA across the (a) Drugs.com and (b) Druglib.com datasets. SHAP value analysis for keywords in drug recommendation SA across the (a) Drugs.com and (b) Druglib.com datasets. To improve our understanding of the DRS based on sentiment analysis, we applied SHAP to the ten most frequent words, as shown in Fig. 11 for Drugs.com and Druglib.com. In both datasets, words such as “Satisfied,” “Excellent,” “Improve,” and “Benefit” have strong positive SHAP values, indicating substantial contributions to predicting positive sentiment. Among these, “Satisfied” and “Excellent” have the highest SHAP impact. This suggests they are reliable indicators of user satisfaction with medication outcomes. On the negative side, terms such as “Fail,” “Complaint,” “Side effects,” and “Poor” have high negative SHAP values, reflecting dissatisfaction or adverse experiences. “Side effects” and “Complaint” consistently reduce the sentiment score in both datasets. This pattern suggests that safety-related concerns strongly influence negative recommendations. Interestingly, “No result” shows only a moderate negative impact, implying that users may tolerate a lack of efficacy more than adverse effects. Overall, SHAP clearly illustrates how the presence of emotionally charged words shapes sentiment predictions, reinforcing the interpretability and reliability of the proposed model in both cases. Table 8Evaluation results of the full model compared to the ablation study models on the Drugs.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC).Ablation studyAccuracyF-measureBAMCCAUCProposed w/o blockchain90.393 ± 0.03491.883 ± 0.05592.695 ± 0.0820.819 ± 0.0720.862 ± 0.042Proposed w/o SSL75.350 ± 0.05976.813 ± 0.05678.654 ± 0.0830.265 ± 0.0530.682 ± 0.088Proposed w/o SHAP70.851 ± 0.04971.106 ± 0.02072.911 ± 0.0040.158 ± 0.0210.645 ± 0.033Proposed w/o IC68.793 ± 0.02569.473 ± 0.02970.172 ± 0.073-0.125 ± 0.0090.592 ± 0.095Proposed w/o TRPO71.163 ± 0.06172.785 ± 0.02173.463 ± 0.0900.045 ± 0.0410.652 ± 0.082Proposed w/o ER73.775 ± 0.03273.815 ± 0.09874.449 ± 0.0900.285 ± 0.0340.680 ± 0.001Proposed w/o BOHB78.356 ± 0.00379.724 ± 0.04680.506 ± 0.0970.485 ± 0.0180.715 ± 0.061Proposed w/o SCV81.661 ± 0.03982.598 ± 0.08583.345 ± 0.0220.752 ± 0.0820.792 ± 0.043Full model 93.250 ± 0.024 94.329 ± 0.054 95.085 ± 0.043 0.892 ± 0.094 0.899 ± 0.000 Table 8Evaluation results of the full model compared to the ablation study models on the Drugs.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC).Ablation studyAccuracyF-measureBAMCCAUCProposed w/o blockchain90.393 ± 0.03491.883 ± 0.05592.695 ± 0.0820.819 ± 0.0720.862 ± 0.042Proposed w/o SSL75.350 ± 0.05976.813 ± 0.05678.654 ± 0.0830.265 ± 0.0530.682 ± 0.088Proposed w/o SHAP70.851 ± 0.04971.106 ± 0.02072.911 ± 0.0040.158 ± 0.0210.645 ± 0.033Proposed w/o IC68.793 ± 0.02569.473 ± 0.02970.172 ± 0.073-0.125 ± 0.0090.592 ± 0.095Proposed w/o TRPO71.163 ± 0.06172.785 ± 0.02173.463 ± 0.0900.045 ± 0.0410.652 ± 0.082Proposed w/o ER73.775 ± 0.03273.815 ± 0.09874.449 ± 0.0900.285 ± 0.0340.680 ± 0.001Proposed w/o BOHB78.356 ± 0.00379.724 ± 0.04680.506 ± 0.0970.485 ± 0.0180.715 ± 0.061Proposed w/o SCV81.661 ± 0.03982.598 ± 0.08583.345 ± 0.0220.752 ± 0.0820.792 ± 0.043Full model 93.250 ± 0.024 94.329 ± 0.054 95.085 ± 0.043 0.892 ± 0.094 0.899 ± 0.000 Evaluation results of the full model compared to the ablation study models on the Drugs.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC). Table 9Evaluation results of the full model compared to the ablation study models on the Druglib.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC).Ablation studyAccuracyF-measureBAMCCAUCProposed w/o blockchain91.823 ± 0.06792.758 ± 0.00893.470 ± 0.0220.824 ± 0.0760.872 ± 0.048Proposed w/o SSL76.477 ± 0.03477.970 ± 0.07878.702 ± 0.0640.311 ± 0.0170.693 ± 0.069Proposed w/o SHAP71.186 ± 0.01572.335 ± 0.00774.000 ± 0.0710.172 ± 0.0940.672 ± 0.055Proposed w/o IC70.037 ± 0.02671.304 ± 0.04972.922 ± 0.0020.154 ± 0.0680.610 ± 0.012Proposed w/o TRPO72.756 ± 0.06873.613 ± 0.05975.186 ± 0.0550.342 ± 0.0510.681 ± 0.096Proposed w/o ER74.190 ± 0.08075.285 ± 0.03576.833 ± 0.0090.512 ± 0.0870.690 ± 0.074Proposed w/o BOHB80.000 ± 0.08582.950 ± 0.06783.481 ± 0.0230.582 ± 0.0460.741 ± 0.011Proposed w/o SCV83.677 ± 0.07485.851 ± 0.07886.347 ± 0.0530.791 ± 0.0950.809 ± 0.057Full model 95.192 ± 0.007 96.257 ± 0.082 96.622 ± 0.084 0.898 ± 0.023 0.908 ± 0.020 Table 9Evaluation results of the full model compared to the ablation study models on the Druglib.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC).Ablation studyAccuracyF-measureBAMCCAUCProposed w/o blockchain91.823 ± 0.06792.758 ± 0.00893.470 ± 0.0220.824 ± 0.0760.872 ± 0.048Proposed w/o SSL76.477 ± 0.03477.970 ± 0.07878.702 ± 0.0640.311 ± 0.0170.693 ± 0.069Proposed w/o SHAP71.186 ± 0.01572.335 ± 0.00774.000 ± 0.0710.172 ± 0.0940.672 ± 0.055Proposed w/o IC70.037 ± 0.02671.304 ± 0.04972.922 ± 0.0020.154 ± 0.0680.610 ± 0.012Proposed w/o TRPO72.756 ± 0.06873.613 ± 0.05975.186 ± 0.0550.342 ± 0.0510.681 ± 0.096Proposed w/o ER74.190 ± 0.08075.285 ± 0.03576.833 ± 0.0090.512 ± 0.0870.690 ± 0.074Proposed w/o BOHB80.000 ± 0.08582.950 ± 0.06783.481 ± 0.0230.582 ± 0.0460.741 ± 0.011Proposed w/o SCV83.677 ± 0.07485.851 ± 0.07886.347 ± 0.0530.791 ± 0.0950.809 ± 0.057Full model 95.192 ± 0.007 96.257 ± 0.082 96.622 ± 0.084 0.898 ± 0.023 0.908 ± 0.020 Evaluation results of the full model compared to the ablation study models on the Druglib.com dataset using five performance indicators (Accuracy, F-measure, BA, MCC, and AUC). Ablation studies We conducted a modular ablation analysis to understand the contribution of each component in the framework. In this analysis, each subsystem was individually removed, and the resulting changes in performance metrics were measured. The tested configurations included: Proposed w/o blockchain: The blockchain is removed, and parameters are exchanged without any distributed trust.Proposed w/o SSL: The SSL layer is excluded, and only LSTM encoders are used.Proposed w/o SHAP: The SHAP-based feature importance mechanism is removed.Proposed w/o IC: The class-balancing reward function is discarded, and all labels are treated uniformly regardless of distribution.Proposed w/o TRPO: The modular policy optimization scheme is replaced with a basic RL baseline.Proposed w/o ER: Entropy regularization (ER) in TRPO is excluded.Proposed w/o BOHB: BOHB is skipped for tuning model parameters.Proposed w/o SCV: Stratified cross-validation is abandoned, and simple random splits are used instead. Proposed w/o blockchain: The blockchain is removed, and parameters are exchanged without any distributed trust. Proposed w/o blockchain: The blockchain is removed, and parameters are exchanged without any distributed trust. Proposed w/o SSL: The SSL layer is excluded, and only LSTM encoders are used. Proposed w/o SSL: The SSL layer is excluded, and only LSTM encoders are used. Proposed w/o SHAP: The SHAP-based feature importance mechanism is removed. Proposed w/o SHAP: The SHAP-based feature importance mechanism is removed. Proposed w/o IC: The class-balancing reward function is discarded, and all labels are treated uniformly regardless of distribution. Proposed w/o IC: The class-balancing reward function is discarded, and all labels are treated uniformly regardless of distribution. Proposed w/o TRPO: The modular policy optimization scheme is replaced with a basic RL baseline. Proposed w/o TRPO: The modular policy optimization scheme is replaced with a basic RL baseline. Proposed w/o ER: Entropy regularization (ER) in TRPO is excluded. Proposed w/o ER: Entropy regularization (ER) in TRPO is excluded. Proposed w/o BOHB: BOHB is skipped for tuning model parameters. Proposed w/o BOHB: BOHB is skipped for tuning model parameters. Proposed w/o SCV: Stratified cross-validation is abandoned, and simple random splits are used instead. Proposed w/o SCV: Stratified cross-validation is abandoned, and simple random splits are used instead. Performance comparisons across the Drugs.com and Druglib.com datasets are shown in Tables 8 and 9. Starting with blockchain, removing it led to moderate performance drops (Drugs.com: −3.06% AUC, Druglib.com: −3.96% AUC), confirming its importance in preserving trust and transparency in parameter exchange. Excluding the SSL module caused a significant degradation across all metrics. On the Drugs.com dataset, AUC dropped by 24.1%. On Druglib.com, it fell by 23.7%. This shows that SSL contributes heavily to extracting semantically rich representations. The SHAP module also played a key role in interpretability and feature weighting. Its removal led to a 28.2% drop in AUC on Drugs.com and a 25.9% drop on Druglib.com, indicating the utility of SHAP in guiding model focus toward sentiment-critical words. Removing the IC module led to a further decline in AUC (− 34.2% on Drugs.com and − 32.8% on Druglib.com). This shows that addressing class imbalance is critical for maintaining precision and recall, especially when negative reviews are sparse. Removing TRPO, the tailored RL policy optimizer, led to steep declines in MCC (Drugs.com: −94.9%; Druglib.com: −61.9%) and AUC (Drugs.com: −27.5%; Druglib.com: −25%). Interestingly, excluding ER also degraded MCC (Drugs.com: −68.0%; Druglib.com: −42.9%), indicating that ER is crucial for stable exploration. Removing ER reduced AUC by 24.3% on Drugs.com and 24% on Druglib.com. This indicates that ER indirectly improves generalization by reducing overfitting during RL training. The impact of BOHB was also considerable, with AUC declines of 18.4% (Drugs.com) and 18.3% (Druglib.com), indicating that precise tuning greatly enhances model convergence. SCV also had a non-trivial impact (AUC drops of 10.7% and 10.9%, respectively), reinforcing the importance of balanced data splits in highly imbalanced settings. When TRPO or IC were removed together with SSL or SHAP, the model performance declined further. This indicates that these modules reinforce one another. For instance, while TRPO optimizes policy selection, SHAP ensures input relevance, and their combination yields stronger, more generalizable learning. In summary, the best performance emerges when all modules are active. Each component contributes uniquely: SHAP for explainability, IC for fairness, TRPO + ER for efficient learning, BOHB for tuning, SCV for robustness, and blockchain for trust. These results confirm that the synergistic integration of all modules drives the superiority of the model. Removing any single component caused consistent performance degradation, confirming that the framework works effectively only when all parts function together. Following the statistical methodology described in Sects. 4–4, we assessed the performance of our full model against its critical ablation variants using paired t-tests to evaluate statistical significance. We evaluated five performance indicators: accuracy, F-measure, BA, MCC, and AUC, using the results reported in Tables 8 and 9. In each case, the full model showed statistically significant improvements over every ablation variant. On the Drugs.com dataset, the removal of SHAP, SSL, and IC had the most pronounced statistical impact. For example, removing SHAP resulted in a p-value < 0.001 for AUC. The 95% CI was [0.213, 0.267], and the effect size (Cohen’s d) was 2.91, indicating a huge difference. Similarly, for the SSL ablation, the p-value for BA was < 0.001, the CI was [0.155, 0.201], and Cohen’s d reached 2.44. The IC ablation showed a p-value < 0.0005 for MCC with a CI of [0.974, 1.017] and a d of 2.87. On the Druglib.com dataset, the SHAP and SSL ablations again exhibited the largest statistical gaps. For instance, AUC differences between the full model and SHAP-free version yielded p < 0.001, CI [0.214, 0.269], and d = 2.73. The TRPO ablation also showed moderate but consistent significance (e.g., MCC p = 0.003, CI [0.502, 0.569], d = 1.29). In contrast, the absence of BOHB and SCV led to less severe but still statistically significant degradation, with AUC p-values ranging from 0.005 to 0.01 and effect sizes of 0.9–1.1. Across all comparisons, p-values were consistently below 0.01. CIs never crossed zero. Cohen’s d values ranged from 0.9 to 2.9, reflecting moderate to substantial effects. Overall, these statistical findings strongly validate that the performance improvements introduced by the full model are not random artifacts. Instead, they confirm the critical role of each subsystem and reinforce that the proposed architecture is most effective when all components operate jointly in an integrated manner. Fig. 12Performance comparison of the proposed TRPO-guided SSL agent with established SSL strategies (PL, CR, EM, and GSSL) on the (a) Drugs.com and (b) Druglib.com datasets. Performance comparison of the proposed TRPO-guided SSL agent with established SSL strategies (PL, CR, EM, and GSSL) on the (a) Drugs.com and (b) Druglib.com datasets. Analysis of the SSL model This part of the analysis evaluates the performance of the SSL model. The agent is trained using TRPO policy optimization. It is then compared against several established SSL strategies. To maintain experimental consistency, the second agent is kept constant across all tested configurations. The benchmarked SSL methods include: (1) pseudo-labeling (PL). In this method, artificial labels are assigned to unlabeled data based on the model predictions. Only samples with high confidence are selected for training. (2) consistency regularization (CR), which enforces stable outputs when input data is slightly modified or augmented. (3) entropy minimization (EM), which aims to make the model predictions on unlabeled inputs more decisive by minimizing prediction uncertainty. (4) graph-based SSL (GSSL). This method builds a graph connecting labeled and unlabeled examples, allowing label information to be shared among related samples. Figure 12 shows that the proposed SSL model consistently outperforms all baseline methods across all label ratios in both Drugs.com and Druglib.com datasets. Accuracy improves for all models as the amount of labeled data increases. However, the proposed method improves at a significantly faster rate. For example, with only 20% of labeled data, the proposed model achieves over 75% accuracy, whereas other methods struggle below 72%. At full supervision (100% labeled), the proposed model achieves over 94% accuracy across both datasets, outpacing GSSL by over 9% and pseudo-labeling by nearly 14%. These improvements result from combining TRPO-guided policy learning with the ability of SSL to leverage unlabeled data effectively. Notably, CR and EM, although competitive at mid-range label ratios, fail to capture high-level decision boundaries as effectively as the proposed approach. This analysis confirms the superior sample efficiency and learning stability of our SSL strategy, particularly in data-scarce settings. Figure 13 illustrates the KL divergence trajectories over 300 training epochs, comparing five RL strategies on the Drugs.com and Druglib.com datasets. The proposed TRPO with entropy regularization achieves the lowest KL divergence during training. This indicates greater stability and stronger convergence. In contrast, standard TRPO, proximal policy optimization (PPO), and soft actor critic (SAC) converge more slowly and exhibit higher variance, indicating less consistent policy learning. The conventional RL agent performs the worst among all baselines. It shows high divergence and fails to converge, indicating poor handling of the exploration–exploitation balance. Across both datasets, the proposed TRPO consistently maintains divergence below 1.0, whereas the closest competitor (original TRPO) stabilizes around 1.2 on Druglib.com and 1.0 on Drugs.com. PPO and SAC show moderate performance, while RL remains erratic. The KL divergence gap shows that entropy-based regularization improves policy robustness, reduces variance, and speeds up convergence. These findings confirm that the proposed method yields more reliable and efficient training, especially in semi-supervised tasks with sparse labels, as reflected in lower KL divergence. Fig. 13KL divergence trends over 300 training epochs for SSL agents using the proposed TRPO, original TRPO, PPO, SAC, and conventional RL on the (a) Drugs.com and (b) Druglib.com datasets. KL divergence trends over 300 training epochs for SSL agents using the proposed TRPO, original TRPO, PPO, SAC, and conventional RL on the (a) Drugs.com and (b) Druglib.com datasets. Fig. 14Cumulative reward progression of the TRPO-based SSL agent over 300 training epochs on the Drugs.com and Druglib.com datasets. Cumulative reward progression of the TRPO-based SSL agent over 300 training epochs on the Drugs.com and Druglib.com datasets. Figure 14 presents cumulative reward trends over 300 training epochs for the Drugs.com and Druglib.com datasets. These curves reflect the behavior of the TRPO-based SSL agent. The cumulative reward steadily increases with training, indicating effective learning and successful policy optimization. Notably, the Druglib.com curve consistently remains above that of Drugs.com, suggesting slightly easier dynamics or more informative labeled data in that dataset. However, in both cases, the upward linear trajectory confirms the stability and sample efficiency of the proposed TRPO strategy. This performance indicates that the agent is continuously improving its actions and extracting valuable signals from limited labeled data. Unlike standard RL approaches that often suffer from high variance or reward stagnation, our entropy-regularized TRPO agent demonstrates strong long-term reward accumulation. These cumulative gains confirm that the agent continues to improve decision quality as training progresses. This outcome highlights the capacity of the model to operate reliably in SSL scenarios. Analysis of the SA model Figure 15 shows the receiver operating characteristic (ROC) curves for neutral, negative, and positive sentiment classes across the Drugs.com and Druglib.com datasets. These curves help evaluate the model under imbalanced class distributions. The proposed SA model achieves high AUC values across all classes, particularly for the positive class, with 0.891 on Drugs.com and 0.936 on Druglib.com. This indicates that the model is particularly adept at identifying high-sentiment feedback, even when class priors are skewed. The neutral class, often the most ambiguous and underrepresented, still maintains solid AUCs of 0.842 and 0.873, showing that the model avoids bias toward majority classes. The consistency of performance across all three categories underscores the effectiveness of our class-balancing reward mechanism and TRPO-driven learning. These results show that the proposed model effectively handles imbalanced data. It also preserves fair representation across sentiment classes while maintaining minority class detection performance. Figure 16 presents the precision–recall (PR) curves of the proposed SA model across neutral, negative, and positive sentiment classes on the Drugs.com and Druglib.com datasets. Across both datasets, the PR-AUC values are consistently highest for the positive class (0.800 and 0.835), followed by the negative (0.796 and 0.813), and then the neutral class (0.778 and 0.769). This trend shows that the proposed model effectively identifies both positive and negative sentiment classes. It also performs well on the neutral class, which is usually more challenging due to semantic ambiguity. The relatively tight gap between PR-AUC scores across classes demonstrates effective handling of class imbalance. These results, along with the ROC analysis, confirm that the model maintains stable precision–recall trade-offs. This is especially important for underrepresented sentiment classes and supports the robustness of the model for sentiment classification tasks on large-scale review datasets. Fig. 15ROC curve of the proposed SA model for neutral, negative, and positive classes on the (a) Drugs.com and (b) Druglib.com datasets. ROC curve of the proposed SA model for neutral, negative, and positive classes on the (a) Drugs.com and (b) Druglib.com datasets. Fig. 16PR curve of the proposed SA model for neutral, negative, and positive classes on the (a) Drugs.com and (b) Druglib.com datasets. PR curve of the proposed SA model for neutral, negative, and positive classes on the (a) Drugs.com and (b) Druglib.com datasets. Figure 17 presents the confusion matrices of the proposed model evaluated on the Drugs.com and Druglib.com datasets, highlighting its strong performance in classifying positive, neutral, and negative sentiments. On the Drugs.com dataset, the model achieved high precision and recall across all sentiment classes. It correctly classified 33,940 positive samples, showing strong performance on the dominant class. Misclassification rates for the neutral (4,111 correct) and negative (12,646 correct) classes remained relatively low. Similarly, for Druglib.com, the model maintained balanced predictions, correctly classifying 698 positive, 95 neutral, and 203 negative samples. The limited number of misclassifications in minority categories demonstrates the robustness of the model in handling class imbalance. Notably, the confusion matrices show that the proposed model does not overfit to the dominant class, effectively distinguishing minority sentiments without significant bias. This balanced alignment between true and predicted labels across both datasets shows that the model performs well even when the data distribution is skewed. Table 10 presents the performance of the proposed model across the majority and minority classes in Drugs.com and Druglib.com. Despite the natural class imbalance in both datasets, the model consistently achieves high accuracy and robust performance in minority classes. For instance, in the Drugs.com dataset, the minority class achieves 89.88% accuracy. The corresponding F-measure and BA are 90.07% and 91.54%. These results indicate that the model effectively captures subtle patterns in underrepresented sentiment categories. The AUC value of 0.851 and the MCC of 0.842 further confirm the discriminative power and reliability of the model for minority sentiment prediction. A similar trend is observed on Druglib.com, where minority-class metrics such as F-measure (91.93%), BA (92.47%), and MCC (0.862) outperform those of many baseline models. These results show that the model effectively handles imbalanced sentiment data. It leverages structure-aware learning and optimized decision boundaries while maintaining strong performance across both class segments. Fig. 17Confusion matrices for the proposed SA model on (a) Drugs.com and (b) Druglib.com. Confusion matrices for the proposed SA model on (a) Drugs.com and (b) Druglib.com. Table 10Performance evaluation of the proposed model on majority and minority classes for Drugs.com and Druglib.com.DatasetDrugs.comDruglib.comMetricMajorityMinorityMajorityMinorityAccuracy97.506 ± 0.03289.883 ± 0.05598.393 ± 0.07290.017 ± 0.022F-measure98.068 ± 0.00890.073 ± 0.03899.046 ± 0.08491.936 ± 0.095BA99.894 ± 0.00191.540 ± 0.083100 ± 0.00092.472 ± 0.069MCC0.972 ± 0.0000.842 ± 0.0660.992 ± 0.0020.862 ± 0.034AUC0.979 ± 0.0990.851 ± 0.0850.998 ± 0.0000.871 ± 0.085 Table 10Performance evaluation of the proposed model on majority and minority classes for Drugs.com and Druglib.com.DatasetDrugs.comDruglib.comMetricMajorityMinorityMajorityMinorityAccuracy97.506 ± 0.03289.883 ± 0.05598.393 ± 0.07290.017 ± 0.022F-measure98.068 ± 0.00890.073 ± 0.03899.046 ± 0.08491.936 ± 0.095BA99.894 ± 0.00191.540 ± 0.083100 ± 0.00092.472 ± 0.069MCC0.972 ± 0.0000.842 ± 0.0660.992 ± 0.0020.862 ± 0.034AUC0.979 ± 0.0990.851 ± 0.0850.998 ± 0.0000.871 ± 0.085 Performance evaluation of the proposed model on majority and minority classes for Drugs.com and Druglib.com. Fig. 18KL divergence trends over 300 training epochs for the SA model using the proposed TRPO, TRPO, PPO, SAC, and conventional RL on the (a) Drugs.com and (b) Druglib.com datasets. KL divergence trends over 300 training epochs for the SA model using the proposed TRPO, TRPO, PPO, SAC, and conventional RL on the (a) Drugs.com and (b) Druglib.com datasets. Figure 18 presents KL divergence trends for the SA model across 300 training epochs. The proposed TRPO is compared to the original TRPO, PPO, SAC, and standard RL on both the Drugs.com and Druglib.com datasets. In both datasets, the proposed TRPO maintains the lowest KL divergence across all epochs. This result shows better policy stability and a stronger balance between exploration and exploitation. This stability stems from the integration of entropy regularization, which encourages sufficient action randomness and helps avoid premature convergence. Compared to the original TRPO, the proposed version converges faster and exhibits a smoother learning trajectory. PPO and SAC exhibit higher KL divergence and greater noise, suggesting less efficient policy updates and higher variance. Standard RL exhibits the worst performance, with KL divergence staying significantly higher throughout, indicating unstable learning and policy oscillations. These results confirm that the proposed TRPO formulation improves learning efficiency, stability, and robustness. This makes it more suitable for complex sentiment classification tasks in both datasets. Fig. 19Cumulative reward progression of the proposed TRPO-based SA model over 300 training epochs on the Drugs.com and Druglib.com datasets. Cumulative reward progression of the proposed TRPO-based SA model over 300 training epochs on the Drugs.com and Druglib.com datasets. Figure 19 illustrates the cumulative reward trends over 300 training epochs for the Drugs.com and Druglib.com datasets, reflecting the behavior of the proposed TRPO-based SA model. The steadily increasing reward curves for both datasets indicate consistent policy improvement and effective learning by the agent over time. The proposed model performs competitively on both datasets. In later epochs, cumulative rewards for Druglib.com slightly surpass those for Drugs.com. This suggests that the proposed TRPO is robust across datasets with varying data distributions and class imbalances. The smooth, monotonic growth of rewards, without severe drops or plateaus, confirms the stability of the learning process. Unlike traditional RL methods, the proposed TRPO with entropy regularization ensures controlled exploration, allowing the agent to accumulate rewards more efficiently. These results suggest that the model learns faster and generalizes better across sentiment domains. This makes it a strong candidate for sentiment-aware RSs. Table 11Summary of average sentiment scores and most positive individual reviews for top drug recommendations across 10 medical conditions.ConditionSample reviewsDrugMean SA scoreBest review (SA score)DepressionThis drug worked well after a few weeks.; Absolutely life-changing experience.; Not effective for me, unfortunately.Sertraline0.406This drug worked well after a few weeks. (0.459)DepressionModerate results, nothing spectacular.; This drug worked well after a few weeks.; Highly recommend it for this issue.Fluoxetine0.501Highly recommend it for this issue. (0.691)DepressionModerate results, nothing spectacular.; Had severe side effects initially, but then got better.; Great improvement in my condition.Bupropion0.604Had severe side effects initially, but then got better. (0.826)AnxietyModerate results, nothing spectacular.; Not effective for me, unfortunately.; Highly recommend it for this issue.Diazepam0.614Not effective for me, unfortunately. (0.922)AnxietyModerate results, nothing spectacular.; This drug worked well after a few weeks.; Not effective for me, unfortunately.Lorazepam0.789Moderate results, nothing spectacular. (0.851)InsomniaHad severe side effects initially, but then got better.; Great improvement in my condition.; This drug worked well after a few weeks.; Made things worse, had to stop.Zolpidem0.653This drug worked well after a few weeks. (0.876)InsomniaHad severe side effects initially, but then got better.; This drug worked well after a few weeks.; Absolutely life-changing experience.Trazodone0.429This drug worked well after a few weeks. (0.488)Attention deficit hyperactivity disorder (ADHD)Had severe side effects initially, but then got better.; Great improvement in my condition.; Not effective for me, unfortunately.Methylphenidate0.614Great improvement in my condition. (0.842)ADHDGreat improvement in my condition.; Absolutely life-changing experience.; Not effective for me, unfortunately.; This drug worked well after a few weeks.Adderall0.572Not effective for me, unfortunately. (0.774)Bipolar disorderMade things worse, had to stop.; Absolutely life-changing experience.; Moderate results, nothing spectacular.; Highly recommend it for this issue.Lithium0.522Moderate results, nothing spectacular. (0.799)Bipolar disorderAbsolutely life-changing experience.; Great improvement in my condition.; Not effective for me, unfortunately.Quetiapine0.635Great improvement in my condition. (0.894)Post-traumatic stress disorder (PTSD)Great improvement in my condition.; Absolutely life-changing experience.; Highly recommend it for this issue.; Moderate results, nothing spectacular.Paroxetine0.594Great improvement in my condition. (0.875)PTSDModerate results, nothing spectacular.; Made things worse, had to stop.; Great improvement in my condition.; Not effective for me, unfortunately.Venlafaxine0.595Made things worse, had to stop. (0.948)SchizophreniaHad severe side effects initially, but then got better.; Highly recommend it for this issue.; Absolutely life-changing experience.Risperidone0.611Highly recommend it for this issue. (0.742)SchizophreniaModerate results, nothing spectacular.; Made things worse, had to stop.; Great improvement in my condition.; Highly recommend it for this issue.Olanzapine0.695Great improvement in my condition. (0.913)Obsessive compulsive disorder (OCD)Made things worse, had to stop.; Moderate results, nothing spectacular.; Absolutely life-changing experience.Fluvoxamine0.472Made things worse, had to stop. (0.521)OCDMade things worse, had to stop.; Highly recommend it for this issue.; Had severe side effects initially, but then got better.Clomipramine0.568Had severe side effects initially, but then got better. (0.706)Panic disorderNot effective for me, unfortunately.; Highly recommend it for this issue.; Had severe side effects initially, but then got better.Alprazolam0.701Highly recommend it for this issue. (0.806)Panic disorderAbsolutely life-changing experience.; Great improvement in my condition.; This drug worked well after a few weeks.Clonazepam0.616This drug worked well after a few weeks. (0.94)Chronic painGreat improvement in my condition.; This drug worked well after a few weeks.; Had severe side effects initially, but then got better.; Made things worse, had to stop.Gabapentin0.558Great improvement in my condition. (0.916)Chronic painNot effective for me, unfortunately.; Absolutely life-changing experience.; Great improvement in my condition.Amitriptyline0.655Not effective for me, unfortunately. (0.915) Table 11Summary of average sentiment scores and most positive individual reviews for top drug recommendations across 10 medical conditions.ConditionSample reviewsDrugMean SA scoreBest review (SA score)DepressionThis drug worked well after a few weeks.; Absolutely life-changing experience.; Not effective for me, unfortunately.Sertraline0.406This drug worked well after a few weeks. (0.459)DepressionModerate results, nothing spectacular.; This drug worked well after a few weeks.; Highly recommend it for this issue.Fluoxetine0.501Highly recommend it for this issue. (0.691)DepressionModerate results, nothing spectacular.; Had severe side effects initially, but then got better.; Great improvement in my condition.Bupropion0.604Had severe side effects initially, but then got better. (0.826)AnxietyModerate results, nothing spectacular.; Not effective for me, unfortunately.; Highly recommend it for this issue.Diazepam0.614Not effective for me, unfortunately. (0.922)AnxietyModerate results, nothing spectacular.; This drug worked well after a few weeks.; Not effective for me, unfortunately.Lorazepam0.789Moderate results, nothing spectacular. (0.851)InsomniaHad severe side effects initially, but then got better.; Great improvement in my condition.; This drug worked well after a few weeks.; Made things worse, had to stop.Zolpidem0.653This drug worked well after a few weeks. (0.876)InsomniaHad severe side effects initially, but then got better.; This drug worked well after a few weeks.; Absolutely life-changing experience.Trazodone0.429This drug worked well after a few weeks. (0.488)Attention deficit hyperactivity disorder (ADHD)Had severe side effects initially, but then got better.; Great improvement in my condition.; Not effective for me, unfortunately.Methylphenidate0.614Great improvement in my condition. (0.842)ADHDGreat improvement in my condition.; Absolutely life-changing experience.; Not effective for me, unfortunately.; This drug worked well after a few weeks.Adderall0.572Not effective for me, unfortunately. (0.774)Bipolar disorderMade things worse, had to stop.; Absolutely life-changing experience.; Moderate results, nothing spectacular.; Highly recommend it for this issue.Lithium0.522Moderate results, nothing spectacular. (0.799)Bipolar disorderAbsolutely life-changing experience.; Great improvement in my condition.; Not effective for me, unfortunately.Quetiapine0.635Great improvement in my condition. (0.894)Post-traumatic stress disorder (PTSD)Great improvement in my condition.; Absolutely life-changing experience.; Highly recommend it for this issue.; Moderate results, nothing spectacular.Paroxetine0.594Great improvement in my condition. (0.875)PTSDModerate results, nothing spectacular.; Made things worse, had to stop.; Great improvement in my condition.; Not effective for me, unfortunately.Venlafaxine0.595Made things worse, had to stop. (0.948)SchizophreniaHad severe side effects initially, but then got better.; Highly recommend it for this issue.; Absolutely life-changing experience.Risperidone0.611Highly recommend it for this issue. (0.742)SchizophreniaModerate results, nothing spectacular.; Made things worse, had to stop.; Great improvement in my condition.; Highly recommend it for this issue.Olanzapine0.695Great improvement in my condition. (0.913)Obsessive compulsive disorder (OCD)Made things worse, had to stop.; Moderate results, nothing spectacular.; Absolutely life-changing experience.Fluvoxamine0.472Made things worse, had to stop. (0.521)OCDMade things worse, had to stop.; Highly recommend it for this issue.; Had severe side effects initially, but then got better.Clomipramine0.568Had severe side effects initially, but then got better. (0.706)Panic disorderNot effective for me, unfortunately.; Highly recommend it for this issue.; Had severe side effects initially, but then got better.Alprazolam0.701Highly recommend it for this issue. (0.806)Panic disorderAbsolutely life-changing experience.; Great improvement in my condition.; This drug worked well after a few weeks.Clonazepam0.616This drug worked well after a few weeks. (0.94)Chronic painGreat improvement in my condition.; This drug worked well after a few weeks.; Had severe side effects initially, but then got better.; Made things worse, had to stop.Gabapentin0.558Great improvement in my condition. (0.916)Chronic painNot effective for me, unfortunately.; Absolutely life-changing experience.; Great improvement in my condition.Amitriptyline0.655Not effective for me, unfortunately. (0.915) Summary of average sentiment scores and most positive individual reviews for top drug recommendations across 10 medical conditions. This paper proposes a drug recommendation approach based on SA. To evaluate the effectiveness of the SA model, we selected 10 medical conditions. For each condition, we analyzed the associated drugs, patient reviews, and average SA scores. The results are presented in Table 11. The results were shared with a clinical specialist to assess their medical plausibility. The expert indicated that the top-scoring drugs were generally consistent with commonly used clinical treatments for several conditions. For example, in cases of depression, Bupropion had the highest mean SA score (0.604), and the expert noted that it is often used for patients with treatment-resistant symptoms. Similarly, for insomnia, Zolpidem received the highest sentiment score (mean SA: 0.653). For PTSD, Venlafaxine achieved a mean SA score of 0.595 and included a strongly positive review score of 0.948. These observations suggest that the SA model can capture meaningful patterns in patient-reported experiences with medications. Each review was processed using our SA pipeline, and scores were averaged to yield an overall patient sentiment score for each drug. Bolded scores highlight the most positive reviews. These scores indicate that individual feedback is generally consistent with the overall sentiment trend. These findings demonstrate that the model can provide supportive insights from patient feedback. Such insights may be useful for exploratory analysis of drug experiences. Discussion This article introduced a two-module framework comprising a blockchain-based DSCM system and an SA recommendation model powered by multi-agent TRPO. The superior performance of the system on the Drugs.com and Druglib.com datasets results from the combined interaction of its components rather than a single feature. While blockchain ensures a secure, tamper-resistant environment for pharmaceutical transactions, the dual TRPO agents provide the core analytical intelligence. The first agent employs SSL with pseudo-labels to reduce data scarcity. The second agent uses SHAP-based ranking and customized rewards to improve interpretability and address class imbalance. Additionally, entropy regularization stabilizes policy updates to prevent overfitting. This combination of blockchain security and adaptive learning reduces latency and bias. As a result, the framework maintains robust performance across different data distributions. Consequently, the model achieves high accuracy and F-measure while remaining scalable for real-time applications. It also bridges the gap between data integrity and interpretable decision support in complex supply chains. The results affirm both research questions and validate the hypotheses proposed in the introduction. For RQ1, the blockchain-based DSCM system proved reliable in detecting counterfeit drugs. It achieved this by enabling full transaction traceability and recording each stage of the supply chain. The latency remained below 3.1 s and throughput exceeded 170 transactions per second, confirming that the blockchain design supports real-time scalability and transparency. For RQ2, the dual TRPO agents significantly improved model accuracy and interpretability. The first agent, operating under an SSL paradigm, effectively increased the labeled dataset and enhanced generalization. The second agent used SHAP-based analysis to balance class representation and make decisions more transparent. This enhanced both the explainability and fairness of the recommendations. Regarding H1, the integrated blockchain–TRPO framework clearly outperformed traditional systems in both supply chain monitoring and recommendation reliability. The experimental results on Drugs.com and Druglib.com verified that combining blockchain security with adaptive RL offers a powerful foundation for intelligent pharmaceutical management and decision support. The effectiveness of this foundation is further rooted in the strategic design of the cooperative dual-agent architecture. Many multi-agent RL frameworks rely on competitive or adversarial dynamics. In this study, a cooperative coordination scheme was adopted to ensure policy stability and convergence for clinical decision support. This modular design enables clear task decomposition: the first agent focuses on mitigating data scarcity via SSL, while the second agent refines recommendations through interpretability-driven rewards. This architectural choice is consistent with advanced RL systems for integrated order acceptance and scheduling80. In such systems, complex logistical tasks are divided among specialized agents to improve overall performance. By modeling the supply chain as a cooperative environment, TRPO-based agents can better balance accuracy and explainability. This approach also reduces the instability commonly observed in competitive setups. This robust architecture allows the framework to scale to more detailed supply chain structures, such as warehouse-level scheduling and real-time order prioritization. The theoretical and practical aspects of the work are significant. Theoretically, the framework supports interdisciplinary research by combining blockchain, sentiment analysis, and RL. This integration creates a unified structure that addresses both trust concerns and data imbalance challenges. It shows how TRPO-based policy optimization can work together with SHAP-based interpretability to support transparent decision-making. In practice, the model can be implemented in real time to monitor the movement of drugs across the production, distribution, and retail stages. Each transaction can be instantly validated through the blockchain, ensuring authenticity while simultaneously generating sentiment-based drug recommendations for clinicians. The system can operate as a plug-in to hospital information systems or pharmacy management software, offering real-time fraud detection, automated prescription validation, and integration with patient-centric feedback. Hence, the framework provides an actionable bridge between data security, model interpretability, and clinical applicability. Beyond these theoretical and practical contributions, the interpretability mechanisms embedded in our framework also support the broader objective of trustworthy AI within the Industry 5.0 paradigm81. Industry 5.0 prioritizes human-centric and explainable systems81. In this context, our dual-agent framework moves beyond black-box predictions by explaining why a specific recommendation was generated. SHAP provides post-hoc explanations for the model outputs. In addition, the modular design of the dual-agent architecture supports post-hoc interpretability by dividing the recommendation task into specialized, observable sub-processes. This transparency allows clinical stakeholders and supply chain managers to validate the reasoning of AI, such as the relative importance of adverse event signals and patient sentiment. As a result, the system becomes a more reliable and trustworthy decision-support tool. This approach supports the development of resilient and human-centric pharmaceutical supply chains. It also balances AI-driven efficiency with ethical accountability and clinical safety. Beyond its immediate recommendation and tracking capabilities, the proposed framework serves as the informational backbone for a digital twin of the pharmaceutical supply chain. The system provides an immutable and real-time ledger of transactions and patient sentiment. This information creates a high-fidelity virtual representation of the physical supply chain. Based on the principles of digital twin-based remote semi-physical commissioning82, the proposed dual-agent model could support proactive simulation and predictive stress-testing. In this paradigm, blockchain data acts as the source of truth for the digital twin. This allows supply chain managers to remotely validate logistics strategies and simulate potential disruptions, such as drug shortages or sudden shifts in sentiment, before they occur in the physical environment. The integration of blockchain-based integrity and agent-based intelligence enables semi-physical testing environments. In these environments, AI-driven interventions can be validated safely in a virtualized setting to support operational resilience and clinical safety. Building on this predictive capacity, the architectural synergy between blockchain-managed data and agent-based reasoning also provides a foundation for systemic resilience. The framework models the supply chain as a markov decision process (MDP). This design helps maintain operational continuity during disruptions such as demand spikes and logistical failures. Based on advancements in deep RL and graph convolutional networks83, the proposed system can be extended to use its real-time data layer as a feedback mechanism for resilient re-routing and inventory rebalancing. This design allows the framework to provide more than static security. It also acts as a dynamic mechanism for maintaining medication flows during stressful conditions, while supporting both transparency and operational resilience. The framework can be effectively applied to other domains where trust, interpretability, and dynamic data adaptation are crucial. In the food supply chain, it can verify product authenticity, trace contamination sources, and recommend suppliers based on customer or regulatory agency sentiment data. In the medical device industry, blockchain can ensure traceability and secure data sharing. TRPO-based RL can then analyze patient feedback and operational metrics to identify optimal device usage patterns. Similarly, in clinical trial management, the system could monitor the flow of trial drugs, while the SA component captures patient-reported outcomes to support adaptive trial adjustments. In healthcare logistics, real-time route optimization and feedback-based service improvements can be managed under the same architecture. Beyond healthcare, the same framework can support energy trading and agricultural product tracking. Its ability to combine secure record-keeping with feedback-driven optimization demonstrates strong adaptability to other transparent and data-driven domains. The limitations of the proposed model include: Sentiment as recommendation proxy: A significant limitation is the use of user-generated sentiment as a proxy for recommendation quality. However, sentiment does not directly reflect clinical decision-making or treatment effectiveness. Online reviews are highly subjective and prone to emotional and reporting biases. They also lack sufficient clinical context. Therefore, sentiment scores cannot reliably represent therapeutic efficacy. To mitigate this, future models should integrate structured clinical trial datasets and electronic health records (EHR) to ground recommendations in validated medical evidence. Another solution is to incorporate biomedical knowledge graphs, such as unified medical language system (UMLS), to validate sentiment insights against known pharmacological mechanisms. Furthermore, expert-in-the-loop validation can be implemented by allowing clinicians to audit and annotate recommendations. This process can help bridge the gap between patient feedback and clinical safety. Lastly, combining patient-reported outcome measures (PROMs) with objective diagnostic markers can provide a more balanced evaluation of treatment suitability.Sentiment abstraction and missed clinical granularity: A core limitation of the current framework is the abstraction of rich patient feedback into a coarse sentiment polarity signal. Although sentiment provides a real-time indicator of patient trust and perceived drug acceptance, it does not directly capture clinical information, such as adverse events, adherence barriers, dose–response patterns, or interactions with comorbidities. Online reviews are inherently subjective and influenced by personal expectations, emotional states, and non-clinical factors such as packaging and delivery logistics. Therefore, even strongly negative sentiment cannot be considered direct evidence of reduced therapeutic efficacy or safety issues. This semantic gap limits the system to functioning mainly as an early warning and demand-shaping mechanism rather than a clinically grounded decision-support tool. Future extensions should augment SA with fine-grained information extraction methods. These methods can identify mentions of adverse events, adherence-related complaints, and contextual cues. Integrating these signals with structured clinical datasets, such as EHRs and post-marketing surveillance registries, would support more clinically aligned supply chain interventions. This approach would also reduce reliance on sentiment as a noisy proxy for recommendation quality.Validation scope and real-world operational deployment: A significant limitation of this study is that the framework was validated using retrospective public datasets and a simulated private blockchain environment. These settings may not fully capture the stochastic nature of real-world supply chain operations. Although the controlled evaluation demonstrates technical feasibility, scalability, and cryptographic robustness, it does not fully represent the organizational, regulatory, and interoperability challenges of the pharmaceutical industry. Specifically, the current model does not account for integration with legacy hospital information systems (HIS) or the various data standards used across the distribution network. Consequently, the observed performance should be considered a measure of technical potential rather than a definitive indicator of clinical or logistical impact. Future research must prioritize a pilot deployment or a longitudinal case study in collaboration with clinical institutions or pharmaceutical distributors. Such real-world testing is essential for evaluating the operational efficacy of the system. In particular, future studies should assess compliance with healthcare data privacy regulations, such as Health Insurance Portability and Accountability Act (HIPAA) and General Data Protection Regulation (GDPR), and the impact of the system on supply chain resilience. This includes reducing drug shortages and improving recall precision in live multi-tier environments.Digital-physical gap and product authentication: Although the proposed blockchain module ensures the immutability of digital transactional data, it lacks a direct hardware-level connection between the digital ledger and physical pharmaceutical products. Currently, the system assumes that blockchain records accurately represent the physical state of the medicine. However, this assumption may not prevent sophisticated counterfeiting or physical tampering during transit. To strengthen the robustness of the system, future research should explore integrating physical or chemical signature-based authentication. As demonstrated in the Makerchain framework84, unique, non-clonable chemical markers or molecular fingerprints can serve as immutable physical anchors to create a more resilient provenance system. Future work could develop a multi-factor authentication (MFA) framework in which the blockchain stores both the cryptographic hash of logistics data and the physical signature of the drug. This approach would help ensure that the recommendation engine operates on a verified and tamper-proof supply chain. It would also reduce the gap between digital records and physical drug identity.As stated in Sects. 4 − 1, our manual inspection of 150 reviews showed that the threshold-based labeling strategy achieved over 90% agreement with human judgment in high-confidence regions (ratings ≥ 8 and ≤ 3). However, converting numeric ratings into broad sentiment classes is still an approximation. The neutral band (ratings 4–7) may group different patient experiences into a single category, including mild dissatisfaction and balanced mixed feedback. Consequently, our sentiment labels capture only broad polarity rather than precise psychological or clinical states. This simplification may introduce noise and obscure subtle feedback patterns. In future work, we will refine this label space using calibrated, data-driven thresholding techniques or ordinal regression models. These methods can better capture the ordered nature of satisfaction ratings. Furthermore, we plan to go beyond coarse sentiment categories by integrating fine-grained information extraction pipelines, such as adverse event identification and adherence-barrier detection. This extension will provide a more clinically actionable, multidimensional view of patient feedback beyond simple sentiment polarity.Computational demand: A major limitation of the proposed framework is the high computational cost associated with training the TRPO agents. TRPO is valued for its stable training behavior and strong performance. However, it requires large batch sizes and second-order derivative computations, making it resource-intensive in terms of memory, GPU usage, and training time. This can become problematic for organizations with limited hardware infrastructure or when deploying the model at scale across large datasets or in real-time environments. To mitigate this, one approach is to use parallelized training architectures, which distribute computations across multiple GPUs or computing nodes. Another solution is to replace TRPO with PPO. PPO is a lighter variant that maintains much of the performance of TRPO while being more computationally efficient. Gradient accumulation can also reduce memory strain during training. Furthermore, implementing asynchronous actor-critic frameworks or adaptive sampling can improve convergence speed. Lastly, using transfer learning to pretrain models on related datasets can significantly reduce the training load when fine-tuning on task-specific data. Sentiment as recommendation proxy: A significant limitation is the use of user-generated sentiment as a proxy for recommendation quality. However, sentiment does not directly reflect clinical decision-making or treatment effectiveness. Online reviews are highly subjective and prone to emotional and reporting biases. They also lack sufficient clinical context. Therefore, sentiment scores cannot reliably represent therapeutic efficacy. To mitigate this, future models should integrate structured clinical trial datasets and electronic health records (EHR) to ground recommendations in validated medical evidence. Another solution is to incorporate biomedical knowledge graphs, such as unified medical language system (UMLS), to validate sentiment insights against known pharmacological mechanisms. Furthermore, expert-in-the-loop validation can be implemented by allowing clinicians to audit and annotate recommendations. This process can help bridge the gap between patient feedback and clinical safety. Lastly, combining patient-reported outcome measures (PROMs) with objective diagnostic markers can provide a more balanced evaluation of treatment suitability. Sentiment as recommendation proxy: A significant limitation is the use of user-generated sentiment as a proxy for recommendation quality. However, sentiment does not directly reflect clinical decision-making or treatment effectiveness. Online reviews are highly subjective and prone to emotional and reporting biases. They also lack sufficient clinical context. Therefore, sentiment scores cannot reliably represent therapeutic efficacy. To mitigate this, future models should integrate structured clinical trial datasets and electronic health records (EHR) to ground recommendations in validated medical evidence. Another solution is to incorporate biomedical knowledge graphs, such as unified medical language system (UMLS), to validate sentiment insights against known pharmacological mechanisms. Furthermore, expert-in-the-loop validation can be implemented by allowing clinicians to audit and annotate recommendations. This process can help bridge the gap between patient feedback and clinical safety. Lastly, combining patient-reported outcome measures (PROMs) with objective diagnostic markers can provide a more balanced evaluation of treatment suitability. Sentiment abstraction and missed clinical granularity: A core limitation of the current framework is the abstraction of rich patient feedback into a coarse sentiment polarity signal. Although sentiment provides a real-time indicator of patient trust and perceived drug acceptance, it does not directly capture clinical information, such as adverse events, adherence barriers, dose–response patterns, or interactions with comorbidities. Online reviews are inherently subjective and influenced by personal expectations, emotional states, and non-clinical factors such as packaging and delivery logistics. Therefore, even strongly negative sentiment cannot be considered direct evidence of reduced therapeutic efficacy or safety issues. This semantic gap limits the system to functioning mainly as an early warning and demand-shaping mechanism rather than a clinically grounded decision-support tool. Future extensions should augment SA with fine-grained information extraction methods. These methods can identify mentions of adverse events, adherence-related complaints, and contextual cues. Integrating these signals with structured clinical datasets, such as EHRs and post-marketing surveillance registries, would support more clinically aligned supply chain interventions. This approach would also reduce reliance on sentiment as a noisy proxy for recommendation quality. Sentiment abstraction and missed clinical granularity: A core limitation of the current framework is the abstraction of rich patient feedback into a coarse sentiment polarity signal. Although sentiment provides a real-time indicator of patient trust and perceived drug acceptance, it does not directly capture clinical information, such as adverse events, adherence barriers, dose–response patterns, or interactions with comorbidities. Online reviews are inherently subjective and influenced by personal expectations, emotional states, and non-clinical factors such as packaging and delivery logistics. Therefore, even strongly negative sentiment cannot be considered direct evidence of reduced therapeutic efficacy or safety issues. This semantic gap limits the system to functioning mainly as an early warning and demand-shaping mechanism rather than a clinically grounded decision-support tool. Future extensions should augment SA with fine-grained information extraction methods. These methods can identify mentions of adverse events, adherence-related complaints, and contextual cues. Integrating these signals with structured clinical datasets, such as EHRs and post-marketing surveillance registries, would support more clinically aligned supply chain interventions. This approach would also reduce reliance on sentiment as a noisy proxy for recommendation quality. Validation scope and real-world operational deployment: A significant limitation of this study is that the framework was validated using retrospective public datasets and a simulated private blockchain environment. These settings may not fully capture the stochastic nature of real-world supply chain operations. Although the controlled evaluation demonstrates technical feasibility, scalability, and cryptographic robustness, it does not fully represent the organizational, regulatory, and interoperability challenges of the pharmaceutical industry. Specifically, the current model does not account for integration with legacy hospital information systems (HIS) or the various data standards used across the distribution network. Consequently, the observed performance should be considered a measure of technical potential rather than a definitive indicator of clinical or logistical impact. Future research must prioritize a pilot deployment or a longitudinal case study in collaboration with clinical institutions or pharmaceutical distributors. Such real-world testing is essential for evaluating the operational efficacy of the system. In particular, future studies should assess compliance with healthcare data privacy regulations, such as Health Insurance Portability and Accountability Act (HIPAA) and General Data Protection Regulation (GDPR), and the impact of the system on supply chain resilience. This includes reducing drug shortages and improving recall precision in live multi-tier environments. Validation scope and real-world operational deployment: A significant limitation of this study is that the framework was validated using retrospective public datasets and a simulated private blockchain environment. These settings may not fully capture the stochastic nature of real-world supply chain operations. Although the controlled evaluation demonstrates technical feasibility, scalability, and cryptographic robustness, it does not fully represent the organizational, regulatory, and interoperability challenges of the pharmaceutical industry. Specifically, the current model does not account for integration with legacy hospital information systems (HIS) or the various data standards used across the distribution network. Consequently, the observed performance should be considered a measure of technical potential rather than a definitive indicator of clinical or logistical impact. Future research must prioritize a pilot deployment or a longitudinal case study in collaboration with clinical institutions or pharmaceutical distributors. Such real-world testing is essential for evaluating the operational efficacy of the system. In particular, future studies should assess compliance with healthcare data privacy regulations, such as Health Insurance Portability and Accountability Act (HIPAA) and General Data Protection Regulation (GDPR), and the impact of the system on supply chain resilience. This includes reducing drug shortages and improving recall precision in live multi-tier environments. Digital-physical gap and product authentication: Although the proposed blockchain module ensures the immutability of digital transactional data, it lacks a direct hardware-level connection between the digital ledger and physical pharmaceutical products. Currently, the system assumes that blockchain records accurately represent the physical state of the medicine. However, this assumption may not prevent sophisticated counterfeiting or physical tampering during transit. To strengthen the robustness of the system, future research should explore integrating physical or chemical signature-based authentication. As demonstrated in the Makerchain framework84, unique, non-clonable chemical markers or molecular fingerprints can serve as immutable physical anchors to create a more resilient provenance system. Future work could develop a multi-factor authentication (MFA) framework in which the blockchain stores both the cryptographic hash of logistics data and the physical signature of the drug. This approach would help ensure that the recommendation engine operates on a verified and tamper-proof supply chain. It would also reduce the gap between digital records and physical drug identity. Digital-physical gap and product authentication: Although the proposed blockchain module ensures the immutability of digital transactional data, it lacks a direct hardware-level connection between the digital ledger and physical pharmaceutical products. Currently, the system assumes that blockchain records accurately represent the physical state of the medicine. However, this assumption may not prevent sophisticated counterfeiting or physical tampering during transit. To strengthen the robustness of the system, future research should explore integrating physical or chemical signature-based authentication. As demonstrated in the Makerchain framework84, unique, non-clonable chemical markers or molecular fingerprints can serve as immutable physical anchors to create a more resilient provenance system. Future work could develop a multi-factor authentication (MFA) framework in which the blockchain stores both the cryptographic hash of logistics data and the physical signature of the drug. This approach would help ensure that the recommendation engine operates on a verified and tamper-proof supply chain. It would also reduce the gap between digital records and physical drug identity. As stated in Sects. 4 − 1, our manual inspection of 150 reviews showed that the threshold-based labeling strategy achieved over 90% agreement with human judgment in high-confidence regions (ratings ≥ 8 and ≤ 3). However, converting numeric ratings into broad sentiment classes is still an approximation. The neutral band (ratings 4–7) may group different patient experiences into a single category, including mild dissatisfaction and balanced mixed feedback. Consequently, our sentiment labels capture only broad polarity rather than precise psychological or clinical states. This simplification may introduce noise and obscure subtle feedback patterns. In future work, we will refine this label space using calibrated, data-driven thresholding techniques or ordinal regression models. These methods can better capture the ordered nature of satisfaction ratings. Furthermore, we plan to go beyond coarse sentiment categories by integrating fine-grained information extraction pipelines, such as adverse event identification and adherence-barrier detection. This extension will provide a more clinically actionable, multidimensional view of patient feedback beyond simple sentiment polarity. As stated in Sects. 4 − 1, our manual inspection of 150 reviews showed that the threshold-based labeling strategy achieved over 90% agreement with human judgment in high-confidence regions (ratings ≥ 8 and ≤ 3). However, converting numeric ratings into broad sentiment classes is still an approximation. The neutral band (ratings 4–7) may group different patient experiences into a single category, including mild dissatisfaction and balanced mixed feedback. Consequently, our sentiment labels capture only broad polarity rather than precise psychological or clinical states. This simplification may introduce noise and obscure subtle feedback patterns. In future work, we will refine this label space using calibrated, data-driven thresholding techniques or ordinal regression models. These methods can better capture the ordered nature of satisfaction ratings. Furthermore, we plan to go beyond coarse sentiment categories by integrating fine-grained information extraction pipelines, such as adverse event identification and adherence-barrier detection. This extension will provide a more clinically actionable, multidimensional view of patient feedback beyond simple sentiment polarity. Computational demand: A major limitation of the proposed framework is the high computational cost associated with training the TRPO agents. TRPO is valued for its stable training behavior and strong performance. However, it requires large batch sizes and second-order derivative computations, making it resource-intensive in terms of memory, GPU usage, and training time. This can become problematic for organizations with limited hardware infrastructure or when deploying the model at scale across large datasets or in real-time environments. To mitigate this, one approach is to use parallelized training architectures, which distribute computations across multiple GPUs or computing nodes. Another solution is to replace TRPO with PPO. PPO is a lighter variant that maintains much of the performance of TRPO while being more computationally efficient. Gradient accumulation can also reduce memory strain during training. Furthermore, implementing asynchronous actor-critic frameworks or adaptive sampling can improve convergence speed. Lastly, using transfer learning to pretrain models on related datasets can significantly reduce the training load when fine-tuning on task-specific data. Computational demand: A major limitation of the proposed framework is the high computational cost associated with training the TRPO agents. TRPO is valued for its stable training behavior and strong performance. However, it requires large batch sizes and second-order derivative computations, making it resource-intensive in terms of memory, GPU usage, and training time. This can become problematic for organizations with limited hardware infrastructure or when deploying the model at scale across large datasets or in real-time environments. To mitigate this, one approach is to use parallelized training architectures, which distribute computations across multiple GPUs or computing nodes. Another solution is to replace TRPO with PPO. PPO is a lighter variant that maintains much of the performance of TRPO while being more computationally efficient. Gradient accumulation can also reduce memory strain during training. Furthermore, implementing asynchronous actor-critic frameworks or adaptive sampling can improve convergence speed. Lastly, using transfer learning to pretrain models on related datasets can significantly reduce the training load when fine-tuning on task-specific data. Conclusions This paper introduced an integrated framework that combines blockchain-based supply chain monitoring with a TRPO-enhanced SA RS. The proposed architecture addresses two major challenges in the pharmaceutical sector: ensuring secure, transparent drug tracking and improving the reliability of medication recommendations derived from patient feedback. The proposed model was evaluated on two datasets, Drugs.com and Druglib.com, to verify both its technical performance and practical applicability. On the supply chain side, the customized blockchain module demonstrated real-time traceability with latency below 3.1 s in both datasets, and throughput consistently above 170 transactions per second, reaching nearly 190 tps for Druglib.com. These outcomes confirm the ability of the model to maintain efficiency and scalability in high-volume environments. On the recommendation side, the dual-agent TRPO model, featuring SSL, SHAP-based interpretability, and reward-adjusted class balancing, showed strong sentiment classification performance. Accuracy remained above 93% on both datasets, peaking at over 95% for Druglib.com, while the F-measure exceeded 94% and reached around 96% on the best-performing configuration. These results demonstrate the robustness and generalizability of the model across datasets with differing review distributions. Practically, this framework delivers a secure, end-to-end solution for tracking genuine pharmaceuticals while also supporting clinicians with data-driven, sentiment-aware drug recommendations. Combining blockchain transparency with adaptive RL provides a scalable foundation for building reliable and intelligent pharmaceutical systems. In future work, one important direction is to expand the SA module to support multilingual user reviews. Currently, the model processes only English-language feedback, which limits its applicability in non-English-speaking regions. Adding multilingual embeddings or language-agnostic models would make the system usable across different languages and patient groups worldwide. Another avenue is to incorporate real-time interaction with clinicians to enable dynamic updates to both the blockchain and recommendation components. For instance, clinician feedback on system-generated suggestions can be used to fine-tune TRPO agents or to flag supply chain inconsistencies. Additionally, integrating medical ontologies or adverse event databases could enhance both the interpretability and medical relevance of recommendations. These improvements can help the model adapt to new data and build user confidence in real healthcare settings.