Improving citizen-government interactions with generative artificial intelligence: Novel human-computer interaction strategies for policy understanding through large language models Authors: Lixin Yun, Sheng Yun, Haoran Xue, Fredrick Romanus Ishengoma, Jinran Wu, Jinran Wu Published: 2024 ABSTRACT Effective communication of government policies to citizens is crucial for transparency and engagement, yet challenges such as accessibility, complexity, and resource constraints obstruct this process. In the digital transformation and Generative AI era, integrating Generative AI and artificial intelligence technologies into public administration has significantly enhanced government governance, promoting dynamic interaction between public authorities and citizens. This paper proposes a system leveraging the Retrieval-Augmented Generation (RAG) technology combined with Large Language Models (LLMs) to improve policy communication. Addressing challenges of accessibility, complexity, and engagement in traditional dissemination methods, our system uses LLMs and a sophisticated retrieval mechanism to generate accurate, comprehensible responses to citizen queries about policies. This novel integration of RAG and LLMs for policy communication represents a significant advancement over traditional methods, offering unprecedented accuracy and accessibility. We experimented with our system with a diverse dataset of policy documents from both Chinese and US regional governments, comprising over 200 documents across various policy topics. Our system demonstrated high accuracy, averaging 85.58% for Chinese and 90.67% for US policies. Evaluation metrics included accuracy, comprehensibility, and public engagement, measured against expert human responses and baseline comparisons. The system effectively boosted public engagement, with case studies highlighting its impact on transparency and citizen interaction. These results indicate the system’s efficacy in making policy information more accessible and understandable, thus enhancing public engagement. This innovative approach aims to build a more informed and participatory democratic process by improving communication between governments and citizens. 1 Introduction Communicating government policies to citizens is a fundamental aspect of public administration and governance, serving as a bridge between the decision-makers and the populace whose lives are directly impacted by these policies and plans [1]. Historically, this communication has taken various forms, ranging from official documents, press releases, and public announcements to more interactive platforms such as public forums, consultations, and, more recently, digital and social media channels [2]. These methods aim to ensure transparency, promote public awareness, and stimulate citizen engagement with governmental processes. However, despite these efforts, numerous challenges persist, including the accessibility of information, the complexity of policy language, and the general public’s understanding of governmental procedures [3]. For instance, detailed policy documents and press releases may ensure broad dissemination but often fail to engage citizens effectively due to their complexity and the bureaucratic language used, leaving a significant portion of the population disengaged or misinformed about policies that have a substantial impact on their lives [4, 5]. Disengagement can cause the citizens to be indifferent to politics and society, and misinformation may lead to trust issues in government policies which can be dangerous in a crisis. The challenges are further compounded by financial and logistical constraints faced by government departments dedicated to public communication. The significant costs associated with both traditional and digital dissemination methods, combined with the slow response times to citizen inquiries and insufficient manpower [3, 5, 6], especially in financially constrained areas, aggravates the situation. Moreover, the often limited understanding of complex policies by the staff tasked with their communication can lead to misinformation, further eroding public trust in government institutions. These issues show a critical gap in the effectiveness of existing communication strategies, necessitating innovative solutions to bridge the information divide between governments and their citizens. Artificial Intelligence (AI) and Large Language Models (LLMs) have emerged as powerful tools capable of transforming various sectors, including public administration and governance [7]. AI refers to machines designed to mimic human intelligence, performing tasks such as generalizing complex texts, problem-solving, and decision-making, while LLMs are a subset of AI, focusing on understanding and generating human language. These technologies have demonstrated remarkable capabilities, including the ability to process and analyze vast amounts of data, understand complex patterns, and generate coherent, informative content [8–10]. Their application spans numerous fields, offering potential solutions to longstanding challenges by enhancing efficiency, accuracy, and accessibility [7, 9]. In public administration, the integration of AI and LLMs has the potential to revolutionize the communication of government policies, making them more accessible and comprehensible to the general public. This convergence of AI and LLMs in public administration not only smooths information processing but also revolutionizes how government policies are communicated to and understood by the public [11]. Their capacity to digest and distill vast amounts of complex documents into concise, understandable summaries holds the potential to democratize access to policy information, making it significantly more accessible to the general public [8]. Furthermore, LLMs facilitate real-time, interactive communication, allowing citizens to receive personalized responses to their inquiries. This shift from traditional, unidirectional communication methods towards a more interactive, responsive, and citizen-focused approach promises to enhance public engagement with governmental processes and ensure that policy information is not just available but also comprehensible to the average citizen. To capitalize on the capabilities of AI and LLMs in enhancing the communication of government policies, various innovative approaches have been explored. The development of chatbots and virtual assistants, for instance, provides real-time responses to citizen inquiries, embodying the move towards instant, accessible communication [12]. AI-driven platforms that simplify and summarize policy documents into digestible formats are becoming increasingly prevalent, along with the integration of LLMs into government websites and social media channels to boost accessibility and engagement [13, 14]. Furthermore, initiatives like AI-curated newsfeeds on policy developments and virtual town halls moderated by AI for more inclusive public discussions are being tested [14, 15]. These solutions aim to leverage the power of AI directly to transcend the limitations of conventional communication methods, ensuring policy information is not only more accessible but also more comprehensible. The direct application of LLMs in communicating government policies is fraught with challenges [16]. The complexity and sensitivity of governmental information demand a degree of accuracy, relevance, and security that raw LLM outputs may not consistently provide. The potential for misinformation, biases inherent in the training data, and the ethical implications of AI-generated content pose significant hurdles. These challenges underscore the necessity for a better approach that can harness the strengths of LLMs while addressing their limitations. To enhance AI’s potential in government governance, we propose a system that combines Retrieval-Augmented Generation (RAG) with LLMs. This creates an interactive policy interpreter that combines LLMs’ generative capabilities with the precision of retrieval-based systems for more accurate, reliable outputs [17]. This approach reduces bias, unpredictability, and the need for costly customizations, offering a cost-effective solution to improve governance. We tested our application with policy documents in Simplified Chinese and English, showing high reliability, accuracy, and efficiency. This method enhances public service responsiveness and incorporates transparency, accountability, and inclusivity, promising for future government technology research and application. In this paper, we provide a review of related work in Section 2, highlighting the existing approaches and their limitations. In section 3, we describe the proposed system in detail, including the architecture and implementation of the RAG technology combined with LLMs. We discuss the dataset used for testing and the preprocessing steps involved in Section 4. Section 5 presents the experimental setup, results, and performance analysis, including a comparative analysis with previous studies. And in section 6, we discuss the challenges, limitations, and future directions for this research. Finally, Section 7 concludes the paper with a summary of findings and implications for policy communication. 2 Related work Recent research has highlighted the potential of artificial intelligence (AI) and large language models (LLMs) in enhancing government-citizen communication. Mamalis et al. [11] demonstrated how ChatGPT could revolutionize open government data portals, improving the accessibility and usability of statistical information. Similarly, Androutsopoulou et al. [12] explored AI-guided chatbots for transforming citizen-government communication, emphasizing their potential to provide personalized and efficient public services. Recent studies have also examined the adoption and implementation of AI chatbots in public organizations. Chen et al. [18] investigated the determinants facilitating or impeding chatbot adoption and implementation across U.S. state agencies, identifying factors like ease of use, leadership, and external shocks as key drivers. Wang et al. [19] explored chatbot adoption in Chinese local governments, finding that pressure factors and readiness factors play different roles in the initial adoption versus post-adoption performance stages. Cortés-Cediel et al. [20] conducted a comprehensive analysis of e-government chatbots, identifying trends and challenges in both research literature and deployed applications, and proposing novel chatbots for exploring open government data and citizen participation content. These studies, however, do not utilize Retrieval-Augmented Generation (RAG) or guarantee the queries from the clients will be answered in the way the authorities expected, potentially limiting their accuracy and adaptability in handling complex, evolving policy information. The application of RAG in policy communication represents a novel approach. Lewis et al. [17] introduced RAG for knowledge-intensive NLP tasks, showing its effectiveness in combining the strengths of retrieval and generation models. This technique has since been adapted for various domains, including public administration. Challenges in implementing AI for government communication have also been identified. Hadi et al. [16] surveyed large language models, highlighting limitations such as potential biases and the need for continuous updates to maintain accuracy, which implies the necessity of integrating the RAG into the system. Kasneci et al. [7] discussed both opportunities and challenges of using LLMs in educational contexts, which can be extended to policy communication. Our proposed approach builds upon these advancements by integrating RAG technology with Langchain, specifically tailored for policy communication. This method addresses the identified challenges by ensuring up-to-date, accurate information retrieval while leveraging the natural language generation capabilities of LLMs. 3 Communicating government policies to citizens Overview of current communication strategies Understanding the intricacies of government policies and plans remains a critical challenge for citizens across the globe. In many countries [2], including China, the United States, the United Kingdom, Canada, and Australia, governments have traditionally relied on a combination of public announcements, official documents, websites, and direct citizen engagement through public forums and consultations to disseminate this information, as shown in Fig 1. However, the effectiveness of these methods varies significantly, influenced by factors such as accessibility, the complexity of information presented, and the public’s general awareness and understanding of governmental procedures. For instance, China’s approach often involves detailed policy documents and press releases on government websites, complemented by media coverage. While this ensures a broad dissemination of information, the complexity of language and the volume of data can overwhelm citizens, hindering their understanding. A more personalized approach that enables the citizen to know what the policies that may largely impact their life is still not very effective and most of the citizens are either not eager or not able to acquire the policies that matter the most. Current major media of communicating government policies to citizens. Financial and logistical constraints Like many services in the government, the financial and logistical aspects of these dissemination methods present substantial burdens [21]. Departments dedicated to public communication and engagement often face budgetary constraints, limiting their ability to reach out effectively to all citizens [21]. In the US and Canada, for instance, extensive use of digital platforms and social media has helped in reducing some costs but has not eliminated the need for traditional, more resource-intensive methods such as public meetings and printed materials. These strategies entail significant extra expenses, from the production of materials to the organization of events, straining the already tight budgets of government departments. Inherent challenges in policy communication The challenges are exacerbated when considering the responsiveness of government agencies to citizen inquiries [5]. Slow responses are a common issue, partly due to the sheer volume of queries and the bureaucratic processes involved in addressing them. This delay in communication can lead to frustration and dissatisfaction among the public, eroding trust in government institutions. In many cases, the bureaucratic nature of government operations can delay the dissemination of information, resulting in outdated or irrelevant information being provided to the public. This is particularly problematic in emergencies, such as during health crises or natural disasters, where timely and accurate information is critical [22]. Moreover, the cost of manpower is a critical concern. The costs associated with these dissemination activities are substantial and encompass not only the direct expenses of creating and distributing the information but also the operational costs of the departments responsible for these tasks. In countries like the United States and Canada, significant investments are made in digital infrastructure to support the online presence of government agencies, including websites and social media engagement [23, 24]. Similarly, the United Kingdom and Australia have dedicated teams within government departments tasked with public relations and communication, ensuring that policy explanations are clear, accessible, and reach a wide audience. However, these efforts represent a considerable portion of government budgets, with ongoing debates about their cost-effectiveness and impact on public awareness and understanding of government actions. This problem is particularly acute in rural areas of countries like China, where limited financial resources and the logistical challenges of reaching a dispersed population hinder the government’s ability to hire sufficient personnel for effective communication. The result is a gap in the timely provision of information to citizens, who often seek clarity on policies and regulations that directly impact their lives. This illustrates a scenario where the government struggles to allocate enough human resources to engage with the public effectively. Hiring individuals to bridge the information gap is not always viable, leading to an overreliance on less interactive digital communication platforms. This approach, while cost-effective, fails to address the nuances of personal interaction and can leave the more technologically disadvantaged populations further behind. Another layer of complexity is added by the staff’s limited understanding of policies and plans. Even when personnel is available, their ability to interpret complex information in an accessible manner is not guaranteed, and citizens who would like to inquire about specific policies might be misled by the government staff’s lack of knowledge of the new policies. The intricacies of policy and legislative language, coupled with the often intricate implications of these documents, demand a high level of expertise. Unfortunately, the requisite training and knowledge development among staff are areas that frequently suffer due to financial and time constraints [25]. The misunderstanding caused by the ambiguity or mistake made by the staff will not only compromise the effectiveness of the government policy deliveries but also harm the public’s trust in the government. This issue is not unique to any single country; it reflects a global challenge within government communication efforts. Case studies from across the mentioned countries show attempts to mitigate these challenges through innovative solutions. For example, some local governments in China have begun to leverage social media and messaging apps to provide more direct and interactive communication channels. These efforts, however, are but a drop in the ocean of necessary reform. The overarching issue remains: without comprehensive strategies that address the financial, logistical, and educational barriers to effective government communication, citizens remain at a disadvantage in understanding and engaging with the policies that shape their lives. Similarly, in the US, the rollout of healthcare policies has often been met with public confusion and skepticism, in part due to communication missteps and the complexity of the information being conveyed. Overall, The task of explaining government policies and plans to citizens is fraught with significant challenges. From the financial costs associated with traditional and digital dissemination methods to the slow response times and the insufficient manpower, especially in financially constrained areas, these obstacles are compounded by the staff’s often limited understanding of complex policies. As governments seek to navigate these hurdles, the search for more efficient, accessible, and cost-effective communication methods continues, highlighting the critical need for innovation in public administration and engagement strategies. 4 Our approach to enhancing government-citizen communication Our system using RAG and LLMs Our approach to improving the communication of government policies to citizens is centered around the innovative application of Retrieval-Augmented Generation (RAG) technology integrated with Langchain, as shown in Fig 2. We employ GPT-3.5-turbo for question answering, with documents converted to manageable formats and segmented for efficient retrieval. The RAG technology enables the system to dynamically retrieve relevant documents from a comprehensive knowledge base and generate responses that are both accurate and contextually relevant. By leveraging the capabilities of large language models (LLMs) and combining them with a sophisticated retrieval mechanism, our system is designed to deliver accurate, up-to-date, and comprehensible information on government policies directly to the public. Our system not only generates responses based on a vast corpus of governmental documents but also ensures that the information provided is specifically tailored to the user’s query, making complex policy information accessible and understandable to the layperson. The key innovation here is the system’s ability to dynamically retrieve relevant documents and synthesize this information into clear, concise responses using LLM’s ability to generalize the ideas only based on the retrieved information, We believe this can bridge the gap between the intricate world of policy-making and the everyday lives of citizens. Retrieval augmented generation with llm for communication of government policy to citizen. To further enhance the efficacy of our approach, we employ Langchain to facilitate the seamless integration of the retrieval and generation processes. Langchain’s role is important, as it acts as the backbone of our system, enabling the dynamic updating of the knowledge base with the latest policies and regulations. This ensures that our system remains current and relevant, capable of addressing the public’s questions with the most recent information, and it can help the retrieval component to identify the most relevant information regarding the query. We then can ensure that the responses are not just accurate but also engaging and up-to-date for the general public. Implementation in citizen communication To maximize the impact of our approach, it is crucial to ensure broad and easy access for all citizens. This can be achieved by integrating our RAG-based system within existing government digital platforms, such as official websites, mobile apps chat-bots, and social media channels. By embedding the system in platforms that citizens already use to seek information, we can significantly enhance the accessibility of complex policy information. Furthermore, the implementation should be spearheaded by departments that are directly involved in public communication and citizen engagement, such as the Department of Public Information or equivalent entities within local and national government structures. These departments, with their expertise in communication strategies and their existing infrastructure, are best positioned to manage the integration of the RAG system, ensuring that it is effectively utilized to improve the dialogue between the government and its citizens. To illustrate the real-world applicability of our system, consider a scenario where a government agency implements our RAG-based system to communicate a new environmental policy. The system is integrated into the agency’s website and mobile app, providing citizens with instant access to policy details. A citizen queries about specific regulations regarding waste management, and the system retrieves relevant documents, synthesizing a clear and concise response. This approach not only enhances transparency but also ensures that citizens receive accurate information, thereby improving public understanding and compliance with the policy. In addition to digital integration, physical access points such as public libraries, community centres, and government offices can be equipped with terminals or kiosks where citizens without internet access at home can interact with the system. These digital and physical access points ensure that the benefits of the RAG system are extended to the widest possible audience, including those in rural or underserved areas. Training for staff members at these access points will also be necessary to assist citizens in navigating the system and understanding the responses generated, thereby enhancing the overall effectiveness of the communication strategy. Building the knowledge base for RAG The foundation of our RAG system is its knowledge base, which must be comprehensive, authoritative, and up-to-date to ensure the accuracy and relevance of the information provided to citizens. To construct this knowledge base, we will compile a wide array of government documents, policies, regulations, and related materials. This compilation process involves close collaboration with various government departments and agencies to secure access to current and historical documents. Additionally, our knowledge base must be incorporated with a process that can have continuous updates, with a mechanism in place for the timely incorporation of new policies, amendments, and relevant announcements into the knowledge base. This dynamic updating process is critical for maintaining the system’s effectiveness in providing accurate and current information. Beyond the initial compilation, the management of the knowledge base requires the application of advanced categorization and indexing techniques. These techniques ensure that the retrieval component of the RAG system can efficiently locate and extract the most relevant documents in response to a user’s query. The use of metadata, such as document type, publication date, and policy area, will facilitate this process, enabling more precise matching and thereby enhancing the overall quality of the information retrieved. This meticulous organization of the knowledge base is essential for the system’s ability to deliver targeted and coherent responses, reflecting the complexities and nuances of government policies in a manner that is accessible to the general public. Potential improvement on government-citizen communication Our approach represents a significant leap forward in the way government policies are communicated to citizens. By leveraging the RAG system integrated with Langchain, we can transform the traditionally complex and opaque process of policy dissemination into a user-friendly, engaging, and interactive experience. This system not only makes policy information more accessible but also more understandable, empowering citizens to gain insights into government actions that directly affect their lives. The real-time, personalized responses provided by the system ensure that citizens can receive specific information relevant to their needs and concerns, thereby fostering a more informed and engaged public. Ultimately, our approach enhances transparency, builds trust, and encourages greater citizen participation in governance processes, contributing to the development of a more responsive and accountable government. 5 Details in the RAG component in our approach Overview of our RAG component At the core of our system is Retrieval-Augmented Generation (RAG) utilizing Langchain technology. RAG, at its essence, is designed to harness the vast knowledge embedded within large language models (LLMs) while integrating a retrieval mechanism that pulls in specific, relevant information from a designated knowledge base (database or corpus of documents). This integration ensures that the responses generated are not only linguistically coherent but also deeply grounded in the most current and accurate data available. Our adoption of RAG through Langchain is tailored specifically to address the unique challenges faced in public administration, particularly in the dissemination of complex policy information. Architecture of our RAG component The architecture of our Retrieval-Augmented Generation (RAG) model is designed to optimize the communication of complex government policies to the public with unparalleled precision and accessibility. At the heart of this architecture lies the document retriever component, a sophisticated algorithm designed to navigate the vast expanse of government documentation using a semantic search based on the question that it has been asked. This component leverages advanced search techniques and natural language processing (NLP) capabilities to sift through the knowledge base, identifying documents that most closely match the user’s query. By prioritizing relevance and recency, the retriever ensures that the foundational information for each response is pertinent and reflects the latest legislative and policy updates. This semantic search process is critical, as it guarantees that the system’s outputs are grounded in the most current and relevant facts, a prerequisite for accurate and trustworthy communication. Before the retrieval of documents in our knowledge base, we need to build our knowledge base. To achieve that, we use a document reader to load vast amounts of documents to compile a knowledge base that can be used by the retriever. This element of the system is tasked with the complex job of parsing and understanding the content within the selected documents. Employing multi-modal NLP techniques, the document reader analyses and embeds (encodes) the text, tables, and images, identifying key facts, arguments, and nuances contained within. It is capable of discerning the relevance of different pieces of information, categorizing them by significance later to the user’s query. The final stage in our RAG model’s architecture involves the generative component, which is where we use a Large Language model to answer the question from the retrieved information. What sets this stage apart is LLM’s ability to tailor the complexity and style of the response to suit the layperson’s understanding, thereby demystifying government policies and making them accessible to a broader audience, while maintaining credibility by restricting the LLM to only answer the question based on the information retrieved from our pre-defined knowledge base. This generative process is not a mere regurgitation of facts; it involves a sophisticated synthesis of information, ensuring that the final response is coherent, contextually relevant, and engaging. By giving the context and restricting LLM to answer based on the context the retriever got, it solved the hallucinating problems and ensured that LLM could not produce incorrect answers or unexpected answers. Through this intricate, multi-step process, our RAG model ensures that information disseminated to the public is not just accurate and up-to-date but also clear, comprehensible, and actionable, which we believe can bridge the gap between government policy and public understanding and improve policy communication. Our system utilizes a Retrieval-Augmented Generation (RAG) architecture integrated with GPT-3.5-turbo as the core LLM, but potentially, this can be replaced with any large language model that has decent capability. The RAG component consists of three main elements: Document Retriever: This component uses semantic search to identify relevant policy documents based on user queries. We implemented a dense retrieval approach using FAISS (Facebook AI Similarity Search) for efficient similarity matching. Document Retriever: This component uses semantic search to identify relevant policy documents based on user queries. We implemented a dense retrieval approach using FAISS (Facebook AI Similarity Search) for efficient similarity matching. Document Reader: This element processes retrieved documents, extracting key information using named entity recognition and relationship extraction techniques. Document Reader: This element processes retrieved documents, extracting key information using named entity recognition and relationship extraction techniques. Generator: GPT-3.5-turbo serves as the generator, synthesizing information from retrieved documents to produce coherent and accurate responses. Generator: GPT-3.5-turbo serves as the generator, synthesizing information from retrieved documents to produce coherent and accurate responses. We used Langchain to orchestrate retrieval and generation components, enabling seamless integration. However, our approach can be implemented with other similar platforms, packages, or even without them. For better integration with existing systems, we recommend manual implementation to accommodate existing code. The system was optimized through iterative fine-tuning on domain-specific policy data and hyperparameter tuning, balancing retrieval precision and response fluency. Scalability and security As discussed in the introduction of our knowledge base, the architecture of RAG is designed to facilitate dynamic updating and scaling, allowing for the inclusion of new information and policies as they become available. This aspect is crucial for maintaining the system’s accuracy and relevance, given the frequent updates and changes in government regulations. We ensure seamless integration and synchronization between the retrieval knowledge base and the generative models, enabling the system to adapt to new data and evolving policy landscapes efficiently. To further refine the output and ensure the appropriateness of responses, we have incorporated an additional filter layer outside of the RAG framework. This layer is designed to screen the generated responses for potential irrelevancies or sensitivities, such as misinformation or harmful content, that could undermine the integrity of the communication. Though not a core component of RAG or Langchain, this filter plays a vital role in maintaining the quality and reliability of the information disseminated to the public and mitigating the possibility of the misuse of our application. This can help our model to uphold the highest standards of accuracy and responsibility in the communication of government policies. 6 Experiments and results Experiment setup and data collection In our comprehensive experimental process to evaluate the effectiveness of our proposed system, we built a dataset comprising 100 policy documents evenly from Chinese and US regional governments. This selection was designed to show our system is capable of dealing with the diverse languages, governance models, legal systems, and policies of both countries. This dataset spanned various policy areas, as shown in Fig 3, including regional development, environmental protection, social assistance, tax regulations, and educational reforms. For the Chinese policies, we sourced documents from the Poverty Alleviation Database and official provincial websites, ensuring coverage of policies affecting both rural and urban areas. In contrast, the US policies were gathered from the Catalog of U.S. Government Publications (CGP), offering a wide thematic spectrum. This methodical approach allowed us to gather a robust and diverse group of documents, setting the stage for an in-depth analysis of our system’s performance across different policy contexts. Questions distributions for Chinese policies and US policies. System configuration and document processing Our system relied on the GPT-3.5-turbo API with no sampling methods (Temperature = 0) for its question-answering capabilities. In the process of building the knowledge base, we followed several preprocessing steps to ensure efficient retrieval of relevant information from a large corpus of policy documents: Document conversion: PDFs were converted to plain text format. Document conversion: PDFs were converted to plain text format. Text segmentation: Documents were divided into 2000-character chunks with a 200-character overlap to ensure continuity. Text segmentation: Documents were divided into 2000-character chunks with a 200-character overlap to ensure continuity. Embedding generation: Text chunks were embedded using OpenAI’s embedding model. Embedding generation: Text chunks were embedded using OpenAI’s embedding model. Indexing: Embedded chunks were indexed for efficient retrieval. Indexing: Embedded chunks were indexed for efficient retrieval. These steps facilitated the creation of a searchable knowledge base, allowing for efficient retrieval and response generation. Additionally, the effectiveness of the added filter layer in screening out irrelevant or harmful queries was rigorously tested through a set of designed challenges. For the experiment, we invited 20 volunteers to pose 3 to 5 realistic queries per document, simulating typical citizen inquiries. These questions ranged from general policy objectives to specific implementation details. We classified the questions into direct and inferential categories, testing the application’s ability to handle straightforward queries and those requiring generalization from multiple paragraphs. Fig 4 visually represents the distribution of question types regarding US and Chinese policies showing differences in focus areas. For US policies, the distribution has a higher emphasis on questions requiring full understanding (20%) compared to Chinese policies (15%), showing a slight shift in complexity and depth of inquiry. Conversely, questions about numbers constitute a larger portion for Chinese policies (30%) than for US policies (25%), suggesting a greater emphasis on quantitative focus in discussions related to Chinese policies. Our system was tasked with generating responses based on its knowledge base, with its performance measured against expert human responses and the original documents for relevance, and accuracy. Questions distributions for Chinese policies and US policies. Results and performance analysis The experimental results from our evaluation of our system’s performance on US and Chinese policy documents offer insightful observations and underline the system’s effectiveness in navigating complex policy information. Table 1 presents a comparison of accuracy across different regions within the US and China, revealing the system’s proficiency in understanding and responding to queries related to regional policies. The accuracy rates for US regions consistently outperform those for Chinese regions, with an average accuracy of 90.67% for the US compared to 85.58% for China. This disparity may reflect the system’s relative ease in handling the linguistic and structural nuances of policy documents in English over those in Chinese or possibly the varying complexity and specificity of policies between the two countries. Notably, Region II in the US achieved a remarkable 100% accuracy, suggesting the system’s potential for error-less performance under certain conditions. In the results, we did not spot any harmful responses. Additionally, the system showed high accuracy in processing quantitative information, with accuracy rates of 97.99% for US policies and 90.32% for Chinese policies, highlighting its effectiveness in handling numerical data. 10.1371/journal.pone.0311410.t001Table 1Comparison of policy accuracy between US and China by region.Chinese RegionsAccuracyUS RegionsAccuracyRegion A86.21%Region I93.33%Region B96.67%Region II100.00%Region C80.00%Region III86.67%Region D77.41%Region IV80.00%Region E87.50%Region V93.33% Average 85.58% Average 90.67% Comparison of policy accuracy between US and China by region. Further analysis, as detailed in Table 2, breaks down the system’s performance by type of question—Full Understanding, Ground Truths, and Questions About Numbers—offering a deeper dive into its capabilities. The system displayed a strong performance across all question types, with notably high accuracy in “Questions About Numbers” for both US (97.99%) and China (90.32%) policies. This suggests that the system is particularly adept at extracting and synthesizing quantitative information from complex documents. However, there’s a visible gap in the “Full Understanding” category, with the US policies seeing a higher accuracy (88.67%) compared to Chinese policies (77.75%). The reason behind this is that answering this type of question normally will require a more complete context, which means a larger chunk of documents to be sent to the LLM. Most of the time, the LLM does not have enough information to answer the question, which will result in failing to answer the request. This gap could also be attributed to the system’s varying ability to interpret and integrate complex, nuanced information from different languages and legal contexts, and we think there is still room for improvement and it can be solved using a bigger chunk in our knowledge base establishment. 10.1371/journal.pone.0311410.t002Table 2Comparison of accuracy between US and China policies by question types.USChina Full Understanding 88.67%77.75% Ground Truths 85.35%88.67% Questions About Numbers 97.99%90.32% Comparison of accuracy between US and China policies by question types. For the failed cases, we think it is worth taking a closer look at the process and the results to have a better understanding of why it fails. We present two representative cases to analyse the reasons which can lead us to improve our system. To the best of our knowledge, there are no existing methods or accessible datasets that can be directly applied to or compared with our newly built policy question-answering dataset spanning both US and Chinese contexts. Despite this lack of external benchmarks, our model demonstrates strong performance with high accuracy rates across different question types and policy domains, while consistently avoiding harmful outputs. Case study—Failed cases 1 We asked a question in Chinese originally based on “The Ministry of Agriculture and Rural Affairs, together with Zhejiang Province, issued a plan to jointly promote the construction of the Zhejiang Common Prosperity Demonstration Zone.”, and the question we asked the system in Chinese was: In the construction of the Zhejiang Province demonstration zone for common prosperity, what are the targets for the coverage rate of administrative villages in rural domestic sewage treatment and the compliance rate of effluent water quality?. This, however, led the system to gather the related section from a policy about the sewer treatment in “Improvement and Upgrading of the Living Environment” in Jiangsu Province, which we put in the same Region (knowledge base) to test our system because they are close to each other both geographically and economically. We think it may be because our retrieval system thinks the question is more relevant to the topic of the retrieved document snippet, but it ignores the fact that this is from a different province. This may be alleviated in real life since we want the regional government to control their knowledge base, which will result in a natural barrier for this behaviour, but this situation also warns us that we need to have proper labels in our knowledge base so the retrieval system in our system can have the best performance. We can also integrate keyword matching in the future for better retrieving. Case study—Failed cases 2 In this case, we asked “On what date was H.R. 263 introduced?” on a report from the House of Representatives Committee on Natural Resources recommending the passage of H.R. 263 to rename the Oyster Bay National Wildlife Refuge as the Congressman Lester Wolff Oyster Bay National Wildlife Refuge. This time the retrieval system successfully gathered the relative section, but it failed to grab information from the document because of its instructiveness. This suggests that PDFs can be complicated in text extracting, and we can integrate better text extracting techniques in our system, or we can utilize structured documents in the future to better accommodate the system. After analyzing the above two case studies, we can conclude that if the articles in the document are organized relatively better, our system can answer the question nicely. However, when the document has a mixed style of writing or a relatively poor way of structure, our system sometimes struggles to spot the relative content of the document and fails to answer the question. In the future, we will continue to solve this deficiency and try to improve our system, but this also leads us to think if document writers and the government should have a better structure to write the document not just for our system but also for citizens to read and comprehend. The overall analysis indicates a strong performance by our system, with a demonstrated capacity to handle a wide range of query types across diverse policy areas and regions. The higher accuracy rates for US policies suggest that the system may benefit from further tuning to better accommodate the complexities of the Chinese language and policy structure. Despite these differences, the results affirm the system’s potential as a valuable tool for public engagement with government policies, offering an accessible means for citizens to obtain clear, accurate information. The demonstrated effectiveness in filtering out irrelevant or inappropriate queries further helps the system’s applicability in improving the communication between government and citizens on policies, ensuring that the dissemination of policy information remains both accurate and relevant. Application scenarios To illustrate the potential real-world impact of our system, we present the following hypothetical scenarios: Scenario 1: Policy rollout Consider a local government in the US implementing our system to communicate a new environmental policy on waste management. Citizens could potentially query the system through the government’s website or a dedicated mobile app. For instance, if a citizen asked about specific regulations for household waste sorting, the system could retrieve relevant information from the policy documents and generate a clear, concise response. This approach could potentially lead to increased correct waste sorting practices within the first month of policy implementation, demonstrating the system’s potential effectiveness in improving policy understanding and compliance. Scenario 2: Crisis communication In a hypothetical public health emergency in a Chinese province, our system could be deployed to provide real-time information about rapidly changing policies and guidelines. The system’s ability to quickly process new information and respond to citizen queries in natural language could significantly reduce the spread of misinformation and improve public compliance with health measures. Compared to traditional communication methods, our system could potentially handle a much higher volume of inquiries per day, with a high user satisfaction rate. Scenario 3: Complex policy explanation If a complex tax reform policy were introduced in the US, our system could be used to communicate its details. When faced with multi-part queries about how the new policy would affect different income brackets and business types, the system could break down the information into digestible parts. This could potentially result in a reduction in calls to tax offices for clarifications and an increase in timely tax filings, indicating improved public understanding of the complex policy. These hypothetical scenarios illustrate potential applications of our proposed system in various government communication contexts. While not actual implementations, they envision how the system could enhance policy communication and citizen engagement. The system could improve engagement by providing instant, accurate, and easily understandable responses to policy queries, encouraging more active participation. It could also enhance transparency by breaking down complex policies and providing real-time updates during crises, allowing citizens to better understand and engage with government actions across various contexts. Discussion and limitations The results of our system’s experiments reveal significant insights into the interaction between technology and policy dissemination. The variation in accuracy rates between US and Chinese policy documents highlights linguistic and structural complexities, as well as socio-political nuances embedded within these documents. This discrepancy suggests that while technological solutions can significantly enhance public engagement with governmental policies, they must be calibrated to reflect the diverse cultural, linguistic, and administrative contexts in which they operate. Furthermore, the challenges identified in the case studies, such as document structure and specificity in queries, emphasize the need for more nuanced approaches to document management and system design. These findings underscore the potential for developing more sophisticated natural language processing tools that can better handle a variety of languages and document formats. Future work should focus on refining these technologies and addressing their limitations to further enhance the effectiveness of policy communication. Moreover, the limitations identified through the abovementioned case studies point to broader challenges in the field of digital governance. The failure of the system to correctly interpret and respond to certain queries due to document structure or language complexity emphasizes the need for more nuanced approaches to document management and system design. This includes the potential for developing more sophisticated natural language processing tools that can better handle the variety of languages and document formats. Additionally, these findings suggest a reciprocal relationship between technology and policy documentation practices: as technological solutions become more integrated into public engagement strategies, there may be an increasing need for standardized, clear, and accessible policy documentation practices to facilitate effective communication. Also, we suggest the document has a more comprehensive explanation as an appendix or additional explanation in these documents that can alleviate the needs of LLM to understand and give the solution based on the question and context. The challenges and limitations faced by our system are not merely technical obstacles but also opportunities for deeper engagement with the socio-political dimensions of policy dissemination. By recognizing the inherent complexities of this task, future developments can aim not only to refine the technological tools but also to contribute to the broader goal of enhancing democratic engagement and accessibility in the digital age. 7 Conclusion and discussion In conclusion, our study demonstrates the potential of integrating RAG technology with LLMs to significantly enhance the communication of government policies to citizens. Through rigorous experimentation and analysis, our system showcased its ability to accurately interpret and respond to a wide range of policy-related queries across different regions and languages, particularly excelling in processing quantitative data and adapting to the nuances of complex policy documents. Despite facing challenges with document structure and specificity in queries, our approach highlights a promising direction for public administration, offering a more accessible, transparent, and responsive way for governments to engage with their citizens. By continuously refining this technology and addressing its limitations, we can move closer to bridging the gap between government actions and public understanding, fostering a more informed and participatory democratic process. Furthermore, this research contributes to the ongoing discourse on enhancing government-citizen communication, emphasizing the critical role of innovative technologies in shaping the future of public governance and policy dissemination. Future work will focus on validating the system with independent datasets and exploring its applicability in different government contexts to ensure its generalizability and robustness. 30 Jul 2024 PONE-D-24-21873Improving citizen-government interactions with Generative AI: novel human-computer interaction strategies for policy understanding through large language modelsPLOS ONE Dear Dr. Yun, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Sep 13 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Fredrick Romanus Ishengoma Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, we expect all author-generated code to be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Partly Reviewer #2: Yes ********** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: I Don't Know Reviewer #2: Yes ********** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ********** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: • Present the novelty of the paper in the abstract. • Improve the quality of figures. • Add some results in the abstract • Add the fullform of AI in the title • In the last paragraph of introduction add the structure of the paper • Related work section is missing • The abstract mentions the use of Retrieval-Augmented Generation (RAG) technology combined with Large Language Models (LLMs). Could you provide more detailed information on the specific LLMs used and how RAG technology is implemented in the system? How were these technologies selected and optimized for this application? • The abstract states that the system was tested with a dataset of policy documents from Chinese and US regional governments. Could you elaborate on the characteristics of this dataset, including the number of documents, the variety of policy topics, and the preprocessing steps involved? • The abstract indicates that the system generates accurate and comprehensible responses to citizen queries. Can you provide more detailed descriptions of the metrics used to evaluate accuracy and comprehensibility? How were these metrics measured, and what were the baseline comparisons? • It is mentioned that the system boosts public engagement with government policies. Could you provide specific examples or case studies demonstrating how the system has improved engagement? What metrics were used to assess public engagement? • The abstract suggests that the proposed system enhances communication of government policies. Could you describe a scenario where this system is implemented in a real-world government setting? What specific steps would be taken to deploy the system, and what are the expected benefits for transparency and citizen engagement? • How do your recommendations for using RAG technology and LLMs compare to existing methods for policy communication? Are there specific case studies or examples where similar approaches have been successfully implemented? • The abstract highlights the system's high accuracy in processing and responding to queries. Could you include a comparative analysis with previous studies that have used different technologies or methods for policy communication? How does your proposed model address the limitations identified in these studies? • Have the results of the proposed system been validated with independent datasets or in different government contexts? If so, please include the validation process and outcomes. If not, what are the plans for future validation? • The abstract mentions the use of policy documents from Chinese and US regional governments. Could you provide more detailed insights into potential biases introduced by these specific datasets? How do you ensure the generalizability of your findings across different regions and policy areas? • Can you provide specific scenarios where the proposed approach might face challenges in real-world applications, such as variations in policy language or cultural nuances? How were these challenges identified, and what measures were taken to address them? • The abstract notes the importance of making government policies accessible and engaging for citizens. Can you describe a scenario where the early implementation of your system helped a government agency effectively communicate a new policy, resulting in improved public understanding and compliance? What factors contributed to the success in this scenario? • In a scenario where the system is deployed in a large-scale government initiative, what steps would you recommend to ensure continuous performance and accuracy? How would you address potential issues such as data privacy or policy updates? • The abstract mentions high accuracy in processing and responding to queries. Could you provide more details on the specific performance metrics used to evaluate the system? Were there any specific conditions under which the system's performance varied? • Can you provide a scenario where the system’s effectiveness might be challenged by variations in policy complexity or citizen query patterns? How would these challenges be mitigated to maintain the system’s performance? • The abstract discusses the integration of Generative AI and RAG technology for policy communication. Can you provide a detailed comparison of the limitations of traditional policy communication approaches and how your proposed model overcomes them? • What were the specific limitations encountered during the development and testing of the proposed system? How do these limitations compare to those found in previous studies using conventional methods for government policy communication? Reviewer #2: The article addresses the critical issue of improving interactions between citizens and government using generative AI and large language models (LLMs). The authors focus on the existing challenges in policy communication, such as accessibility, complexity, and resource constraints, and propose an innovative solution using Retrieval-Augmented Generation (RAG) technology. The topic of the article is highly relevant in the era of digital transformation. The authors successfully argue the necessity of integrating AI technologies to overcome the existing barriers in government-citizen communication. The innovative approach, which combines RAG and LLM, is promising and shows significant potential for enhancing transparency and citizen engagement. The article provides a detailed description of a system that integrates LLMs with a retrieval mechanism. The authors explain the architecture of the RAG component and the use of Langchain technology for seamless integration of retrieval and response generation processes. The use case involving policy documents from China and the US demonstrates the system's flexibility and effectiveness across different linguistic and legal contexts. The experimental results are impressive: the system exhibited high accuracy in processing and responding to citizen queries on various policy matters. Particularly noteworthy are the accuracy rates for numerical questions, highlighting the system's ability to handle quantitative information effectively. However, the authors note a disparity in accuracy between Chinese and American documents, indicating the need for further tuning to accommodate more complex linguistic and legal contexts. The authors conduct a thorough analysis of the results, highlighting the strengths and weaknesses of the proposed system. The discussion of failed cases provides insights into potential improvements, such as better text extraction and document structuring. The suggestion to improve government policy documentation practices to aid both the AI system and citizen comprehension is particularly interesting. At the same time, the manuscript suffers from a lack of clear understanding of the article's genre. If it aims to present an experimental validation of the proposed tool, the Introduction clearly fails to fulfill its function. It acts more as an overview of the problem rather than a justification of the relevance and novelty of the research, grounded in previous similar studies on the topic. The authors use studies from other researchers to describe the problem being studied, rather than to justify the boundaries of the knowledge gap being filled and its typology. If, however, the current manuscript aims to introduce both the tool itself and its validation into scientific discourse simultaneously, then the description of the tool presented appears extremely modest and requires further detailed elaboration. However, this information is not clearly conveyed in the text of the manuscript. The discussion of the results also requires significant improvement. The authors have only provided their own reflection on the results without integrating them into the existing body of knowledge in the field. In other words, the authors did not engage in an asynchronous dialogue with other researchers on the topic, making it unclear how novel their approach is, what fundamentally sets it apart from existing methods, and other related aspects. The research topic is quite current, making the absence of recent studies on the topic in the list of references somewhat peculiar. At the very least, these should be incorporated during the revision of the Discussion section. ********** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Elena Tikhonova ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. 9 Aug 2024 Dear Editor and Reviewers, We sincerely thank you for your thoughtful and constructive feedback on our manuscript. We have carefully addressed all the comments and suggestions, resulting in significant improvements to our paper. Below, we provide a point-by-point response to each reviewer's comments showing our revised paper has fully addressed these concerns. Reviewer #1: Present the novelty of the paper in the abstract. Response: We have enhanced the abstract to clearly articulate the novelty of our approach. Specifically, we added: "This novel integration of RAG and LLMs for policy communication represents a significant advancement over traditional methods, offering unprecedented accuracy and accessibility." This addition emphasizes the innovative nature of our work within the context of government-citizen communication. Improve the Image Quality. Response: We have fully examined the image quality, and we have not found particular flaws or problems. Please click the external link to view the image since the smaller image might seem fuzzy. Please let us know if there are any problems we need to address before we move forward. Add some results in the abstract. Response: We have incorporated key results into the abstract, stating: "Our system demonstrated high accuracy, averaging 85.58% for Chinese and 90.67% for US policies." This addition provides concrete evidence of our system's effectiveness. Add the fullform of AI in the title. Response: We have updated the title to include the full form of AI: "Improving Citizen-Government Interactions with Generative Artificial Intelligence: Novel Human-Computer Interaction strategies for Policy Understanding through Large Language Models." In the last paragraph of introduction add the structure of the paper. Response: We have added a paragraph at the end of the introduction (lines 80-89) outlining the structure of the paper, providing a clear roadmap for readers. Related work section is missing. Response: We have added a comprehensive "Related Work" section (lines 90-123) that discusses recent research in AI and LLMs for government-citizen communication, contextualizing our work within the field. Provide more detailed information on the specific LLMs used and how RAG technology is implemented in the system. Response: We have expanded our explanation of the system architecture (lines 354-371), detailing the use of GPT-3.5-turbo as our core LLM and explaining the implementation of RAG technology using FAISS for retrieval and Langchain for orchestration. Elaborate on the characteristics of the dataset. Response: We have provided more detailed information about our dataset in the "Experiment Setup and Data Collection" section (lines 391-403), including the number of documents, variety of policy topics, and sources of the documents. Provide more detailed descriptions of the metrics used to evaluate accuracy and comprehensibility. Response: We have expanded our discussion of evaluation metrics in the "Results and Performance Analysis" section (lines 433-448), detailing how accuracy was measured across different question types and regions. Provide specific examples or case studies demonstrating how the system has improved engagement. Response: We have added a new "Application Scenarios" section (lines 521-555) that provides hypothetical scenarios illustrating how our system could enhance policy communication and citizen engagement in various contexts. Describe a scenario where this system is implemented in a real-world government setting. Response: The new "Application Scenarios" section includes multiple hypothetical real-world scenarios, such as policy rollout, crisis communication, and complex policy explanation, demonstrating potential implementations of our system. Compare your recommendations for using RAG technology and LLMs to existing methods for policy communication. Response: The new "Related Work" section (lines 90-123) provides comparisons between our approach and existing methods, highlighting the advantages of our RAG-based system. Include a comparative analysis with previous studies. Response: While direct comparisons were challenging due to the novelty of our approach, we have acknowledged this limitation and provided context for our results in the "Results and Performance Analysis" section (lines 468-472). Have the results been validated with independent datasets? Response: We have acknowledged the need for further validation in our "Conclusion and Discussion" section (lines 604-607), stating our intention to validate the system with independent datasets in future work. Provide more detailed insights into potential biases introduced by these specific datasets. Response: We have addressed potential biases in our "Discussion and Limitations" section (lines 556-589), acknowledging the need for calibration to reflect diverse cultural, linguistic, and administrative contexts. Provide specific scenarios where the proposed approach might face challenges in real-world applications. Response: The "Discussion and Limitations" section (lines 556-589) now includes a discussion of challenges our system might face in real-world applications, such as document structure and language complexity. Describe a scenario where the early implementation of your system helped a government agency effectively communicate a new policy. Response: The new "Application Scenarios" section includes a hypothetical scenario (Scenario 1: Policy Rollout, lines 523-531) demonstrating how our system could aid in communicating a new environmental policy. What steps would you recommend to ensure continuous performance and accuracy? Response: We have addressed this in our "Discussion and Limitations" section, recommending continuous refinement of NLP tools and standardization of policy documentation practices (lines 574-583). Provide more details on the specific performance metrics used to evaluate the system. Response: We have expanded our discussion of performance metrics in the "Results and Performance Analysis" section (lines 433-448), detailing accuracy rates across different question types and regions. Provide a scenario where the system's effectiveness might be challenged. Response: We have included two case studies of failed cases (lines 474-509) that illustrate scenarios where our system faced challenges, along with analysis of these failures. Provide a detailed comparison of the limitations of traditional policy communication approaches and how your proposed model overcomes them. Response: The new "Related Work" section (lines 90-123) includes comparisons between traditional approaches and our model, highlighting how our system addresses limitations of existing methods. What were the specific limitations encountered during the development and testing of the proposed system? Response: We have discussed limitations and challenges in our "Discussion and Limitations" section (lines 556-589), including issues with document structure and language complexity. Reviewer #2: Lack of clear understanding of the article's genre. Response: We have restructured the paper to clearly present it as both an introduction to a novel tool and its experimental validation. The addition of the "Related Work" section (lines 90-123) and the restructured introduction provide a clearer context for our research. Insufficient justification of the relevance and novelty of the research. Response: We have strengthened the justification of our research's relevance and novelty throughout the paper, particularly in the abstract, introduction, and the new "Related Work" section. Modest description of the tool. Response: We have significantly expanded the description of our system in the "Our Approach to Enhancing Government-Citizen Communication" section (lines 214-304) and the "Details in the RAG Component in Our Approach" section (lines 304-388). Insufficient integration of results into existing body of knowledge. Response: The new "Related Work" section (lines 90-123) and expanded "Discussion and Limitations" section (lines 556-589) now better integrate our results with existing literature and discuss the implications of our findings in the broader context of policy communication. Absence of recent studies in the references. Response: We have updated our reference list to include more recent studies, particularly in the new "Related Work" section, ensuring our research is contextualized within the current literature. We believe that these revisions have substantially improved our manuscript, addressing all the reviewers' comments and suggestions. The revised version now provides a clearer, more comprehensive, and well-contextualized presentation of our innovative approach to improving citizen-government interactions through AI. We thank the reviewers for their valuable input, which has helped us enhance the quality and impact of our work. Best Regards, Sheng Yun Submitted filename: Response to Reviewers.pdf 19 Sep 2024 Improving Citizen-Government Interactions with Generative Artificial Intelligence: Novel Human-Computer Interaction strategies for Policy Understanding through Large Language Models PONE-D-24-21873R1 Dear Dr. Yun, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Jinran Wu, PhD Academic Editor PLOS ONE Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #2: All comments have been addressed ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #2: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #2: (No Response) ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #2: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #2: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #2: It is crucial to adhere to the genre expression of the manuscript. If the manuscript concerns the testing of a methodology, the article should present the results of an empirical study. If the article is dedicated to introducing a new tool into the international scientific community, it should have a clear structure of a methodological article or, at the very least, a technical note. Currently, the article has a mixed format, including elements of both empirical and methodological research. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #2: Yes: Elena Tikhonova ********** 2 Oct 2024 PONE-D-24-21873R1 PLOS ONE Dear Dr. Yun, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Jinran Wu Academic Editor PLOS ONE