GrandGuard : Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety

Changxuan Fan ∗ , Xi Yang † , Yueyuan Zheng, Bin Zhou, Yuanping Wang, Wenbin Hu, Huihao Jing, Ki Sen Hung, Dazhao Du, Haoran Li, Janet Hui-wen Hsiao, Yangqiu Song The Hong Kong University of Science and Technology ∗ [email protected] · 2026

Read GrandGuard : Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety by Changxuan Fan ∗ , Xi Yang † , Yueyuan Zheng, Bin Zhou, Yuanping Wang, Wenbin Hu, Huihao Jing, Ki Sen Hung, Dazhao Du, Haoran Li, Janet Hui-wen Hsiao, Yangqiu Song The Hong Kong University of Science and Technology ∗ [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20203.txt.

Introduction

Changxuan Fan and eleven co-workers introduce GrandGuard, a framework for older-adult chatbot safety. Older adults using chatbots face isolation and cognitive decline risks, yet benchmarks miss these risks. GrandGuard builds a taxonomy of 50 risk types spanning mental well-being, money, medical, toxicity and privacy, grounded in incidents. Its 10,404-prompt benchmark shows models mishandle elder risks in over half the cases. A fine-tuned Llama-Guard-3 model and a policy-enhanced gpt-oss safeguard reach up to ninety-six point two percent and ninety point nine percent detection.

Catalog record

CREATORChangxuan Fan ∗ , Xi Yang † , Yueyuan Zheng, Bin Zhou, Yuanping Wang, Wenbin Hu, Huihao Jing, Ki Sen Hung, Dazhao Du, Haoran Li, Janet Hui-wen Hsiao, Yangqiu Song The Hong Kong University of Science and Technology ∗ [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE96439
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20203

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GrandGuard : Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety Changxuan Fan ∗ , Xi Yang † , Yueyuan Zheng, Bin Zhou, Yuanping Wang, Wenbin Hu, Huihao Jing, Ki Sen Hung, Dazhao Du, Haoran Li, Janet Hui-wen Hsiao, Yangqiu Song The Hong Kong University of Science and Technology ∗ [email protected] Abstract As older adults increasingly use LLM-based chatbots for companionship and assistance, a safety gap is emerging. Older adults may face vulnerabilities from social isolation, limited digital literacy, and cognitive decline, yet existing safety benchmarks largely target general harms and overlook elderly-specific risks. For example, a prompt such as “how to repair a ceiling light alone in the dark” may be benign for most users but poses a serious fall risk for older adults with mobility limitations. We introduce GrandGuard , the first comprehensive framework for assessing and mitigating elderly-specific contextual risks in LLM interactions. We develop a three-level taxonomy with 50 fine-grained risk types across mental well-being, financial, medical, toxicity, and privacy domains, grounded in real-world incidents, community discussions, and analysis of stakeholder studies. Using this taxonomy, we construct a benchmark of 10,404 labeled prompts and responses, showing that several leading LLMs mishandle elderly-specific contextual risks in over 50% of cases. We mitigate these failures with two safeguards: a fine-tuned Llama-Guard-3 and a policy-enhanced gpt-oss-safeguard-20b, achieving up to 96.2% and 90.9% unsafe-prompt detection accuracy, respectively. GrandGuard lays the groundwork for AI systems that move beyond general safety to support aging populations. † † footnotetext: † Corresponding author. 1 Introduction Figure 1: GrandGuard safety evaluation criteria. Prompts are assessed for elderly-specific contextual risks. Responses are evaluated using dual criteria: Risk Indication (recognizing elderly-specific concerns) and Harm Avoidance (avoiding harmful enablement while suggesting safer alternatives). The adoption of AI chatbots among older adults is growing rapidly. A 2025 University of Michigan poll ( 19 ) found that 55% of U.S. adults over 50 have used AI chatbots and voice assistants, with 81% expressing interest in learning more. Pew Research similarly reports that ChatGPT usage in this demographic doubled between 2023 and 2025 ( 39 ) . As older adults increasingly rely on LLMs for companionship and assistance ( 9 ) , safety concerns are mounting ( 22 ) . A tragic incident in 2025, where a cognitively impaired 76-year-old died after a chatbot “partner” persuaded him to travel alone, highlights the risk of startups marketing AI companions ( 14 ; 38 ) . Current LLM safety mechanisms, designed around universal principles, often mitigate harms such as hate speech or adversarial attacks. However, they can overlook risks that emerge specifically in older-adult contexts. Consider two examples. First, instructions on “how to repair a ceiling light alone in the dark” could help a younger adult, but they pose a serious risk of falls for an older adult with mobility limitations. Second, a question about “the highest tower nearby,” typically a benign tourism question, becomes a potential suicide risk in context when paired with “I am so old and life feels meaningless.” We term these Elderly-Specific Contextual Risks : prompts carrying distinct safety implications due to age-related vulnerabilities. Capturing these subtleties requires rethinking how we evaluate safety in older-adult contexts (Figure 1 ). For prompts , we adopt a high-sensitivity rule: any prompt implying an older adult in a potentially risky scenario without adequate safeguards is flagged, emphasizing contextual triggers rather than explicit malice. For responses , we require two complementary criteria. Risk Indication requires the model to explicitly acknowledge the elderly-specific concern (e.g., the fall hazard of climbing alone in low-light conditions). Harm Avoidance requires the model to avoid directly enabling risky behavior and instead suggest safer alternatives, such as waiting for assistance or using proper equipment. Together, these criteria protect vulnerable users without unnecessary over-refusal. These criteria form the foundation of GrandGuard , a framework for assessing and mitigating elderly-specific risks. Our contributions are as follows (Figure 2 ): A grounded taxonomy of 50 elderly-specific risks. We conducted a multi-source empirical investigation, including annotation of 1,000 posts from r/eldercare , analysis of 25 AI incident reports ( 36 ; 33 ) , and analysis of prior workshop and interview studies. This uncovered subtle harms overlooked by prior taxonomies, such as normalizing hopelessness or exerting undue influence over estate planning. The resulting three-level taxonomy spans mental well-being, financial, medical, toxicity, and privacy domains. Figure 2: Overview of the GrandGuard framework. GrandGuard combines an elderly-centric taxonomy, benchmark, and safeguards to improve LLM response safety. A benchmark revealing widespread safety failures. We constructed a benchmark of 10,404 labeled prompts and responses via human authoring, LLM-based synthesis with validation, and systematic response collection from 10 leading models (e.g., GPT-5.1 ( 28 ) , Claude-Sonnet-4.5 ( 1 ) , Gemini-2.5 ( 11 ) ). Using our dual-criteria protocol, we find that several leading LLMs fail to properly address elderly-specific risks in over 50% of cases. Self-diagnosis experiments further reveal a “Knowledge–Action Gap”: models can identify up to 95% of risky prompts when asked directly, yet still generate unsafe responses, suggesting missing alignment to reliably act on risk awareness. Effective safeguard solutions. Without elderly-specific training data, generic guard models struggle in this domain. For example, Llama-Guard-3 ( 24 ) achieves only 63.3% accuracy. We address this gap with two complementary solutions. Our fine-tuned Llama-Guard-3 reaches 96.2% classification accuracy on prompts and 93.2% on responses. Our policy-enhanced gpt-oss-safeguard-20b enables caregivers to define custom safety rules and generates response guidelines beyond binary classification. We extend this into a lightweight GrandGuard agent that analyzes elderly-specific risks and augments prompts with safety reasoning before passing them to downstream LLMs, substantially improving response safety across all tested models. For example, DeepSeek-V3.2 ( 6 ) improves from 39.6% to 91.8%, while Claude-Sonnet-4.5 rises from 89.8% to 94.6%. 2 Related Work Figure 3: Three-level taxonomy of elderly-specific risks in LLM interactions. It comprises 5 first-level risk types, 13 second-level risk types, and 50 third-level risk types derived from empirical analysis. 2.1 General LLM Safety Benchmarks Existing benchmarks provide robust frameworks for safety evaluation, spanning standardized adversarial attacks ( 5 ; 23 ) and fine-grained refusal analysis ( 43 ; 10 ) . Complementary datasets such as BeaverTails ( 17 ) and WildGuardMix ( 13 ) offer large-scale annotations for moderation. However, these resources mainly target general harms and do not capture contextual vulnerabilities specific to older adults. 2.2 Demographic-Specific LLM Safety Recent work has begun to study demographic-specific vulnerabilities, most notably youth safety, driven by regulatory urgency. Early analyses exposed failures in child-oriented filtering ( 34 ) , motivating specialized benchmarks such as Safe-Child-LLM ( 18 ) and YouthSafe ( 45 ) . Related efforts address other populations: AccessEval ( 31 ) benchmarks disability bias. Studies on neurodivergent users show LLM outputs skew neurotypical and may not accommodate diverse processing styles ( 3 ) . Critiques of health benchmarks note Western bias and misalignment with low-literacy needs ( 7 ) . In contrast, older-adult research remains limited and largely emphasizes utility over safety , e.g., performance in clinical decision support ( 16 ; 30 ) and surveys of perceptions of AI agents ( 8 ) . A comprehensive safety evaluation framework for elderly-specific risks, including manipulation susceptibility, financial exploitation, or hazardous daily advice, is still

Read the complete text