Linguistic comparison of AI- and human-written responses to online mental health queries Authors: Koustuv Saha, Yoshee Jain, Violeta J Rodriguez, Munmun De Choudhury Published: 2026 ABSTRACT AbstractThe ubiquity and widespread use of digital and online technologies have transformed mental health support, with online mental health communities (OMHCs) providing safe spaces for peer support. More recently, generative AI and large language models (LLMs) have introduced new possibilities for scalable, around-the-clock mental health assistance that could potentially augment and supplement the capabilities of OMHCs. Although genAI shows promise in delivering immediate and personalized responses, their effectiveness in replicating the nuanced, experience-based support of human peers remains an open question. In this study, we harnessed 24,114 posts and 138,758 online community (OC) responses from 55 OMHCs on Reddit. We prompted several state of the art LLMs (GPT-4-Turbo, Llama-3, and Mistral-7B) with these posts, and compared their responses to human-written (OC) responses based on a variety of linguistic measures across psycholinguistics and lexico-semantics. Our findings revealed that AI responses are more verbose, readable, and analytically structured, but lack linguistic diversity and personal narratives inherent in human-human interactions. Through a qualitative examination, we found validation as well as complementary insights into the nature of AI responses, such as its neutrality of stance and the absence of seeking back-and-forth clarifications. We discuss the ethical and practical implications of integrating generative AI into OMHCs, advocating for frameworks that balance AI’s scalability and timeliness with the irreplaceable authenticity, social interactiveness, and expertise of human connections that form the ethos of online support communities.Subject terms: Mathematics and computing, Computer science Introduction Advancements in digital technologies over the recent decades have been redefining how individuals engage with mental health care and support systems1–5. Traditional therapy and peer support avenues have expanded into online spaces, offering new modes of interaction and assistance. Online mental health communities (OMHCs) exemplify these developments, providing safe, anonymous platforms where users can openly express their thoughts, seek advice, and connect with others experiencing similar challenges1,6,7. These communities thrive on mutual support, where the collective lived experience of participants helps foster understanding and empathy. The success of OMHCs lies in their ability to create environments conducive to open self-disclosure, promising to reduce the stigma often associated with discussing mental health concerns in offline settings1,7–9. Recent technological developments, particularly in generative artificial intelligence (genAI), have introduced new opportunities for mental health support through sophisticated conversational agents and large language models (LLMs), above and beyond OMHCs. People are not only appropriating OMHCs for mental health help seeking, but also exploring the use of LLMs as chatbots during times of distress10–13. Research has subsequently sought to understand the potential benefits of LLMs—GPT-4 has demonstrated the ability to mimic human-like conversation and adapt responses to users’ needs14–16. Peer support is a key therapeutic approach to tackling mental health concerns7,17,18, and LLMs promise to offer that kind of support around the clock in a scalable fashion, extending the reach of human-based peer support available in OMHCs. This, in turn, can open doors to significantly extending and scaling mental health services, especially given the paucity of trained mental health professionals in the US19. Finally, research has shown that being able to confide in sensitive or challenging life experiences to a trusted peer can alleviate feelings of distress7,20–22, and LLMs can potentially serve as those “trusted peers” in OMHCs that can provide non-judgmental help and advice to people with mental health struggles. In doing so, LLMs carry the potential ability to augment and supplement support received from human peers in OMHCs. Although these emerging AI technologies show immediate promise, LLMs as mental health support tools are yet to be thoroughly assessed against the organic, nuanced responses generated within human interactions in online communities23–25. This paper aims to bridge this gap by analyzing the lexico-semantics and overall effectiveness of 24,114 AI responses generated by each of three state-of-the-art LLMs—GPT-4-Turbo, Llama-3.1, and Mistral-7B— in comparison to those of human-written content on 55 OMHCs on Reddit. Our study identifies previously under-explored linguistic phenomena distinguishing AI and human peer mental health support, including greater reliance on generic framing, reduced contextual grounding, and constrained expressions of emotionally attuned validation in AI responses. Our goal is to examine how these responses compare in linguistic structure, emotional and informational support, adaptability, and potential limitations, thereby providing one of the first population-level characterizations of how LLM-generated mental health support diverges from human-written peer mental health support at scale. Together, this work represents one of the first large-scale, cross-model linguistic comparisons of AI versus human mental health support in naturally occurring online contexts. This study offers many implications. While AI may offer scalable and empathetic support, its lack of linguistic diversity and creativity could limit its effectiveness in long-term or nuanced therapeutic contexts. The findings also emphasize a dual role for AI in supplementing peer support—providing consistent assistance while recognizing and addressing its gaps in personalization and context-driven empathy. This research contributes to the broader discourse on the ethical and practical integration of AI in mental health care. It underscores the necessity of developing frameworks that harness the benefits of AI’s capabilities while maintaining the irreplaceable human touch of peer and professional support. Results We collected 24,114 posts and 138,758 human-written (OC) responses from 55 mental health-related subreddits7. To generate AI responses, we queried these posts using state-of-the-art LLMs—GPT-4-Turbo, Llama-3.1, and Mistral-7B. We then conducted a suite of comprehensive psycholinguistic and lexico-semantic analyses comparing AI and OC responses. For ease of exposition, the majority of our results focus on comparisons with GPT-4-Turbo as the representative AI responses. Finally, we performed robustness analyses by extending our comparisons to Llama-3.1 and Mistral-7B. Psycholinguistic analysis Table 1 summarizes the occurrences of psycholinguistic attributes in the AI and OC responses. Although several of the comparisons are significant as per t-tests, we primarily focus on examining the differences which show moderate to large effect size (Cohen’s d>0.20)26. Affect. AI responses contained greater sadness (by 74%) but much lower anger (by 90%) than OC responses. This suggests that while AI responses may align with the emotional depth of distressing discussions, they might adopt a more neutral or supportive tone, minimizing confrontational or angry expressions that may present in OC interactions. This is also likely associated with the heavy degree of moderation and red-teaming the LLMs have undergone to prevent too negative (or abusive) responses. Affect. AI responses contained greater sadness (by 74%) but much lower anger (by 90%) than OC responses. This suggests that while AI responses may align with the emotional depth of distressing discussions, they might adopt a more neutral or supportive tone, minimizing confrontational or angry expressions that may present in OC interactions. This is also likely associated with the heavy degree of moderation and red-teaming the LLMs have undergone to prevent too negative (or abusive) responses. Cognition and Perception. AI responses showed greater occurrences of differentiation (by 68%) and feel (by 49%) than OC responses. In contrast, AI responses show lower occurrences of causation (by 47%), certainty (by 58%), and see (by 67%). This plausibly indicates that AI responses focus more on acknowledging emotions rather than asserting definitive explanations. Cognition and Perception. AI responses showed greater occurrences of differentiation (by 68%) and feel (by 49%) than OC responses. In contrast, AI responses show lower occurrences of causation (by 47%), certainty (by 58%), and see (by 67%). This plausibly indicates that AI responses focus more on acknowledging emotions rather than asserting definitive explanations. Social and Personal Concerns. This category contains several content words relating to social and personal concerns. AI and OC responses show similar occurrences of keywords in several categories with low Cohen’s d. Interestingly, AI responses showed lower use of friend (−58%), female (−58%), and male (−71%) keywords than OC responses. These categories are associated with sharing of personal and social relationships, and in online communities, members often share their personal narratives, which may not be the case for AI. Again, AI responses showed lower relativity-related attributes, including motion (−50%) and time (−60%) than OC responses, which is also likely associated with sharing people’s past experiences. In contrast, AI responses show a significantly higher occurrence of affiliation (25%) and power (192.6%) than OC responses. This could be associated with AI’s tendency to provide more structured guidance, rather than personal anecdotes. Social and Personal Concerns. This category contains several content words relating to social and personal concerns. AI and OC responses show similar occurrences of keywords in several categories with low Cohen’s d. Interestingly, AI responses showed lower use of friend (−58%), female (−58%), and male (−71%) keywords than OC responses. These categories are associated with sharing of personal and social relationships, and in online communities, members often share their personal narratives, which may not be the case for AI. Again, AI responses showed lower relativity-related attributes, including motion (−50%) and time (−60%) than OC responses, which is also likely associated with sharing people’s past experiences. In contrast, AI responses show a significantly higher occurrence of affiliation (25%) and power (192.6%) than OC responses. This could be associated with AI’s tendency to provide more structured guidance, rather than personal anecdotes. Biological Concerns. Under biological concerns, AI responses showed significantly higher occurrence of health (by 207%), but lower occurrences of body (by 26%) and sexual (by 30%) related keywords than OC responses. This suggests that AI responses may tend to focus on health-related advice or information rather than engaging in personal or detailed interactions about physical bodies or sexual topics. In contrast, OC responses may include more personal experiences, concerns, and narratives related to bodily functions and sexual health, which AI responses may avoid. Biological Concerns. Under biological concerns, AI responses showed significantly higher occurrence of health (by 207%), but lower occurrences of body (by 26%) and sexual (by 30%) related keywords than OC responses. This suggests that AI responses may tend to focus on health-related advice or information rather than engaging in personal or detailed interactions about physical bodies or sexual topics. In contrast, OC responses may include more personal experiences, concerns, and narratives related to bodily functions and sexual health, which AI responses may avoid. Function Words. Function words are known to be associated with understanding linguistic style and psychology of expression of individuals27. We find that AI responses showed significantly greater use of articles (by 19%), prepositions (by 14%), auxiliary verbs (by 26%), and conjunctions (by 18%)—all indicative of a more articulate style of writing in AI responses. In contrast, the AI responses showed lower use of negation (−43%), number (−68%), and quantifier (−50%). This suggests that AI responses may avoid overly absolute or uncertain statements. Function Words. Function words are known to be associated with understanding linguistic style and psychology of expression of individuals27. We find that AI responses showed significantly greater use of articles (by 19%), prepositions (by 14%), auxiliary verbs (by 26%), and conjunctions (by 18%)—all indicative of a more articulate style of writing in AI responses. In contrast, the AI responses showed lower use of negation (−43%), number (−68%), and quantifier (−50%). This suggests that AI responses may avoid overly absolute or uncertain statements. Interpersonal Focus (Pronouns). Pronoun use is associated with narrative style and focus of attention in interpersonal conversations28. We find that AI responses show a significantly lower use of first person singular (−70%) and first person plural (−71%) pronouns than OC responses. This indicates the lack of personal narration style and self-attentional focus in the AI responses. In contrast, in OCs, individuals often respond by sharing their personal experiences. Further, the greater use of first person plural (we, us, etc.) reveals the sharing of experiences and solidarity as a collective identity in online communities29,30. Again, the AI responses show greater use of second-person pronouns (39%) and impersonal pronouns (29%) than OC responses—which could associate with how AI is more likely to provide structured information and advice. Further, pronoun usage is also associated with social hierarchy—those in higher social positions are more likely to use second-person pronouns31. Therefore, the observations on pronoun usage could be perceived as the AI assuming a higher social position than the one who asks the question. Interpersonal Focus (Pronouns). Pronoun use is associated with narrative style and focus of attention in interpersonal conversations28. We find that AI responses show a significantly lower use of first person singular (−70%) and first person plural (−71%) pronouns than OC responses. This indicates the lack of personal narration style and self-attentional focus in the AI responses. In contrast, in OCs, individuals often respond by sharing their personal experiences. Further, the greater use of first person plural (we, us, etc.) reveals the sharing of experiences and solidarity as a collective identity in online communities29,30. Again, the AI responses show greater use of second-person pronouns (39%) and impersonal pronouns (29%) than OC responses—which could associate with how AI is more likely to provide structured information and advice. Further, pronoun usage is also associated with social hierarchy—those in higher social positions are more likely to use second-person pronouns31. Therefore, the observations on pronoun usage could be perceived as the AI assuming a higher social position than the one who asks the question. Temporal References. Temporal references in language can indicate how individuals frame their thoughts-whether they reflect on the past, anticipate the future, or focus on the present moment. We find that AI responses showed lower use for past (−62%) and future (−39%) focus, but higher use of present (15%) focus. This suggests that AI responses are less likely to engage in retrospective storytelling or speculation about future outcomes, which are common in OC discussions. Instead, AI tended to provide immediate, present-focused guidance. Temporal References. Temporal references in language can indicate how individuals frame their thoughts-whether they reflect on the past, anticipate the future, or focus on the present moment. We find that AI responses showed lower use for past (−62%) and future (−39%) focus, but higher use of present (15%) focus. This suggests that AI responses are less likely to engage in retrospective storytelling or speculation about future outcomes, which are common in OC discussions. Instead, AI tended to provide immediate, present-focused guidance. Informal Language. Finally, we find AI responses exhibited significantly lower use of the informal language across swear (by 99%), netspeak (by 97%), nonfluent (by 80%), and filler (by 100%) than OC responses. This is plausibly linked to the aspect that LLMs are trained and fine-tuned to generate formal language while avoiding inappropriate or overly casual language. Informal Language. Finally, we find AI responses exhibited significantly lower use of the informal language across swear (by 99%), netspeak (by 97%), nonfluent (by 80%), and filler (by 100%) than OC responses. This is plausibly linked to the aspect that LLMs are trained and fine-tuned to generate formal language while avoiding inappropriate or overly casual language. Table 1Summary of comparison of AI (GPT-4-Turbo) and OC (Reddit) responses in terms of psycholinguistic categories as per Linguistic Inquiry and Word Count (LIWC84), along with Cohen’s d, paired t-tests, and Kolmogorov-Smirnov (KS)-testLIWCAIOCΔ%dtKSAffectPos. Affect0.0320.038−16.59-0.12−13.59***0.28***Neg. Affect0.00.0−97.39-0.03−3.09**0.0Anxiety0.0060.00522.30.089.75***0.19***Anger0.00.002−90.0-0.20−22.53***0.07***Sad0.0080.00574.250.2430.03***0.35***Cognition and PerceptionInsight0.0230.0219.150.089.76***0.33***Causation0.0060.011−47.02−0.30−35.44***0.13***Discrep.0.0140.015−6.11−0.05−5.68***0.36***Tentat.0.0290.0282.310.022.72**0.29***Certainty0.0050.012−58.0−0.29−32.11***0.16***Differ.0.0540.03268.010.6388.92***0.28***Percept0.0170.017−1.12-0.01−0.940.32***See0.0020.005−67.19−0.26−29.41***0.1***Hear0.0030.004−18.03−0.06−6.97***0.15***Feel0.0120.00849.220.2327.91***0.42***Social & Personal ConcernsFamily0.0010.002−62.00−0.14−15.2***0.04***Friend0.0010.002−58.52−0.13−15.01***0.05***Female0.0020.004−58.43−0.17−20.56***0.05***Male0.0010.004−70.69−0.22−25.3***0.06***Work0.0060.0062.480.011.190.2***Leisure0.0010.002−49.01−0.10−10.9***0.07***Home0.0010.001−41.39−0.09−10.0***0.06***Money0.0010.001−31.97−0.06−7.13***0.06***Religion0.00.001−92.23−0.10−10.7***0.02***Death0.00.001−53.16−0.08−9.47***0.01*Motion0.0050.011−50.36−0.31−35.13***0.15***Space0.0340.037−7.15−0.08−9.84***0.22***Time0.0110.029−60.17−0.57−66.03***0.28***Affiliation0.0160.01325.040.1315.59***0.45***Achievement0.0120.0117.60.1112.47***0.4***Power0.0310.011192.60.96132.96***0.52***Reward0.0070.014−53.2−0.33−37.4***0.15***Risk0.0080.00480.220.2936.32***0.35***LIWCAIOCΔ%dtKSBiological ProcessesBody0.0020.003−26.01−0.07−8.31***0.1***Health0.0190.006206.960.6890.23***0.55***Sexual0.00.001−88.34−0.09−9.66***0.01**Ingest0.0010.001−29.78−0.05−5.52***0.07***Function WordsArticle0.0420.03618.80.2327.89***0.23***Preposition0.1030.0914.160.2733.79***0.27***Aux. Verb0.0940.07426.400.4353.54***0.34***Adverb0.0440.052−16.57−0.21−24.77***0.22***Conjunction0.0660.05617.690.2834.15***0.25***Negation0.0080.015−43.22−0.26−29.72***0.16***Verb0.1270.158−19.72−0.46−56.34***0.3***Adjective0.0530.04127.440.2631.17***0.31***Compare0.0170.021−20.15−0.14−15.87***0.36***Interrog.0.0120.012−4.54−0.03−3.33***0.38***Number0.0010.004−67.65−0.28−31.81***0.09***Quantifier0.0090.018−50.38−0.40−46.03***0.2***Interpersonal Focus (Pronouns)1st P. Sin.0.0170.058−70.1−0.93−107.9***0.53***1st P. Plu.0.0010.003−71.3−0.23−27.32***0.1***2nd P.0.0470.03438.600.3237.73***0.38***3rd P. Sin.0.0030.006−56.77−0.22−26.48***0.06***3rd P. Plu.0.0050.005−1.53−0.01−0.750.25***Impersonal Prn.0.0650.0529.320.3745.87***0.26***Temporal ReferencesPast Focus0.0130.035−61.9−0.48−53.59***0.33***Present Focus0.1260.10915.410.2733.72***0.28***Future Focus0.0060.01−38.76−0.24−28.18***0.15***InformalSwear0.00.001−99.22−0.14−15.34***0.05***Netspeak0.00.007−96.89−0.26−28.6***0.16***Assent0.0030.008−63.48−0.13−14.44***0.17***Nonfluent0.00.002−80.11−0.18−19.56***0.06***Filler0.00.0−100.0−0.07−8.08***0.01**p-values reported after Bonferroni correction. (* p<0.05, ** p<0.01, *** p<0.001). Summary of comparison of AI (GPT-4-Turbo) and OC (Reddit) responses in terms of psycholinguistic categories as per Linguistic Inquiry and Word Count (LIWC84), along with Cohen’s d, paired t-tests, and Kolmogorov-Smirnov (KS)-test p-values reported after Bonferroni correction. (* p<0.05, ** p<0.01, *** p<0.001). Lexico-semantic analysis We examined the lexico−semantics of language differences between the AI and OC responses—Table 2 summarizes the results and Fig. 1 show the distributions. We describe our findings below: Table 2Summary of comparing the responses by AI (GPT-4-Turbo) and by humans on online communities (Reddit) in terms of effect size (Cohen’s d), paired t-test, and KS-test (*p < 0.05, **p < 0.01, ***p < 0.001)CategoriesAIRedditDifference %Cohen’s dt-testKS-testLinguistic StructureVerbosity: Words Per Response160.3577.35107.300.6369.25***0.35***Verbosity: Words Per Sentence19.2813.7640.120.1719.14***0.34***Readability11.196.5870.060.7179.13***0.64***Repeatability0.330.2066.560.8895.54***0.40***Complexity4.633.3139.970.7583.52***0.50***Linguistic StyleCategorical Dynamic Index (CDI)9.666.9040.040.2933.04***0.25***Formality0.870.6730.140.97107.43***0.51***Empathy0.840.7118.760.6369.19***0.23***Politeness0.790.6717.990.5763.54***0.26***Adaptability to QuerySemantic Similarity0.710.5921.260.5263.13***0.30***Linguistic Style Accommodation0.770.718.830.2123.28***0.22***Diversity/Creativity0.060.13-57.04-0.89-103.37***0.46***Social SupportEmotional Support0.790.4962.430.7889.90***0.57***Informational Support0.620.5219.900.2425.69***0.36*** Summary of comparing the responses by AI (GPT-4-Turbo) and by humans on online communities (Reddit) in terms of effect size (Cohen’s d), paired t-test, and KS-test (*p < 0.05, **p < 0.01, ***p < 0.001) Comparison of the distribution of lexico-semantic measures between GPT-4-Turbo (AI) and Reddit (OC) responses. The dotted lines show respective means. Prior research highlights the critical role of linguistic structure in effective psychotherapy, as it shapes the depth, clarity, and impact of therapeutic communication32. Linguistic structure can influence both comprehension and engagement in supportive interactions. We examined four dimensions of linguistic structure, as described below: Verbosity The depth and thoroughness of a response play a crucial role in its effectiveness in providing social support33. Prior research has emphasized that both the quality and quantity of information contribute to supportive communication, with verbosity correlating to the level of detail and elaboration34. We found statistically significant differences between AI and OC responses (Table 2), with AI responses exhibiting greater verbosity at both the response-level (Cohen’s d=0.63) and the sentence-level (Cohen’s d=0.17), indicating a more detailed and lengthier style of communication in AI responses. Verbosity The depth and thoroughness of a response play a crucial role in its effectiveness in providing social support33. Prior research has emphasized that both the quality and quantity of information contribute to supportive communication, with verbosity correlating to the level of detail and elaboration34. We found statistically significant differences between AI and OC responses (Table 2), with AI responses exhibiting greater verbosity at both the response-level (Cohen’s d=0.63) and the sentence-level (Cohen’s d=0.17), indicating a more detailed and lengthier style of communication in AI responses. Readability refers to surface-level textual complexity commonly expressed as an estimated grade level based on structural features such as sentence length and word length35,36. We used the Coleman-Liau Index (CLI) to operationalize this measure as an indicator of linguistic complexity. We found that AI responses showed 70% higher readability than OC responses (Cohen’s d = 0.71). While this suggests that AI responses (mean=11.19) exhibit higher readability, these scores correspond, under conventional interpretations of CLI, to approximately 11 years of formal education required to engage with the text structure. In contrast, OC responses have a much lower readability score (mean = 6.90), corresponding to approximately 7 years of education. We found that AI responses showed 70% higher readability than OC responses (Cohen’s d = 0.71). Under conventional interpretation of CLI, this suggests that AI responses (mean = 11.19) exhibit a higher writing quality, AI responses also require approximately 11 years of education for comprehension. In contrast, OC responses have a much lower readability score (mean = 6.90), corresponding to about 7 years of education for comprehension. Readability refers to surface-level textual complexity commonly expressed as an estimated grade level based on structural features such as sentence length and word length35,36. We used the Coleman-Liau Index (CLI) to operationalize this measure as an indicator of linguistic complexity. We found that AI responses showed 70% higher readability than OC responses (Cohen’s d = 0.71). While this suggests that AI responses (mean=11.19) exhibit higher readability, these scores correspond, under conventional interpretations of CLI, to approximately 11 years of formal education required to engage with the text structure. In contrast, OC responses have a much lower readability score (mean = 6.90), corresponding to approximately 7 years of education. We found that AI responses showed 70% higher readability than OC responses (Cohen’s d = 0.71). Under conventional interpretation of CLI, this suggests that AI responses (mean = 11.19) exhibit a higher writing quality, AI responses also require approximately 11 years of education for comprehension. In contrast, OC responses have a much lower readability score (mean = 6.90), corresponding to about 7 years of education for comprehension. Repeatability refers to the degree of repetition of words37,38. AI responses showed 67% higher repeatability than OC responses (Cohen’s d = 0.88). While repetition can sometimes reinforce key points, in this case, it suggests a potential decrease in writing quality, especially when combined with longer or more verbose responses. This higher repeatability may reflect a redundancy in AI responses, which could impact the conciseness and clarity of communication. Repeatability refers to the degree of repetition of words37,38. AI responses showed 67% higher repeatability than OC responses (Cohen’s d = 0.88). While repetition can sometimes reinforce key points, in this case, it suggests a potential decrease in writing quality, especially when combined with longer or more verbose responses. This higher repeatability may reflect a redundancy in AI responses, which could impact the conciseness and clarity of communication. Complexity captures the sophistication words, based on the average length of words per sentence. AI showed 40% higher complexity than OC responses (Cohen’s d = 0.75). The higher complexity in AI responses indicates a more intricate and detailed use of language. However, the added complexity could also lead to more convoluted sentence structures that may not always be aligned for clear and simple conversational communication. Complexity captures the sophistication words, based on the average length of words per sentence. AI showed 40% higher complexity than OC responses (Cohen’s d = 0.75). The higher complexity in AI responses indicates a more intricate and detailed use of language. However, the added complexity could also lead to more convoluted sentence structures that may not always be aligned for clear and simple conversational communication. Then, we analyzed the linguistic style of expression, another crucial factor in the effectiveness of psychotherapy and social support32,33,39. Linguistic style influences the tone and interactional dynamics of language. We examined the differences in four dimensions of linguistic style, as described below: Categorical-Dynamic Index (CDI) differentiates categorical (analytical, structured) and dynamic (fluid, narrative-driven) language styles40. We found that AI responses show a 60% higher CDI than OC responses (Cohen’s d=0.29). This indicates that AI used a much more analytical writing style, whereas Reddit members used a personal narrative style—aligning with our observations in the psycholinguistic examination. Categorical-Dynamic Index (CDI) differentiates categorical (analytical, structured) and dynamic (fluid, narrative-driven) language styles40. We found that AI responses show a 60% higher CDI than OC responses (Cohen’s d=0.29). This indicates that AI used a much more analytical writing style, whereas Reddit members used a personal narrative style—aligning with our observations in the psycholinguistic examination. Formality captures adherence to grammatical conventions and structured syntax, indicating the degree of professional or casual expressions41,42. We found that AI responses showed a 30% higher formality score than OC responses with statistical significance and large effect size (Cohen’s d=0.97). Formality captures adherence to grammatical conventions and structured syntax, indicating the degree of professional or casual expressions41,42. We found that AI responses showed a 30% higher formality score than OC responses with statistical significance and large effect size (Cohen’s d=0.97). Empathy reflects the extent to which language conveys emotional understanding, validation, and engagement8,43. Our analyses revealed that AI responses demonstrated 19% higher empathy than OC responses, with a moderate effect size (Cohen’s d = 0.63). This suggests that AI responses are more likely to incorporate linguistic cues that convey an empathetic tone. These findings align with recent research indicating that large language models (LLMs) are becoming increasingly adept at simulating empathy, creating interactions that make users feel seen and heard15,44,45. Empathy reflects the extent to which language conveys emotional understanding, validation, and engagement8,43. Our analyses revealed that AI responses demonstrated 19% higher empathy than OC responses, with a moderate effect size (Cohen’s d = 0.63). This suggests that AI responses are more likely to incorporate linguistic cues that convey an empathetic tone. These findings align with recent research indicating that large language models (LLMs) are becoming increasingly adept at simulating empathy, creating interactions that make users feel seen and heard15,44,45. Politeness includes stylistic components to show respect, social harmony, conflict-avoiding, and considerate tone46. AI responses exhibited 18% higher politeness than OC responses, with a moderate effect size (Cohen’s d = 0.57). This suggests that AI generates more courteous and polite responses, potentially enhancing the perceived supportiveness of interactions. Politeness includes stylistic components to show respect, social harmony, conflict-avoiding, and considerate tone46. AI responses exhibited 18% higher politeness than OC responses, with a moderate effect size (Cohen’s d = 0.57). This suggests that AI generates more courteous and polite responses, potentially enhancing the perceived supportiveness of interactions. We also measured linguistic adaptability. In interpersonal interactions, people tend to adapt to the language and expressions of each other47. A body of psychotherapeutic and psycholinguistic research reveals how linguistically adaptable and accommodating responses are more effective in support than templated or generic responses33,48,49. Even for human-AI interactions, prior work noted that when an AI responds with more adaptable language to the user, the AI’s perceived anthropomorphism, intelligence, and likeability are higher50. We examined three dimensions of linguistic adaptability as described below: Semantic Similarity measures how semantically coherent a response is to its query. We found that AI responses exhibited a 21.49% higher semantic similarity compared to OC responses with statistical significance (Cohen’s d = 0.52). This suggests that AI responses are more closely aligned with the specific content and intent of the query, demonstrating a greater ability to generate contextually relevant replies. Semantic Similarity measures how semantically coherent a response is to its query. We found that AI responses exhibited a 21.49% higher semantic similarity compared to OC responses with statistical significance (Cohen’s d = 0.52). This suggests that AI responses are more closely aligned with the specific content and intent of the query, demonstrating a greater ability to generate contextually relevant replies. Linguistic Style Accommodation measures how well a response aligns with the query’s linguistic style, focusing on similarities of non-content words51. We found that AI responses exhibited 9% higher linguistic style accommodation than OC responses (Cohen’s d = 0.76). This indicates that AI responses can linguistically accommodate with their queries, potentially enhancing their effectiveness in facilitating supportive and engaging interactions, similar to online communities7,51. Linguistic Style Accommodation measures how well a response aligns with the query’s linguistic style, focusing on similarities of non-content words51. We found that AI responses exhibited 9% higher linguistic style accommodation than OC responses (Cohen’s d = 0.76). This indicates that AI responses can linguistically accommodate with their queries, potentially enhancing their effectiveness in facilitating supportive and engaging interactions, similar to online communities7,51. Diversity/Creativity refers to the uniqueness and variation of a response compared to others. Greater diversity indicates more variation in language use. We measured diversity using cosine distances from the centroid of the response set, based on word embeddings. We found that AI responses had 57% lower cosine distance from the centroid compared to OC responses, with statistically significant differences (Cohen’s d=−1.21, t=−90.92, p<0.001). This observed difference in linguistic diversity suggests that, while AI may generate relevant and coherent responses, it tends to reuse the content across several responses, potentially indicating a lack of creativity in addressing individual concerns. On the other hand, online community members are likely to provide “out-of-the-box” suggestions based on their lived experiences. Diversity/Creativity refers to the uniqueness and variation of a response compared to others. Greater diversity indicates more variation in language use. We measured diversity using cosine distances from the centroid of the response set, based on word embeddings. We found that AI responses had 57% lower cosine distance from the centroid compared to OC responses, with statistically significant differences (Cohen’s d=−1.21, t=−90.92, p<0.001). This observed difference in linguistic diversity suggests that, while AI may generate relevant and coherent responses, it tends to reuse the content across several responses, potentially indicating a lack of creativity in addressing individual concerns. On the other hand, online community members are likely to provide “out-of-the-box” suggestions based on their lived experiences. Finally, we measured social support. Social support plays a crucial role in mitigating psychological distress, acting as a protective buffer against mental health challenges18,52. Online support-seeking has proven effective in reducing depression and enhancing self-efficacy and quality of life53. Again, in the context of suicide, social support within Reddit communities may lower the risk of future suicidal ideation49. According to the Social Support Behavioral Code54, two key forms of support, which have also received significant empirical and theoretical attention, are emotional support (ES) and informational support (IS). Both of these forms of support prevalent and effective in online interactions7,55. We found that AI responses showed significantly higher social support–−62% higher ES and 20% higher IS—than OC responses. This suggests that AI can generate responses that appear more supportive. However, unlike AI, online community members often engage in follow-up discussions or general commentary, which may not always directly convey support. Robustness of findings and additional insights To ensure the robustness of our findings and gain further insights, we conducted two additional analyses: 1) a qualitative evaluation, 2) an expert clinical evaluation of factual accuracy and safety in AI responses, and 3) a comparison of responses from state-of-the-art LLMs. First, we qualitatively analyzed a sample of 50 posts and their corresponding OC and AI responses in our dataset. We noted a large number of posts in which individuals struggling with mental health concerns—or their caregivers—sought informational support. We describe the key themes below: AI responses lack personal narratives, unlike OC responses. Although most of the posts sought advice, some sought connections with others who had similar lived experiences or specifically valued advice rooted in shared experiences. In responses, while OC members often shared personal stories, either to alleviate their own emotional burdens or to contribute to the greater good, AI provided results that were a rich source of information. Across these posts, we identified key themes highlighting similarities and differences between responses from AI and OC members. AI responses lack personal narratives, unlike OC responses. Although most of the posts sought advice, some sought connections with others who had similar lived experiences or specifically valued advice rooted in shared experiences. In responses, while OC members often shared personal stories, either to alleviate their own emotional burdens or to contribute to the greater good, AI provided results that were a rich source of information. Across these posts, we identified key themes highlighting similarities and differences between responses from AI and OC members. AI’s responses are structured, whereas OC responses involve more conversational engagement. We found that AI responses were formal and well-structured. In most of the responses, AI first acknowledged the challenges faced by the user and provided both informational and emotional support. In contrast, the responses from OC members followed a conversational style—with bidirectional communication, where the other members asked clarification and follow-up questions to provide more informed responses. Additionally, the original poster would often express gratitude and acknowledgment after receiving the support. AI’s responses are structured, whereas OC responses involve more conversational engagement. We found that AI responses were formal and well-structured. In most of the responses, AI first acknowledged the challenges faced by the user and provided both informational and emotional support. In contrast, the responses from OC members followed a conversational style—with bidirectional communication, where the other members asked clarification and follow-up questions to provide more informed responses. Additionally, the original poster would often express gratitude and acknowledgment after receiving the support. AI responses consist of standardized guidance, whereas community responses are based on personalized experiences. Along the lines of the above, we noted that AI is likely to provide a standardized set of guidance across multiple posts, such as asking to “consult a healthcare professional or a therapist” or “joining support groups. This echoes the findings from our quantitative analyses on the lack of diversity across responses. In contrast, online community members often elaborated on their experience and implicitly provided advice by detailing their journey with the struggles—leading to higher diversity across responses based on distinct experiences of individuals. Members also acknowledged the struggles of the original poster and wished well for them. Overall, AI responses typically presented suggestions in bullet points, often reiterating similar phrases and words, whereas OC comments offered more nuanced insights, often elaborating on a single suggestion through personal experience. This contributed to greater verbosity and repeatability in AI responses, as also observed in our quantitative analyses. AI responses consist of standardized guidance, whereas community responses are based on personalized experiences. Along the lines of the above, we noted that AI is likely to provide a standardized set of guidance across multiple posts, such as asking to “consult a healthcare professional or a therapist” or “joining support groups. This echoes the findings from our quantitative analyses on the lack of diversity across responses. In contrast, online community members often elaborated on their experience and implicitly provided advice by detailing their journey with the struggles—leading to higher diversity across responses based on distinct experiences of individuals. Members also acknowledged the struggles of the original poster and wished well for them. Overall, AI responses typically presented suggestions in bullet points, often reiterating similar phrases and words, whereas OC comments offered more nuanced insights, often elaborating on a single suggestion through personal experience. This contributed to greater verbosity and repeatability in AI responses, as also observed in our quantitative analyses. AI tends to show a neutrality in stance. If the author wanted to learn more about a certain product or drug, AI provided both positive impact and negative side-effects, whereas the responses on OC were mostly one-sided depending on the commenter’s experience with the product. If the author asked for “experiences,” then the responses from AI included an acknowledgment that “As an AI, I don’t have personal experiences,” and then it provided positive and negative side-effects based on its training data. In contrast, the OC members included a stance that was complemented by an explanation for their stance. For example, in a post asking about “experiences with Lexapro”, while one user responded “Lexapro did not work for them,” another responded “it worked for me because it helped stop my hot flashes and tingles that I would get at heated moments”. AI tends to show a neutrality in stance. If the author wanted to learn more about a certain product or drug, AI provided both positive impact and negative side-effects, whereas the responses on OC were mostly one-sided depending on the commenter’s experience with the product. If the author asked for “experiences,” then the responses from AI included an acknowledgment that “As an AI, I don’t have personal experiences,” and then it provided positive and negative side-effects based on its training data. In contrast, the OC members included a stance that was complemented by an explanation for their stance. For example, in a post asking about “experiences with Lexapro”, while one user responded “Lexapro did not work for them,” another responded “it worked for me because it helped stop my hot flashes and tingles that I would get at heated moments”. AI responses have boundaries in providing experience-based support. In OMHCs, individuals often seek advice based on others’ personal experiences with similar symptoms or treatments, such as “Has anyone taken Zoloft? Any advice would help.” While AI generated a response based on summarizing reviews from its training data, it did not directly address the query, stating, “I’m an AI and can provide general advice based on available information, but everyone’s experience with medication may vary [..].” Also, in posts where authors provided detailed accounts of their challenges and sought general advice on how to navigate life while managing their mental health struggles, AI was unable to provide any suggestions. It responded with “I’m unable to provide the help that you need” and recommended that the author talk to a healthcare professional or a family member. AI responses have boundaries in providing experience-based support. In OMHCs, individuals often seek advice based on others’ personal experiences with similar symptoms or treatments, such as “Has anyone taken Zoloft? Any advice would help.” While AI generated a response based on summarizing reviews from its training data, it did not directly address the query, stating, “I’m an AI and can provide general advice based on available information, but everyone’s experience with medication may vary [..].” Also, in posts where authors provided detailed accounts of their challenges and sought general advice on how to navigate life while managing their mental health struggles, AI was unable to provide any suggestions. It responded with “I’m unable to provide the help that you need” and recommended that the author talk to a healthcare professional or a family member. Despite the guardrails, AI can hallucinate. We noted a key strength that AI can often accurately recognize the abbreviations used in posts related to mental health disorders (e.g., BPD for Borderline Personality Disorder). In addition, we noted that the AI responses are often curated with guardrails to caution the user, such as, “it’s important to remember that I’m an AI and not a professional” and to “discuss any plans with a healthcare provider.” Adding such warning statements ensures that users only use the advice from AI to complement other resources combating problems with misleading and inaccurate responses. These patterns plausibly stem from the extensive moderation and red-teaming that state-of-the-art LLMs undergo before deployment. However, we also noted examples of hallucinations in AI responses. For example, one user was looking for suggestions to get their habit of “digging out ingrown hairs from their own legs” in control, where AI first responded with “congratulations on getting your face skin picking under control.” Here, face skin picking was nowhere mentioned in the original post, therefore, such responses could be inaccurate and problematic. Despite the guardrails, AI can hallucinate. We noted a key strength that AI can often accurately recognize the abbreviations used in posts related to mental health disorders (e.g., BPD for Borderline Personality Disorder). In addition, we noted that the AI responses are often curated with guardrails to caution the user, such as, “it’s important to remember that I’m an AI and not a professional” and to “discuss any plans with a healthcare provider.” Adding such warning statements ensures that users only use the advice from AI to complement other resources combating problems with misleading and inaccurate responses. These patterns plausibly stem from the extensive moderation and red-teaming that state-of-the-art LLMs undergo before deployment. However, we also noted examples of hallucinations in AI responses. For example, one user was looking for suggestions to get their habit of “digging out ingrown hairs from their own legs” in control, where AI first responded with “congratulations on getting your face skin picking under control.” Here, face skin picking was nowhere mentioned in the original post, therefore, such responses could be inaccurate and problematic. Next, to assess the factual correctness and potential harmfulness of AI-generated mental health responses, we conducted an expert evaluation by our clinical psychologist coauthor. A randomly sampled subset of 100 AI-generated responses was reviewed across four dimensions: factual accuracy, potential harmfulness, emotional attunement, and contextual responsiveness to users’ expressed mental health concerns. All of these dimensions were rated on a scale of 1–5, with higher values indicating a stronger presence of the rated quality. Table 3 reports a summary of the expert ratings. Overall, the expert evaluation found that AI-generated responses demonstrated consistently high factual accuracy (mean=5.00), with no identified instances of clinically incorrect or unsafe information. The responses primarily relied on general psychoeducational content rather than individualized clinical recommendations, indicating strong informational reliability but limited contextual specificity. Overall, the expert evaluation found that AI-generated responses demonstrated consistently high factual accuracy (mean=5.00), with no identified instances of clinically incorrect or unsafe information. The responses primarily relied on general psychoeducational content rather than individualized clinical recommendations, indicating strong informational reliability but limited contextual specificity. With respect to potential harmfulness (mean=1.14), no direct harmful advice or unsafe recommendations were observed. However, several indirect risk factors were identified, including limited acknowledgment of emotional distress in posts describing anxiety, intrusive thoughts, or interpersonal conflict; reliance on emotionally nonspecific reassurance where more tailored validation may have been beneficial; and the absence of explicitly risk-sensitive or safety-oriented language in posts that could carry implications for well-being. While these limitations do not constitute direct harm, they may introduce vulnerabilities if AI-generated guidance is interpreted without professional supervision or taken out of the appropriate clinical context. With respect to potential harmfulness (mean=1.14), no direct harmful advice or unsafe recommendations were observed. However, several indirect risk factors were identified, including limited acknowledgment of emotional distress in posts describing anxiety, intrusive thoughts, or interpersonal conflict; reliance on emotionally nonspecific reassurance where more tailored validation may have been beneficial; and the absence of explicitly risk-sensitive or safety-oriented language in posts that could carry implications for well-being. While these limitations do not constitute direct harm, they may introduce vulnerabilities if AI-generated guidance is interpreted without professional supervision or taken out of the appropriate clinical context. The weakest ratings emerged in the dimensions of emotional attunement (mean = 1.82) and contextual responsiveness (mean = 2.64). AI responses frequently relied on formulaic supportive phrasing or distancing language (e.g., references to being an AI system) and often failed to integrate the emotional nuance or situational details present in users’ narratives. Consequently, responses tended to be accurate and polite but insufficiently personalized or emotionally aligned to meet the standards of high-quality mental health support. The weakest ratings emerged in the dimensions of emotional attunement (mean = 1.82) and contextual responsiveness (mean = 2.64). AI responses frequently relied on formulaic supportive phrasing or distancing language (e.g., references to being an AI system) and often failed to integrate the emotional nuance or situational details present in users’ narratives. Consequently, responses tended to be accurate and polite but insufficiently personalized or emotionally aligned to meet the standards of high-quality mental health support. Table 3Summary of expert clinical ratings for AI-generated responses (N=100)DimensionNMeanSD% ≥ 4Factual Accuracy1005.000.00100%Potential Harmfulness1001.140.350%Emotional Attunement (Empathy)1001.820.990%Contextual Responsiveness (Addressing)1002.640.670%Ratings were on a 1–5 scale, with higher values indicating stronger performance: greater accuracy, safety, emotional attunement, and contextual responsiveness. Summary of expert clinical ratings for AI-generated responses (N=100) Ratings were on a 1–5 scale, with higher values indicating stronger performance: greater accuracy, safety, emotional attunement, and contextual responsiveness. Therefore, this expert review indicates that while AI-generated responses appear factually reliable and broadly safe, their limitations in emotional depth and contextual engagement underscore the importance of continued human clinical involvement when evaluating and deploying AI-based mental health support tools. Finally, having conducted our analyses with GPT-4-Turbo, we also experimented with other general-purpose LLMs, particularly Llama-3.1 and Mistral-7B. These three LLMs cover a spectrum of models that vary in architecture, training dataset, and optimization methods. We summarize the psycholinguistic and lexico-semantic comparison of these models in Table 4 and Table 5 respectively, including paired t tests compared with OC responses, and Kruskal-Wallis H test across all four modalities (OC, GPT, Llama, and Mistral). We notice that the trends in comparisons (by t-test) are very similar for all three LLMs. Table 4Summary of comparison of Reddit and AI (for GPT-4-Turbo, Llama-3, and Mistral-7B) responses in terms of psycholinguistic categories as per Linguistic Inquiry and Word Count (LIWC84), along with Cohen’s d, paired t-tests, and Kolmogorov-Smirnov (KS)-testOCGPTLlamaMistralCategoriesMeanMeant-testMeant-testMeant-testH-stat.AffectPos. Affect0.0380.032−13.59***0.027−25.2***0.035−6.29***7863.22***Neg. Affect0.00.0−3.09***0.0−2.19***0.0−2.67***43.21***Anxiety0.0050.0069.75***0.00720.07***0.00721.73***24,817.71***Anger0.0020.0−22.53***0.0−20.47***0.0−20.98***1250.36***Sad0.0050.00830.03***0.003−17.63***0.004−10.42***19,752.18***Cognition and PerceptionInsight0.0210.0239.76***0.02418.23***0.02943.76***14,517.03***Causation0.0110.006−35.44***0.008−21.88***0.009−14.44***7082.33***Discrep.0.0150.014−5.68***0.009−34.87***0.013−13.17***5585.53***Tentat.0.0280.0292.72***0.03215.45***0.03947.71***10,975.77***Certainty0.0120.005−32.11***0.007−21.57***0.007−20.84***71,13.61***Differ.0.0320.05488.92***0.0336.29***0.03617.36***10,695.8***Percept0.0170.017−0.94***0.014−13.58***0.015−7.13***7416.95***See0.0050.002−29.41***0.002−21.77***0.002−24.31***6858.96***Hear0.0040.003−6.97***0.002−19.07***0.003−4.15***19,374.27***Feel0.0080.01227.91***0.0096.66***0.00910.23***25,835.34***Social & Personal ConcernsFamily0.0020.001−15.2***0.002−1.78***0.002−3.52***24,678.24***Friend0.0020.001−15.01***0.002−1.17***0.002−0.93***21,835.96***Female0.0040.002−20.56***0.002−18.46***0.002−18.42***583.94***Male0.0040.001−25.3***0.002−23.59***0.001−26.13***840.37***Work0.0060.0061.19***0.0136.34***0.01148.55***40,897.5***Leisure0.0020.001−10.9***0.0021.69***0.002−0.78***32,357.44***Home0.0010.001−10.0***0.00215.8***0.0028.96***40,369.15***Money0.0010.001−7.13***0.0010.57***0.0013.82***16,647.94***Religion0.0010.0−10.7***0.0−10.25***0.0−10.64***587.1***Death0.0010.0−9.47***0.0−7.8***0.0−6.73***574.1***Relativity0.0760.051−57.97***0.053−53.66***0.057−43.02***6968.49***Motion0.0110.005−35.13***0.009−13.43***0.009−10.49***9868.06***Space0.0370.034−9.84***0.03−24.82***0.033−12.44***521.14***Time0.0290.011−66.03***0.014−56.99***0.015−52.44***4374.67***Drives0.0460.0639.31***0.046−0.32***0.05629.39***11,091.23***Affiliation0.0130.01615.59***0.01717.52***0.01927.18***29,398.41***Achievement0.010.01212.47***0.0117.97***0.01317.55***21,406.99***Power0.0110.031132.96***0.01741.61***0.02386.98***52,883.29***Reward0.0140.007−37.4***0.009−26.83***0.01−20.38***5844.18***Risk0.0040.00836.32***0.0042.21***0.0042.04***23,790.9***OCGPTLlamaMistralCategoriesMeanMeant-testMeant-testMeant-testH-stat.Biological ProcessesBody0.0030.002−8.31***0.002−2.62***0.003−1.2***13,152.44***Health0.0060.01990.23***0.00816.76***0.0129.89***47,840.27***Sexual0.0010.0−9.66***0.0−8.65***0.0−9.27***213.35***Ingest0.0010.001−5.52***0.001−4.81***0.0013.51***14,768.61***Function WordsArticle0.0360.04227.89***0.03810.12***0.0418.6***5024.7***Preposition0.090.10333.79***0.09614.94***0.11154.96***7867.38***Aux. Verb0.0740.09453.54***0.068−16.76***0.0814.64***10,565.96***Adverb0.0520.044−24.77***0.026−77.28***0.029−68.74***13,029.35***Conjunction0.0560.06634.15***0.06324.61***0.07566.18***12,180.34***Negation0.0150.008−29.72***0.01−23.09***0.008−29.98***2694.12***Verb0.1580.127−56.34***0.109−90.36***0.133−46.08***17,285.76***Adjective0.0410.05331.17***0.0422.98***0.04715.82***11,915.15***Compare0.0210.017−15.87***0.015−23.7***0.016−19.38***2140.44***Interrog.0.0120.012−3.33***0.009−19.13***0.008−27.73***8116.9***Number0.0040.001−31.81***0.002−25.11***0.002−20.11***8637.13***Quantifier0.0180.009−46.03***0.01−39.79***0.013−25.59***2721.46***Interpersonal Focus (Pronouns)1st P. Sin.0.0580.017−107.9***0.007−135.27***0.008−134.79***31,931.33***1st P. Plu.0.0030.001−27.32***0.001−25.27***0.001−22.31***18,31.54***2nd P.0.0340.04737.73***0.05252.42***0.05253.81***26,344.5***3rd P. Sin.0.0060.003−26.48***0.003−25.27***0.003−26.86***650.38***3rd P. Plu.0.0050.005−0.75***0.004−8.43***0.0056.27***28,430.59***Impersonal Prn.0.050.06545.87***0.041−28.58***0.050.4***8796.2***Temporal ReferencesPast Focus0.0350.013−53.59***0.01−61.2***0.013−54.44***1032.16***Present Focus0.1090.12633.72***0.097−23.47***0.11410.55***7961.49***Future Focus0.010.006−28.18***0.007−19.38***0.0114.25***14,879.24***InformalSwear0.0010.0−15.34***0.0−14.98***0.0−15.34***2971.87***Netspeak0.0070.0−28.6***0.0−28.12***0.0−28.68***8091.97***Assent0.0080.003−14.44***0.002−17.82***0.002−17.62***10,680.26***Nonfluent0.0020.0−19.56***0.001−14.34***0.002−8.62***13,688.5***Filler0.00.0−8.08***0.0−8.09***0.0−8.07***834.13***p−values reported after Bonferroni correction. (* p<0.05, ** p<0.01, *** p<0.001). Summary of comparison of Reddit and AI (for GPT-4-Turbo, Llama-3, and Mistral-7B) responses in terms of psycholinguistic categories as per Linguistic Inquiry and Word Count (LIWC84), along with Cohen’s d, paired t-tests, and Kolmogorov-Smirnov (KS)-test p−values reported after Bonferroni correction. (* p<0.05, ** p<0.01, *** p<0.001). Table 5Summary of comparing the responses on Reddit and by multiple LLMs: GPT-4-Turbo, Llama-3.1, and Mistral-7B, including paired t-tests in comparison with Reddit responses, and a Kruskal-Wallis H-test across all the four modalities—Reddit, GPT, Llama, and Mistral (*p < 0.05, **p < 0.01, ***p < 0.001)OCGPTLlamaMistralCategoriesMeanMeant-testMeant-testMeant-testH-stat.VerbosityWords Per Response77.35160.3569.25***470.55306.3***332.03308.89***51,336.42***Words Per Sentence13.7619.2819.14***18.1939.86***20.1195.78***14,141.34***Linguistic StructureReadability6.5811.1879.13***10.1149.99***10.149.75***24,012.01***Repeatability0.20.3395.54***0.53269.3***0.48277.59***50,929.85***Complexity3.314.6383.52***4.1788.87***4.39110.81***26,327.51***Linguistic StyleCategorical Dynamic Index (CDI)6.99.6633.04***15.87117.63***14.097.92***17,245.87***Formality0.670.87107.43***0.03-299.46***0.04−292.68***72,616.46***Empathy0.710.8469.19***0.03-307.72***0.03−307.23***67,449.95***Politeness0.670.7963.52***0.03-307.47***0.03−308.03***68,856.98***Adaptability to QuerySemantic Similarity0.590.7163.13***0.7583.13***0.7687.94***7331.63***Linguistic Style Accommodation0.710.7723.28***0.8145.56***0.8351.44***3731.16***Diversity/Creativity0.130.06−103.37***0.09-35.11***0.08−44.79***13,080.62***Social SupportEmotional Support0.490.7989.9***0.82101.99***0.86124.4***21,035.19***Informational Support0.520.6225.69***0.94150.25***0.96166.63***30,201.57*** Summary of comparing the responses on Reddit and by multiple LLMs: GPT-4-Turbo, Llama-3.1, and Mistral-7B, including paired t-tests in comparison with Reddit responses, and a Kruskal-Wallis H-test across all the four modalities—Reddit, GPT, Llama, and Mistral (*p < 0.05, **p < 0.01, ***p < 0.001) Discussion While AI chatbots have shown initial promise, their effectiveness as mental health support tools remains largely untested against the organic and nuanced human interactions that develop in online mental health communities (OMHCs). This study aimed to assess how AI-generated responses compare to human-written responses in OMHCs in terms of linguistic features spanning psycholinguistics, linguistic structure, style, adaptability to query, and social support. We conducted our study using 24,114 posts collected from 55 OMHCs on Reddit. We used these posts as queries to state-of-the-art AI chatbots such as GPT-4-Turbo, Llama-3.1, and Mistral-7B, and compared the AI responses to 138,758 human-written responses in these OMHCs. Our analysis revealed that AI responses were more verbose, readable, and complex, indicating they might be somewhat harder to comprehend than human-written responses. AI responses tended to be more formal and structured, demonstrating higher levels of empathy and politeness. Notably, AI responses exhibited a predominantly analytical linguistic style, marked by greater use of articles, prepositions, and auxiliary verbs. In contrast, human responses followed a more narrative-driven approach, incorporating personal disclosures and solidarity expressions. Additionally, AI responses scored higher in semantic similarity and linguistic style accommodation, indicating a greater ability to tailor language to specific queries. However, despite their linguistic richness and supportive tone, AI-generated replies lacked diversity and creative, highlighting challenges in replicating the spontaneous, experience-based advice commonly found in online mental health communities. By conducting a deeper qualitative analysis, we found support for our quantitative linguistic analyses, as well as uncovered a number of themes of language differences in AI and OC responses. For instance, AI responses tend to adopt a neutral stance of highlighting both the positives and negatives of a specific approach, whereas OC responses tend to take sides. Further, people often look for prior experiences with a particular therapy or medication in OMHCs, but AI lacks such experiences and tends to recommend expert/clinical care to such queries. This study presents a computational approach based on natural language analysis to systematically evaluate the language used in AI-generated responses to mental health-related inquiries. These methods and the insights can guide the development of AI-assisted response writing in OMHCs, that offer personalized, empathetic, and timely interventions, ensuring they effectively meet the emotional and informational needs of individuals seeking help and advice in these spaces. Additionally, this computational framework can help identify patterns in language use, shedding light on the types of practical assistance individuals seek in OMHCs when navigating mental health challenges. We list these implications below: It is essential to prioritize empathy, reliability, and transparency in AI. Our findings suggested that AI—in its current form—is more equipped to provide immediate assistance in the form of structured guidance, reinforcing its role as an informational and solution-oriented tool rather than one that shares personal experiences or future aspirations. This aligns with prior findings on LLMs’ abilities to provide empathetic responses11,14,16. These findings underscore the need to design AI tools that not only provide relevant information but also foster empathy. Unlike online community members who draw from personal experiences to offer emotional support, LLMs cannot replicate this depth of connection—as evident from our psycholinguistic and lexico-semantic analysis. The absence of a personal narrative and a sense of belonging in AI may lead to perceptions of “artificial” support, potentially diminishing its effectiveness. This opens up discussions on whether LLM-based tools should serve primarily as informational support agents rather than as emotional support providers. In fact, ideally, end users should have the option to prioritize or disable features based on their needs. Transparency is also crucial—without clear disclosure, users with limited AI/digital literacy may mistake LLM responses to be from humans, leading to ethical concerns. A notable example is the ethical backlash against Koko—a mental health chatbot—whose users felt misled upon realizing they were interacting with AI rather than human counselors56,57.To further elaborate, building on established distinctions between cognitive empathy, which refers to accurately recognizing and understanding another person’s emotional state, and emotional empathy, which involves resonating with and expressing affective alignment with that state58–60, our findings suggest that AI-generated responses primarily demonstrate cognitive empathy through surface-level acknowledgment, summarizing user concerns, and offering general supportive language. However, AI shows clearer limitations in emotional empathy, which requires attunement to the user’s unique emotional context, personalized validation, and relational engagement. Prior work has shown that polite or supportive wording alone is not sufficient for conveying genuine care or fostering meaningful emotional connection48,61. These nuances help explain why AI responses may appear supportive at the linguistic level yet feel emotionally flat or insufficiently responsive to users’ lived experiences11,25,62. It is essential to prioritize empathy, reliability, and transparency in AI. Our findings suggested that AI—in its current form—is more equipped to provide immediate assistance in the form of structured guidance, reinforcing its role as an informational and solution-oriented tool rather than one that shares personal experiences or future aspirations. This aligns with prior findings on LLMs’ abilities to provide empathetic responses11,14,16. These findings underscore the need to design AI tools that not only provide relevant information but also foster empathy. Unlike online community members who draw from personal experiences to offer emotional support, LLMs cannot replicate this depth of connection—as evident from our psycholinguistic and lexico-semantic analysis. The absence of a personal narrative and a sense of belonging in AI may lead to perceptions of “artificial” support, potentially diminishing its effectiveness. This opens up discussions on whether LLM-based tools should serve primarily as informational support agents rather than as emotional support providers. In fact, ideally, end users should have the option to prioritize or disable features based on their needs. Transparency is also crucial—without clear disclosure, users with limited AI/digital literacy may mistake LLM responses to be from humans, leading to ethical concerns. A notable example is the ethical backlash against Koko—a mental health chatbot—whose users felt misled upon realizing they were interacting with AI rather than human counselors56,57. To further elaborate, building on established distinctions between cognitive empathy, which refers to accurately recognizing and understanding another person’s emotional state, and emotional empathy, which involves resonating with and expressing affective alignment with that state58–60, our findings suggest that AI-generated responses primarily demonstrate cognitive empathy through surface-level acknowledgment, summarizing user concerns, and offering general supportive language. However, AI shows clearer limitations in emotional empathy, which requires attunement to the user’s unique emotional context, personalized validation, and relational engagement. Prior work has shown that polite or supportive wording alone is not sufficient for conveying genuine care or fostering meaningful emotional connection48,61. These nuances help explain why AI responses may appear supportive at the linguistic level yet feel emotionally flat or insufficiently responsive to users’ lived experiences11,25,62. As AI continues to evolve, it is crucial to question what would be the role of human support will be in OMHCs in the future. While some users turn to these platforms for advice or resources, many seek solidarity, emotional validation, or a safe space to express thoughts they cannot share with family, caregivers, or in their offline worlds. Although LLM-written responses can fulfill certain informational needs, they may fall short in fostering the sense of community and human connection that many users of OMHCs value. The sustainability of online communities depends on active and continuous user engagement. If support in these platforms is increasingly AI-based and is removed from the lived experience of people, these spaces risk losing their vitality, diminishing the peer support that makes them meaningful. That said, the act of sharing personal experiences—whether through discussions or expressive writing—can be therapeutic, helping individuals to process emotions, gain perspective, and feel less isolated37,63. While AI does not directly enable a similar benefit currently, it can be designed to encourage journaling, offering a private space for users to articulate their thoughts and still receive an interactive “talking-to-someone” experience as well as receive personalized prompts for self-care. However, for such interactions to be truly beneficial in OMHCs, they must be designed with user safety in mind. This also raises a critical question: How would the shift from human-led to AI-driven support impact mental health outcomes over time? Would it weaken the empathy and social bonds that communities provide? As AI continues to evolve, it is crucial to question what would be the role of human support will be in OMHCs in the future. While some users turn to these platforms for advice or resources, many seek solidarity, emotional validation, or a safe space to express thoughts they cannot share with family, caregivers, or in their offline worlds. Although LLM-written responses can fulfill certain informational needs, they may fall short in fostering the sense of community and human connection that many users of OMHCs value. The sustainability of online communities depends on active and continuous user engagement. If support in these platforms is increasingly AI-based and is removed from the lived experience of people, these spaces risk losing their vitality, diminishing the peer support that makes them meaningful. That said, the act of sharing personal experiences—whether through discussions or expressive writing—can be therapeutic, helping individuals to process emotions, gain perspective, and feel less isolated37,63. While AI does not directly enable a similar benefit currently, it can be designed to encourage journaling, offering a private space for users to articulate their thoughts and still receive an interactive “talking-to-someone” experience as well as receive personalized prompts for self-care. However, for such interactions to be truly beneficial in OMHCs, they must be designed with user safety in mind. This also raises a critical question: How would the shift from human-led to AI-driven support impact mental health outcomes over time? Would it weaken the empathy and social bonds that communities provide? Along the lines of the above, another pertinent question comes up: as we invest in AI-driven mental health tools, should we only prioritize efficiency over human connection, or can AI be designed to enhance, rather than replace, community-driven support? Although OMHCs provide platforms for individuals to share sensitive experiences and receive social support, they also present challenges such as delayed responses, exposure to online toxicity and antisocial behaviors (e.g., hateful speech, trolling, misinformation), privacy concerns, and stigmatization6,64. LLM based tools have the potential to mitigate some of these issues by offering immediate responses and interventions while allowing community members to provide more personalized, long-term support. However, such approaches to integrate generative AI in response writing in OMHCs is not without concerns. LLMs can reproduce inappropriate stereotypes65,66, present clinically unvalidated information67, miss cross-lingual or cross-cultural contexts68, and may fail to adequately understand the lived experience of an individual25,69. A hybrid human-AI model holds promise in bridging these gaps. AI can provide scalable, immediate assistance, and human community members can maintain the emotional depth and relational support that AI currently lacks. By carefully integrating AI, we can create a system where the technology supports and complements the core values of peer-driven mental health support. Along the lines of the above, another pertinent question comes up: as we invest in AI-driven mental health tools, should we only prioritize efficiency over human connection, or can AI be designed to enhance, rather than replace, community-driven support? Although OMHCs provide platforms for individuals to share sensitive experiences and receive social support, they also present challenges such as delayed responses, exposure to online toxicity and antisocial behaviors (e.g., hateful speech, trolling, misinformation), privacy concerns, and stigmatization6,64. LLM based tools have the potential to mitigate some of these issues by offering immediate responses and interventions while allowing community members to provide more personalized, long-term support. However, such approaches to integrate generative AI in response writing in OMHCs is not without concerns. LLMs can reproduce inappropriate stereotypes65,66, present clinically unvalidated information67, miss cross-lingual or cross-cultural contexts68, and may fail to adequately understand the lived experience of an individual25,69. A hybrid human-AI model holds promise in bridging these gaps. AI can provide scalable, immediate assistance, and human community members can maintain the emotional depth and relational support that AI currently lacks. By carefully integrating AI, we can create a system where the technology supports and complements the core values of peer-driven mental health support. Overall, as AI becomes increasingly integrated into mental health care, it is important to carefully address concerns related to biases, hallucinations, ethical dilemmas, over-reliance, and the tension between personalization and privacy25,70–72. For example, AI models trained on existing datasets may inadvertently reinforce cultural, racial, and gender biases, leading to inequitable support or harmful moderation practices13,73–75. Additionally, AI hallucinations (as also noted in our qualitative examination)—the generation of misleading, irrelevant, or false information67—can be particularly dangerous in mental health contexts, potentially guiding vulnerable users toward harmful decisions. More broadly, our mixed findings regarding factual accuracy but limited emotional attunement underscore the need for caution in framing LLMs as replacements for human support. AI-generated responses may appear linguistically supportive while lacking deeper emotional resonance, which could create unmet expectations or false reassurance if they are used without appropriate human oversight.Accountability is another critical issue—when an AI offers harmful advice, who is responsible? Should the onus fall on developers, platform administrators, OMHC moderators and community members, or the AI system itself for the harm caused by automated decisions? Furthermore, while AI’s ability to provide personalized support can greatly enhance user experiences, it also raises privacy concerns. Should AI analyze sensitive user data, often shared in OMHC postings to tailor responses, or should its adaptability be limited through strict privacy safeguards? It is important that users retain control over how much personal information AI systems can access and should have the option to opt out of AI-driven personalization. To address these concerns, robust regulation and oversight frameworks are necessary76–78. Such frameworks should emphasize transparency regarding system limitations, mechanisms for continuous auditing of harmful or biased outputs, and clear accountability structures—particularly for high-stakes domains such as mental health support.These should ensure that AI-driven mental health support tools adhere to ethical standards, protect user privacy, and mitigate risks associated with biases and misinformation. In line with these concerns, responsible deployment practices, transparency about system limitations, and continued involvement of experts, clinicians, and community stakeholders are essential when designing, evaluating, and integrating AI-mediated mental health support tools. Striking a balance between efficiency, human connection, and ethical responsibility is crucial to ensuring that AI enhances rather than undermines mental health support in online communities. Overall, as AI becomes increasingly integrated into mental health care, it is important to carefully address concerns related to biases, hallucinations, ethical dilemmas, over-reliance, and the tension between personalization and privacy25,70–72. For example, AI models trained on existing datasets may inadvertently reinforce cultural, racial, and gender biases, leading to inequitable support or harmful moderation practices13,73–75. Additionally, AI hallucinations (as also noted in our qualitative examination)—the generation of misleading, irrelevant, or false information67—can be particularly dangerous in mental health contexts, potentially guiding vulnerable users toward harmful decisions. More broadly, our mixed findings regarding factual accuracy but limited emotional attunement underscore the need for caution in framing LLMs as replacements for human support. AI-generated responses may appear linguistically supportive while lacking deeper emotional resonance, which could create unmet expectations or false reassurance if they are used without appropriate human oversight. Accountability is another critical issue—when an AI offers harmful advice, who is responsible? Should the onus fall on developers, platform administrators, OMHC moderators and community members, or the AI system itself for the harm caused by automated decisions? Furthermore, while AI’s ability to provide personalized support can greatly enhance user experiences, it also raises privacy concerns. Should AI analyze sensitive user data, often shared in OMHC postings to tailor responses, or should its adaptability be limited through strict privacy safeguards? It is important that users retain control over how much personal information AI systems can access and should have the option to opt out of AI-driven personalization. To address these concerns, robust regulation and oversight frameworks are necessary76–78. Such frameworks should emphasize transparency regarding system limitations, mechanisms for continuous auditing of harmful or biased outputs, and clear accountability structures—particularly for high-stakes domains such as mental health support. These should ensure that AI-driven mental health support tools adhere to ethical standards, protect user privacy, and mitigate risks associated with biases and misinformation. In line with these concerns, responsible deployment practices, transparency about system limitations, and continued involvement of experts, clinicians, and community stakeholders are essential when designing, evaluating, and integrating AI-mediated mental health support tools. Striking a balance between efficiency, human connection, and ethical responsibility is crucial to ensuring that AI enhances rather than undermines mental health support in online communities. That said, this study has limitations that suggest interesting future directions. Our findings are not generalizable to the entire population or all generative AI applications, as the data is biased by self-selection—only users who chose to participate in online communities were included. The user base and queries for LLMs in mental health self-management may differ, highlighting the need to examine the representativeness of diverse user groups. Additionally, the study’s design, particularly how we prompted the LLMs, limits the scope of our findings. Responses may vary based on different and more sophisticated prompts, and we did not explore interactive, back-and-forth conversations. Our analyses focus on single-response interactions, consistent with the reply-based structure of online mental health communities. However, future work should investigate multi-turn conversational dynamics to better understand how empathy, personalization, trust, and safety evolve across sustained human-AI interactions. Furthermore, our study examined the linguistic structure, syntax, and semantics of AI responses, but did not evaluate the accuracy of AI-generated information or explore how individuals might collaborate with AI. To address this gap, we conducted an expert clinical evaluation of a subset of AI-generated responses. While responses were found to be factually reliable and free of direct harmful guidance, ratings showed lower emotional attunement and contextual responsiveness. These findings indicate that safety in mental health communication extends beyond informational correctness and also requires emotional sensitivity and situational alignment, and that the absence of these qualities may pose indirect risks if AI-generated guidance is used without human oversight. Future work should include user-centered studies and larger-scale clinical audits to further evaluate how emotional alignment, risk signaling, and adaptive responsiveness shape the real-world effectiveness of AI tools. Our automated linguistic measures operate primarily at the sentence and response levels rather than as direct length-based counts, and responses were normalized within the analytic pipeline to reduce systematic bias from verbosity. However, response length may still influence some measured attributes, and future work can incorporate explicit length-matching or additional statistical controls to further isolate stylistic and empathetic differences post length. Further, our use of automated empathy and support measures captures linguistic proxies of support rather than users’ perceived helpfulness or emotional impact. While our expert evaluation provides complementary human judgment, broader clinical and end-user assessments are needed to fully validate these metrics. Importantly, readability metrics such as the Coleman-Liau Index should be interpreted as indicators of surface-level linguistic complexity and informational density—derived from sentence length and word length—rather than direct measures of reader comprehension, emotional accessibility, or therapeutic effectiveness, particularly in the context of mental health support communication. This study motivates further investigation into the perceptions and effectiveness of such human–AI interactions, and future research could include user studies to evaluate the reliability of AI interactions and investigate user perceptions of LLM responses to health queries, thereby enhancing our understanding of how LLMs can effectively support mental health care. An open question remains about the role of AI in mental health support—is it a friend, a peer supporter, a therapist, or merely a tool for providing information? This definition and its interpretation will likely vary among individuals, influencing the perceived effectiveness of AI in mental health care. Methods Data and study design We sourced our data from Reddit. Reddit is a popular social platform consisting of online communities, called subreddits, which are dedicated to specific themes of discussions. Prior work has noted that pseudonymity, community-driven moderation, and asynchronous peer support on Reddit help individuals overcome stigma and candidly self-disclose their sensitive mental health concerns and seek social support from community members1,6,55. Based on prior work7,79, we identified 55 subreddits dedicated to mental health discussions (e.g., r/depression, r/anxiety, r/SuicideWatch, etc.), and collected their posts and responses between January 01, 2018, and March 31, 2024—leading to our online communities (OC) dataset of 24,114 posts and 138,758 responses. For each of these 24,114 posts, we queried OpenAI’s GPT-4-Turbo model using the OpenAI API with the post body. In addition, we also deployed the open-source models Llama-3.1 and Mistral-7B locally and prompted them with these posts to obtain a diverse set of AI-generated responses to ensure that our findings are generalizable and applicable to a broad range of LLMs. Across all models, we adopted a zero-shot prompting strategy without any additional system instructions, role prompts, safety framing, or stylistic guidance. Responses were generated using default inference parameters, allowing us to attribute observed linguistic differences to model behavior rather than to prompt manipulation or tuning effects. We also deeply considered the ethics of our research. Although all data were drawn from publicly accessible Reddit posts, the content frequently includes highly sensitive personal disclosures. To minimize the risk of re-identification, misrepresentation, or unintended harm, we avoid publicly sharing verbatim posts and comments and report only aggregate-level results. No attempts were made to identify or contact individual users, and all analyses were conducted in accordance with ethical guidance for social media research. Analytic approach To quantify the differences in linguistic measures between AI and OC responses, we obtained the effect size (Cohen’s d) and evaluated statistical significance through paired t-test and Kolmogorov−Smirnov (KS) test. All paired statistical analyses were conducted at the post level using a post-level aggregation strategy. For each post, we computed the mean value of each linguistic measure across all available OC replies, leading to a single aggregate OC score per post. This aggregated OC score was then paired with the corresponding AI response for that same post, obtaining one matched OC–AI observation per post for paired statistical tests. This procedure ensures independence of observations by accounting for multiple human responses within posts. Our metrics are inspired by a rich body of prior work in the space of online language and psychotherapy33,37,48,80–82. A rich body of research has revealed the strong connection between language and psychosocial dynamics28,30,83. The effectiveness of interpersonal interactions often hinges on psycholinguistic markers, which shape self-disclosure and the support exchanged28. To analyze differences in these markers between AI and OC responses, we leveraged the well-validated Linguistic Inquiry and Word Count (LIWC) lexicon84. LIWC categorizes language into over 60 psycholinguistic attributes, broadly categorized into eight dimensions—1) Affect, 2) Cognition and Perception, 3) Social and Personal Concerns, 4) Biological Processes, 5) Function Words, 6) Interpersonal Focus, 7) Temporal References, and 8) Informal Language. We used the LIWC lexicon to obtain the normalized occurrences of these psycholinguistic attributes in AI and OC responses. Further, we operationalized and measured several lexico-semantic dimensions, and we describe our approach below: Verbosity. Response length, or verbosity, has been associated with perceived quality and emotional richness in supportive interactions33,34. To quantify verbosity, we employed two measures: (1) response-level verbosity, defined as the total number of words per response, and (2) sentence-level verbosity, measured by the average number of words per sentence. Higher values indicate more elaborate responses, whereas lower values suggest conciseness. Verbosity. Response length, or verbosity, has been associated with perceived quality and emotional richness in supportive interactions33,34. To quantify verbosity, we employed two measures: (1) response-level verbosity, defined as the total number of words per response, and (2) sentence-level verbosity, measured by the average number of words per sentence. Higher values indicate more elaborate responses, whereas lower values suggest conciseness. Readability. Readability measures the ease with which a reader can engage with the surface structure of a text85. Readability has been established as an important measure within health and online health contexts, both in terms of self-expression33,37 as well as others’ interpretation and comprehension35,36. We adopted the Coleman-Liau Index (CLI)85,86 which assesses readability based on character and word structure within a sentence, calculated as, CLI = (0.0588L-0.296S-15.8), where L is the average number of letters per 100 words, and S is the average number of sentences per 100 words. While higher CLI values are linked to better writing quality, and are conventionally interpreted as corresponding to higher grade-level reading demands, they should not be treated as direct measures of reader comprehension. Readability. Readability measures the ease with which a reader can engage with the surface structure of a text85. Readability has been established as an important measure within health and online health contexts, both in terms of self-expression33,37 as well as others’ interpretation and comprehension35,36. We adopted the Coleman-Liau Index (CLI)85,86 which assesses readability based on character and word structure within a sentence, calculated as, CLI = (0.0588L-0.296S-15.8), where L is the average number of letters per 100 words, and S is the average number of sentences per 100 words. While higher CLI values are linked to better writing quality, and are conventionally interpreted as corresponding to higher grade-level reading demands, they should not be treated as direct measures of reader comprehension. Repeatability and Complexity. Repeatability and complexity are syntactic measures that capture the richness and depth of expression in communication, and are linked to one’s cognitive state through planning, execution, and memory37,38. Repeatability accounts for the frequency with which words are repeated or reused in a piece of text. A higher degree of repetition is often associated with lower linguistic diversity and reduced content quality. Complexity shapes the nature of communication87, as a more linguistically complex text tends to convey greater nuance, precision, and depth in expressing ideas or information. Drawing on prior work33,37,88, we measured repeatability as the normalized occurrence of non-unique words and complexity as the average length of words per sentence. Repeatability and Complexity. Repeatability and complexity are syntactic measures that capture the richness and depth of expression in communication, and are linked to one’s cognitive state through planning, execution, and memory37,38. Repeatability accounts for the frequency with which words are repeated or reused in a piece of text. A higher degree of repetition is often associated with lower linguistic diversity and reduced content quality. Complexity shapes the nature of communication87, as a more linguistically complex text tends to convey greater nuance, precision, and depth in expressing ideas or information. Drawing on prior work33,37,88, we measured repeatability as the normalized occurrence of non-unique words and complexity as the average length of words per sentence. Categorical-Dynamic Index (CDI). Language style can be conceptualized as existing on a continuum between categorical and dynamic modes40. Categorical language reflects a more structured, analytical approach, akin to an “amateur scientist” style, where the focus is on logically organized, abstract concepts and detailed categorization. In contrast, dynamic language is more fluid and personal, commonly seen in socially engaged individuals who convey stories and reflect more on their immediate environment, incorporating personal narratives and emotional expressions. This spectrum of language use is captured by the Categorical-Dynamic Index (CDI), a bipolar measure that quantifies the balance between these two styles. A higher CDI score corresponds to a more categorical style, while a lower score indicates a dynamic, narrative-driven approach40. The CDI is calculated based on the frequency of specific style-related parts of speech in a given text, with the formula: CDI = (30 + article + preposition − personal pronoun − impersonal pronoun − aux. verb − conjunction − adverb − negation). To calculate the above, we obtained the occurrences of these parts of speech using the LIWC lexicon84. Categorical-Dynamic Index (CDI). Language style can be conceptualized as existing on a continuum between categorical and dynamic modes40. Categorical language reflects a more structured, analytical approach, akin to an “amateur scientist” style, where the focus is on logically organized, abstract concepts and detailed categorization. In contrast, dynamic language is more fluid and personal, commonly seen in socially engaged individuals who convey stories and reflect more on their immediate environment, incorporating personal narratives and emotional expressions. This spectrum of language use is captured by the Categorical-Dynamic Index (CDI), a bipolar measure that quantifies the balance between these two styles. A higher CDI score corresponds to a more categorical style, while a lower score indicates a dynamic, narrative-driven approach40. The CDI is calculated based on the frequency of specific style-related parts of speech in a given text, with the formula: CDI = (30 + article + preposition − personal pronoun − impersonal pronoun − aux. verb − conjunction − adverb − negation). To calculate the above, we obtained the occurrences of these parts of speech using the LIWC lexicon84. Formality. Formality is a key sociolinguistic construct which is known to vary across cultures, contexts, and audiences41,42,89. Linguistic formality encompasses the level of sophistication, politeness, and adherence to established linguistic conventions in written communication90. Formal language is marked by grammatical precision, structured syntax, and appropriate vocabulary, making it prevalent in professional settings, academic writing, official documentation, and respectful discourse. In contrast, informal language adopts a more relaxed and conversational tone, often incorporating slang, colloquialisms, and contractions, making it more common in casual or social interactions81,89. To measure formality, we leveraged a RoBERTa-based formality classifier from prior work91. This classifier is built on Grammarly’s Yahoo Answers Formality Corpus (GYAFC)92 and the Online Formality Corpus93, and it achieves an approximate accuracy of 91% on its benchmark dataset. For any given text, this classifier outputs a score between 0 to 1—a higher score indicating a greater degree of formal language. We employed this classifier on our AI and OC response datasets and compared the formality scores. Formality. Formality is a key sociolinguistic construct which is known to vary across cultures, contexts, and audiences41,42,89. Linguistic formality encompasses the level of sophistication, politeness, and adherence to established linguistic conventions in written communication90. Formal language is marked by grammatical precision, structured syntax, and appropriate vocabulary, making it prevalent in professional settings, academic writing, official documentation, and respectful discourse. In contrast, informal language adopts a more relaxed and conversational tone, often incorporating slang, colloquialisms, and contractions, making it more common in casual or social interactions81,89. To measure formality, we leveraged a RoBERTa-based formality classifier from prior work91. This classifier is built on Grammarly’s Yahoo Answers Formality Corpus (GYAFC)92 and the Online Formality Corpus93, and it achieves an approximate accuracy of 91% on its benchmark dataset. For any given text, this classifier outputs a score between 0 to 1—a higher score indicating a greater degree of formal language. We employed this classifier on our AI and OC response datasets and compared the formality scores. Empathy. Given our specific setting, empathy is a key mechanism in providing support8,43,94. Empathy refers to a cognitively complex process in which one can stand in the shoes of another person to understand their perspective, emotions, and the situations they are in43. Prior work evaluated the effectiveness of artificially created empathetic conversations by a chatbot95 and noted the success and positive reception of empathetic responses in online interaction settings8. We leveraged a RoBERTa-based empathy detection model, finetuned on a dataset of empathetic reactions to news stories96,97. Given a text, this model predicts the empathetic nature of a text—higher scores indicate greater expression of empathy. Empathy. Given our specific setting, empathy is a key mechanism in providing support8,43,94. Empathy refers to a cognitively complex process in which one can stand in the shoes of another person to understand their perspective, emotions, and the situations they are in43. Prior work evaluated the effectiveness of artificially created empathetic conversations by a chatbot95 and noted the success and positive reception of empathetic responses in online interaction settings8. We leveraged a RoBERTa-based empathy detection model, finetuned on a dataset of empathetic reactions to news stories96,97. Given a text, this model predicts the empathetic nature of a text—higher scores indicate greater expression of empathy. Politeness. Politeness is essential in therapeutic conversations, as it helps foster a supportive, respectful, and empathetic environment that fosters trust and openness46. To assess politeness levels, we leveraged a pre-trained politeness classification model98, which assigned politeness scores ranging from 0 to 1 to both AI and OC responses.To assess the applicability of the classifiers of formality, empathy, and politeness in our dataset, we conducted a sanity check on a randomly sampled set of 185 AI responses. We found that labels demonstrated high agreement with human judgments, with accuracy rates of 98.9% for formality (183/185), 98.4% for empathy (182/185), and 98.4% for politeness (182/185). This provides additional confidence in the reliability of our automated measures within the mental-health context. Politeness. Politeness is essential in therapeutic conversations, as it helps foster a supportive, respectful, and empathetic environment that fosters trust and openness46. To assess politeness levels, we leveraged a pre-trained politeness classification model98, which assigned politeness scores ranging from 0 to 1 to both AI and OC responses. To assess the applicability of the classifiers of formality, empathy, and politeness in our dataset, we conducted a sanity check on a randomly sampled set of 185 AI responses. We found that labels demonstrated high agreement with human judgments, with accuracy rates of 98.9% for formality (183/185), 98.4% for empathy (182/185), and 98.4% for politeness (182/185). This provides additional confidence in the reliability of our automated measures within the mental-health context. Semantic Similarity. Semantic similarity measures how well a response aligns with the underlying meaning and intent of the original query, reflecting the degree to which the response addresses the core content of the query in a coherent and contextually relevant manner. We obtained the semantic similarity between a query and a response by measuring the pairwise cosine similarity of word embedding representations of the queries and responses—where word embeddings are vector representations of words in latent lexico-semantic dimensions99,100. We used the 300-dimensional pre-trained word embeddings, trained on word-word co-occurrences in the Google News dataset containing about 100 billion words99. Semantic Similarity. Semantic similarity measures how well a response aligns with the underlying meaning and intent of the original query, reflecting the degree to which the response addresses the core content of the query in a coherent and contextually relevant manner. We obtained the semantic similarity between a query and a response by measuring the pairwise cosine similarity of word embedding representations of the queries and responses—where word embeddings are vector representations of words in latent lexico-semantic dimensions99,100. We used the 300-dimensional pre-trained word embeddings, trained on word-word co-occurrences in the Google News dataset containing about 100 billion words99. Linguistic Style Accommodation. While the above semantic similarity measure considers the content-based similarity, linguistic style accommodation focuses on non-content similarity—or how well a response accommodates the linguistic style of its query. More linguistically accommodating responses are known to be associated with more effective online support7,33. We obtained the linguistic style accommodation between a query and a response by obtaining the cosine similarity of the vectors based on the normalized occurrences of linguistic style dimensions of articles, prepositions, pronouns, auxiliary verbs, conjunctions, adverbs, negations etc, as obtained by using the LIWC lexicon28,33. Linguistic Style Accommodation. While the above semantic similarity measure considers the content-based similarity, linguistic style accommodation focuses on non-content similarity—or how well a response accommodates the linguistic style of its query. More linguistically accommodating responses are known to be associated with more effective online support7,33. We obtained the linguistic style accommodation between a query and a response by obtaining the cosine similarity of the vectors based on the normalized occurrences of linguistic style dimensions of articles, prepositions, pronouns, auxiliary verbs, conjunctions, adverbs, negations etc, as obtained by using the LIWC lexicon28,33. Diversity/Creativity. Diverse and creative responses are known to be effective in psychotherapy and social support32,33,48. We measured diversity by leveraging word-embedding representations in the 300-dimensional space99. Within each of the AI and OC datasets, we first obtained the centroid vectors by averaging the word embeddings of the responses. Then, we iterated through each of the responses within the two datasets and measured the cosine distance from the corresponding centroids. This distance measure signals how diverse (or creative) a particular response is to an average of all the responses—the greater the distance, the higher the diversity. Diversity/Creativity. Diverse and creative responses are known to be effective in psychotherapy and social support32,33,48. We measured diversity by leveraging word-embedding representations in the 300-dimensional space99. Within each of the AI and OC datasets, we first obtained the centroid vectors by averaging the word embeddings of the responses. Then, we iterated through each of the responses within the two datasets and measured the cosine distance from the corresponding centroids. This distance measure signals how diverse (or creative) a particular response is to an average of all the responses—the greater the distance, the higher the diversity. Social Support. Online social support, including emotional (empathy, encouragement) and informational (guidance, advice) support, plays a vital role in reducing psychological distress and improving mental health7,18,49,52. To identify support expressions in the responses, we used an expert-appraised dataset and classifier built in prior work7,33. These are binary SVM classifiers that assess the degree (high/low) of emotional (ES) and informational support (IS) in social media data. These classifiers were expert-appraised in prior work7, demonstrating strong performance—achieving k-fold validation accuracies of 0.71 for ES and 0.77 for IS33. We leveraged these classifiers to label the presence of ES and IS in the responses. Social Support. Online social support, including emotional (empathy, encouragement) and informational (guidance, advice) support, plays a vital role in reducing psychological distress and improving mental health7,18,49,52. To identify support expressions in the responses, we used an expert-appraised dataset and classifier built in prior work7,33. These are binary SVM classifiers that assess the degree (high/low) of emotional (ES) and informational support (IS) in social media data. These classifiers were expert-appraised in prior work7, demonstrating strong performance—achieving k-fold validation accuracies of 0.71 for ES and 0.77 for IS33. We leveraged these classifiers to label the presence of ES and IS in the responses. Qualitative analysis of smaller sample of data and expert evaluation We conducted a qualitative analysis on a randomly selected sample of 50 posts and their corresponding OC and AI (GPT-4-Turbo) responses from our dataset. Using an inductive approach, two co-authors first independently performed open coding on an initial subset of 10 posts to surface emergent concepts and develop preliminary interpretive themes. The co-authors then met to conduct collaborative affinity diagramming and thematic synthesis, discussing points of convergence and divergence, reconciling interpretations, and iteratively refining candidate themes through consensus-based discussion. Following this, the second author completed coding of the remaining posts, continuing to apply and refine themes as additional patterns emerged. We then conducted a thematic analysis to integrate codes into higher-level categories, yielding a set of coherent themes characterizing systematic similarities and differences between AI and OC responses. We ensured that each of the 50 posts included at least one corresponding OC response to support direct comparative analysis between human and AI responses. In addition to this qualitative comparison, we conducted an expert clinical evaluation on a randomly sampled set of 100 AI-generated responses. Our psychologist coauthor independently rated each response across four dimensions: factual accuracy, potential harmfulness, emotional attunement, and contextual responsiveness to users’ expressed mental health concerns. All dimensions were rated on a 1–5 scale, with higher values indicating a greater presence of the rated quality. Acknowledgements MDC was partly supported through NIH grants R01MH117172 and P50MH115838, and a grant from the American Foundation for Suicide Prevention. We are also grateful to Microsoft for supporting this research through the ``Foundation Models Academic Research'' program. We thank Sachin Pendse for sharing the Reddit online mental health community dataset and Jiawei Zhou, Shruti Belgali, Navya Khanna, and Sachin Pendse for participating in initial conversations of the work. Author contributions K.S. formulated the problem; K.S. and M.D.C. designed the research; K.S. and M.D.C. conceptualized and developed the analytic techniques; K.S. and Y.J. gathered and analyzed the data; K.S., Y.J., and V.J.R. interpreted the results; V.J.R. contributed clinical expertise and evaluation; All authors drafted and edited the paper. Data availability The datasets generated and/or analyzed during the current study are not publicly available due to the need to protect the privacy of users disclosing mental health concerns in online communities and to mitigate risks of ethical misuse of sensitive content. However, the data are available from the corresponding author on reasonable request, subject to appropriate ethical safeguards and data use agreements where applicable. Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement The datasets generated and/or analyzed during the current study are not publicly available due to the need to protect the privacy of users disclosing mental health concerns in online communities and to mitigate risks of ethical misuse of sensitive content. However, the data are available from the corresponding author on reasonable request, subject to appropriate ethical safeguards and data use agreements where applicable.