Menta: A Small Language Model for On-Device Mental Health Prediction

Tianyi Zhang, Xiangyuan Xue , Lingyan Ruan, Shiya Fu, Feng Xia, Simon D’Alfonso, Vassilis Kostakos, Ting Dang, Hong Jia, The University of Melbourne The University of Auckland RMIT University · 2025

Read Menta: A Small Language Model for On-Device Mental Health Prediction by Tianyi Zhang, Xiangyuan Xue , Lingyan Ruan, Shiya Fu, Feng Xia, Simon D’Alfonso, Vassilis Kostakos, Ting Dang, Hong Jia, The University of Melbourne The University of Auckland RMIT University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-02716.txt.

Introduction

Tianyi Zhang and nine co-authors build a small language model for spotting mental health risk on a phone. Mental illness affects hundreds of millions of people, yet early detection stays limited. Large models show promise here, but their size blocks use on everyday devices.

Small models offer a lighter path, though their use for social media based prediction is barely explored. Menta is designed to run on device, keeping data local and costs low. Private, phone based screening could help flag risk earlier, while sending fewer personal posts to distant servers. On phone checks keep the posts on the phone, a plus for teens and folks in small towns alike.

Catalog record

CREATORTianyi Zhang, Xiangyuan Xue , Lingyan Ruan, Shiya Fu, Feng Xia, Simon D’Alfonso, Vassilis Kostakos, Ting Dang, Hong Jia, The University of Melbourne The University of Auckland RMIT University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-02716

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Menta: A Small Language Model for On-Device Mental Health Prediction Tianyi Zhang, Xiangyuan Xue , Lingyan Ruan, Shiya Fu, Feng Xia, Simon D’Alfonso, Vassilis Kostakos, Ting Dang, Hong Jia, The University of Melbourne The University of Auckland RMIT University Abstract Mental health conditions affect hundreds of millions globally, yet early detection remains limited. While large language models (LLMs) have shown promise in mental health applications, their size and computational demands hinder practical deployment. Small language models (SLMs) offer a lightweight alternative, but their use for social media–based mental health prediction remains largely underexplored. In this study, we introduce Menta, the first optimized SLM fine-tuned specifically for multi-task mental health prediction from social media data. Menta is jointly trained across six classification tasks using a LoRA-based framework, a cross-dataset strategy, and a balanced accuracy–oriented loss. Evaluated against nine state-of-the-art SLM baselines, Menta achieves an average improvement of 15.2% across tasks covering depression, stress, and suicidality compared with the best-performing non–fine-tuned SLMs. It also achieves higher accuracy on depression and stress classification tasks compared to 13B-parameter LLMs, while being approximately 3.25 × \times smaller. Moreover, we demonstrate real-time, on-device deployment of Menta on an iPhone 15 Pro Max, requiring only approximately 3GB RAM. Supported by a comprehensive benchmark against existing SLMs and LLMs, Menta highlights the potential for scalable, privacy-preserving mental health monitoring. Code is available at: https://hong-labs.github.io/menta-project/ 1 Introduction Mental health disorders such as depression, anxiety, and suicidality affect hundreds of millions of people worldwide and constitute one of the leading contributors to the global burden of disease ( Organization et al. 2017 ) . Between 30% and 50% of people globally experience stress ( Piao et al. 2024 ) ; an estimated 5.7% of adults suffer from depression ( World Health Organization 2023 ) ; and more than 720,000 people die by suicide each year ( World Health Organization 2025a ) . As mental health issues have continued to rise globally over the past few decades ( Goodwin et al. 2022 ; Piao et al. 2024 ; Weaver et al. 2025 ) , there is an urgent need to enhance our understanding, diagnosis, and monitoring of these conditions through advanced technologies. Despite growing demand for mental health support, significant diagnosis gaps persist globally. Barriers such as limited clinic hours, geographic inaccessibility, and workforce shortages hinder timely assessment and early self-awareness ( World Health Organization 2025b ) . Structural, patient-level, and systemic obstacles further delay diagnosis for those with serious mental illness ( Wiesepape et al. 2025 ) . This mismatch between high need and limited resources reflects the constraints of traditional diagnostic models ( Stein et al. 2022 ) . Moreover, these traditional delivery of in-person sessions and self-reported questionnaires struggles to scale, and faces systemic bottlenecks, including workforce shortages ( Organization 2025 ; Brahmbhatt & Schpero 2024 ) , high costs ( Patel et al. 2018 ) , geographic disparities ( Yang & Zhang 2025 ) , and stigma ( Clement et al. 2015 ) . Mental health wait times can range from 3 to 18 months, varying by region and provider availability ( McMahan et al. 2022 ; Yang & Zhang 2025 ; Patel et al. 2018 ) . With the growing need for early detection and real-time monitoring, developing more seamless methods to identify early signs is critical for delivering timely support ( Patel et al. 2018 ) . This underscores the importance of portable, resource-efficient solutions such as small language models for scalable mental health detection and intervention. Figure 1: On-device mental health monitoring workflow using SLMs, from user browsing of social media, to data collection, post monitoring, in-situ prediction and meta log saving. The rise of portable, on‑device technologies offers a promising path for the early detection of mental health issues, particularly through passive, real‑time analysis of social media posts and smartphone applications. Smartphone-based linguistic data from social media enables scalable, passive monitoring of psychological states, facilitating early identification of mood disturbances, stress, or suicide risk. For example, such systems can support suicide prevention efforts ( Areán et al. 2016 ) and capture patterns in social media use that are relevant to mental well-being ( Hamilton et al. 2025 ) . Running locally on personal devices, these models preserve user privacy while analyzing language inputs for indicators of depression or stress ( Shin et al. 2023 ) . As a result, on-device deployment enables scalable, privacy-preserving mental health monitoring, extending detection beyond clinical settings and making access easier for a broader population. In this context, language models offer a powerful means of addressing the limitations of traditional approaches to mental health monitoring ( Xu et al. 2024 ) . Language models outperform supervised learning classifiers in mental health prediction tasks for capturing deeper context, information acquisition, and generalization capabilities ( Jin et al. 2025 ) , making them more practical for web-scale and real-world mental health monitoring. However, the application of large language models (LLMs) faces significant challenges: they are computationally expensive to train and deploy, and they raise privacy concerns in terms of processing sensitive data on remote servers ( Jin et al. 2025 ) . Their large size also makes them impractical for resource-constrained on-device settings, underscoring the need for more efficient alternatives. By contrast, Small Language Models (SLMs), offer comparable or even superior potential for digital health applications due to their lightweight architectures, lower inference costs, and feasibility of local deployment ( Abdin et al. 2024 ; Jia et al. 2025 ) . Deploying on digital devices locally, a privacy-preserving, on-device workflow is expected for mental health prediction from social media activity using lightweight SLMs which emphasizes in-situ AI screening across multiple mental health dimensions without storing personal data, enabling real-time, low-resource, and ethically-aligned monitoring (Figure 1 ). Yet, the potential of SLMs for mental health prediction remains largely underexplored. Despite work on the development of SLMs, there is no agreement yet on a standardized definition of SLMs ( Wang et al. 2024 ) . Given the lightweight requirement for such real-time mental health status detection tasks on social media posts, we define SLMs to be models ranging from a minimum of 1B to strictly under 7B parameters are commonly considered ‘small’ in contrast to full-scale LLMs (10B+) while those more than 7B to be large-scale small language models (LSSLMs) ( Abdin et al. 2024 ; Team 2024 ) . In this work, we explore the potential of SLMs in predicting digital mental health outcomes, and prove that SLMs, when carefully designed and optimized, serve as practical alternatives for multi-task mental health assessment on-device. To investigate this, we examine SLM performance across six mental health classification tasks spanning stress, depression, and suicidality. We introduce Menta, a compact model fine-tuned using a LoRA-based multi-task framework with cross-dataset training and a balanced accuracy–oriented loss. Menta performs strongly across diverse mental health tasks, particularly in stress and depression detection. We further demonstrate its practical value by deploying it on an iPhone 15 Pro Max, achieving real-time inference with minimal memory usage, enabling private and scalable on-device mental health support. Our key contributions are as follows: • We present Menta, a fine-tuned SLM tailored for multi-task mental health prediction from social media text, optimized for both

Read the complete text