A scoping review of the use of artificial intelligence as a psychological assessment tool Authors: Vinayak Dev, Nathan S Consedine, Yuan Gao, Rajitha Narayanasamy, Anna Serlachius Published: 2026 ABSTRACT AbstractMental health disorders are highly prevalent worldwide, yet access to timely and effective mental health assessment (and care) remains limited. Artificial intelligence (AI) offers potential solutions, but the literature on its use in psychological assessment contexts has not been comprehensively mapped. This review aimed to systematise existing research, identify gaps, evaluate methodological limitations, and outline future directions. Using a librarian-approved search strategy, 7595 records were retrieved from major databases and screened independently by two coders. Following the eligibility assessment, 320 peer-reviewed articles were included. Studies showed wide variability in sample sizes (1–19,400,000) with no clear temporal trend. Most recruited clinical (21%) or general population (16%) samples from China (24%) or the United States (21%), and focused on depression (54%), anxiety (14%), suicidality (12%) or stress (8%). Supervised (75%) and deep learning (47%) approaches predominated, often with multiple algorithms compared (77% of the studies). Validation commonly relied on cross-validation and convergence with screening instruments, with relatively little use of DSM or ICD diagnostic criteria (71% used neither). Area-Under-the-Receiver-Operating-Characteristics-Curve (AUC) was the most frequently used performance metric, and unsupervised models achieved the highest average AUC. A marginal improvement in performance was evident from 2014 to 2025. Overall, AI shows promise as a psychological assessment tool, but progress is constrained by limited transparency, heavy reliance on self-report data, inconsistent use of validated diagnostic standards, a narrow focus on outcomes, and insufficient demographic and cultural analyses. Future research should prioritise interpretability, ethical and cultural responsiveness, multi-modal data, diverse samples, and clinically meaningful validation.Subject terms: Psychology, Psychiatric disorders Introduction Mental health disorders, particularly depressive and anxiety disorders, affect almost a billion people [1], and account for 32% of years lived with disability and 13% of disability-adjusted life years (DALYs) globally [1, 2]. Despite their substantial contribution to the global burden of disease, the quality of mental healthcare remains suboptimal, with persistent gaps in access and service delivery worldwide [2, 3]. In high-income countries, disparities in psychological assessment persist due to inconsistencies in the availability of trained staff, inconsistent implementation of guidelines, inadequate infrastructure, gender norms, and cultural stigma [4]. These issues are amplified in low- and middle-income countries (LMICs), where mental health is often deprioritised or stigmatised [4]. Rural areas in LMICs are particularly affected by systemic and structural barriers to psychological assessment [4, 5], and often lack even basic access to mental health services [5]. Mental healthcare coverage for major depressive disorder illustrates this disparity, with the coverage being around 33% in high-income countries compared to only 8% in LMICs [1]. Disparities in care quality and outcomes are especially pronounced for racial and ethnic minorities and those from socioeconomically disadvantaged backgrounds [2]. Given these persistent access barriers, many individuals with mental health concerns increasingly turn to emergency services for support, particularly in remote areas [6]. However, even in emergency departments where patients presenting with mental health issues often receive higher triage priority, wait times for psychological assessment are typically longer compared to those with physical health complaints [7]. Patients report experiencing worsening mental health during these delays [8]. Traditional mental healthcare systems are often ill-equipped to meet the increasing demand for services [9], tend to have fragmented or complex service delivery pathways [10, 11], are not uniformly accessible [11, 12], and tend to be costly [13], leading to prolonged waiting times and inadequate support for individuals in distress [14]. The stigma associated with seeking mental health support further hinders many from seeking the help they need [14]. In this context, the emergence of AI-based technologies can offer promising solutions to these challenges [14]. Whilst there is no general legal definition for what constitutes ‘Artificial Intelligence’ (AI) outside of a specific application [15], AI is a term that broadly refers to technology designed to perform activities in a manner that emulates human intelligence [16, 17]. The use of AI-driven technologies can offer several advantages over traditional forms of healthcare provision, including prompt and efficient data-driven insights [18], prediction and early detection of issues [19, 20], ongoing monitoring [19], reduced stigma [14, 18], scalability [18], resource-efficiency [18], reduced practitioner burnout [18], and reduced access barriers [14]. AI technologies may also offer customisation of mannerisms, dialect, and other characteristics to enhance rapport between the virtual clinician and the patient [21], and consequently further reduce mental health support-related stigma. Although such agents exist, there is a notable dearth of attempts to analyse, systematise, and map the trends and gaps in the scientific literature on the use of AI in mental health contexts, particularly as a psychological assessment tool. However, such work is clearly needed. Despite high usage and demand [22, 23] leading to the proliferation of AI agents, the lack of formal analysis or regulation is problematic and creates a range of ethical and legal issues [15]. Issues include privacy concerns [24, 25], concerns regarding risk and safety management [23–25], need for ongoing human oversight (which may not always be feasible) [26], absence of ethical consideration during design or implementation process [23, 24], lack of a defined duty of care [27], transparency [25], balance between technological advancement and human-centric care [14, 25], and frameworks for their regulation, accountability and transparency [14, 23–25, 27]. Many of these issues likely reflect the rapid development in AI technologies (which does not allow the corresponding evidence base to adequately ‘keep up’), a lack of awareness of such technologies amongst groups of mental health professionals [24], and the creation of AI-based tools within siloed professional communities, which tend to involve minimal interactions between computer scientists and healthcare professionals [28]. Given this state of affairs, the primary objective of this review is to (a) map and systematise existing research on the use of AI for psychological assessment; (b) highlight trends and potential gaps in the literature; (c) discuss potential methodological limitations and ways to mitigate these limitations; and (d) identify key research directions. A preliminary search of MEDLINE, the Cochrane Database of Systematic Reviews, and JBI Evidence Synthesis was conducted, and no current or ongoing systematic reviews or scoping reviews on the topic were identified as of 17 November 2025. There were some PROSPERO-registered reviews [29–33], and published reviews [34–40], that focused on the use of AI in psychological contexts. However, these reviews were either (a) not specific to psychological assessment (with one exception, which was limited in sample size) [37]; (b) location-specific (e.g., Asia) [32]; (c) profession-specific (e.g., mental health nursing) [30]; (d) planned (and dated) but not actualised [33]; and/or (e) limited by their sample size (with four [38], nine [39], and 15 studies [37, 41], respectively). Additionally, many published reviews on the use of AI in psychological contexts employed a narrative review methodology [34–37, 40], thus lacking a structured process for gathering and presenting data [42] and an evidence-based synthesis of findings [43]. Scoping reviews are recommended, particularly over systematic reviews, when the aim is to (a) identify the types of available evidence in a given field; (b) clarify key concepts/definitions in the literature; (c) examine how research is conducted on a certain topic or field; (d) identify key characteristics or factors related to a concept; (e) have a precursor to a systematic review; or (f) identify and analyse knowledge gaps [44, 45]. Given these considerations, the novelty and breadth of the subject, and the relatively fragmented nature of the corresponding literature, a scoping review methodology (whilst maintaining the rigour of a systematic review) was selected for this study. Review questions This scoping review seeks to answer the following questions: What are the trends and gaps in existing AI-based human psychological assessment research? What are the trends and gaps in existing AI-based human psychological assessment research? How effective are AI-based tools in human psychological assessment contexts? How effective are AI-based tools in human psychological assessment contexts? What are the methodological issues associated with evaluating the use of AI in human psychological assessment contexts? What are the methodological issues associated with evaluating the use of AI in human psychological assessment contexts? What are the likely future directions associated with the research and application of AI in human psychological assessment contexts? What are the likely future directions associated with the research and application of AI in human psychological assessment contexts? Methods Search strategy and selection criteria The review was conducted in accordance with JBI [46] and PRISMA [47] guidelines, and was pre-registered on OSF Registries (osf.io/7z6nh) [48]. The search strategy was devised in consultation with a medical librarian and conducted on 17 November 2025 using the Cochrane Library (including CENTRAL), AMED, MEDLINE, EMBASE, APA, PsycINFO, OvidJournals, IEEE Xplore, and Scopus (i.e., a combination of general, health and technology-related sources). A combination of subheadings (MeSH terms/EMTREEs) and keywords was used, using standardised terms that refer to (a) artificial intelligence (e.g., AI, machine learning, deep learning, etc.); (b) assessment (e.g., symptom assessment, patient outcome assessment, diagnosis, etc.); and (c) psychology and psychiatry. To capture as much of the literature as possible, the ‘explode’ and ‘many places’ features were also used on key terms, such as ‘artificial intelligence’, ‘psychology’ and ‘diagnosis’ (see Appendix). Subsequently, limits were set to include only studies published since 2014 and those conducted using human data. Inclusion criteria included studies that were written in English, peer-reviewed, published during or after 2014, empirical in nature (i.e., not reviews, theoretical articles, opinion pieces, book chapters, corrigenda or the like), exclusively focused on assessment (not treatment), primarily psychological in nature (i.e., had at least one psychological primary outcome and predictor), used human data, and involved the use of AI. Conversely, studies that were excluded from the review were not in English, were biological, neurological or neuropsychological in nature (including a focus on neurodevelopmental disorders, such as ADHD or ASD), were focused on treatment (including treatment prognosis and assessment), used methodologies that are considered para-AI but not AI (e.g., Structural Equation Modelling) [49], focused on suicide attempts or self-harm behaviours without associated psychological components as outcomes, or focused on alcohol or other drug addictions as outcomes—due to the addictions sector being a niche area of clinical practice, having a relatively distinct literature base and unique aims and challenges [50–54]. Screening The screening was conducted via Rayyan, an online screening platform [55]. Of the identified records (n = 14,283), 11,590 were identified as duplicates by Rayyan, and 6,688 were removed manually. The two coders (VD and YG) independently screened the articles using Rayyan based on their titles and abstracts. Of the 7,595 screened studies, the majority were excluded for having the wrong outcome (n = 4,790), the wrong publication type (n = 1,233), or the wrong predictors (n = 853). There were conflicts regarding 224 studies (i.e., a 2.95% disagreement rate). All but two of these conflicts were resolved through a discussion between the two coders; 51 conflicts were resolved in favour of one coder and the rest in favour of the other. The two remaining articles were independently reviewed by the supervisory team (AS and NC), and both were included in the review (one in favour of each coder). The conflicts were primarily in relation to (a) ambiguity regarding what constituted a ‘psychological’ variable (e.g., self-narratives, or age of onset, of a psychological disorder); (b) uncertainty regarding what constituted a neurological or neuropsychological variable as opposed to a psychological variable (e.g., cognition); (c) ambiguity regarding behaviours without underlying psychological components being focused on as outcomes (e.g., suicidal attempts being considered an outcome without a focus on depression or suicidal ideation); (d) assessment being a minor component of a primarily intervention-focused study; (e) machine learning-based creation of psychometric scales for assessment, instead of direct assessment of psychological disorders; (f) machine learning-based prediction of ‘risk factors’ as outcomes; (g) studies with both, neurological and psychological, assessment outcomes; (h) conference papers being mistaken as journal articles; and (i) the included study being in a non-English language despite the abstract being in English. Of the articles assessed as meeting initial inclusion criteria (n = 333), 13 were excluded due to not being peer-reviewed empirical journal articles (n = 4), having biological predictors (n = 3), lacking full-text availability (n = 2), having a non-AI-based methodology (n = 2), having the wrong outcome (i.e., chatbot usability; n = 1), and having a qualitative methodology (n = 1). Ultimately, 320 articles were included in the review (Fig. 1). Flowchart of Study Identification, Screening, Eligibility, and Inclusion Process. Note. Adapted from Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021;372:n71. https://doi.org/10.1136/bmj.n71. Licensed under CC BY 4.0. Data extraction Data were extracted from the included articles by two authors (RN and VD) and verified by another author (YG). Trends across these articles were analysed by coding and examining common variables using Microsoft Excel [56]. These variables included year of publication, sample size, origin and type (e.g., patients, general population, women, etc.), area of assessment (clinical or non-clinical), AI algorithm type (e.g., random forest, neural network, support vector machine, etc.), validity assessment process (e.g., algorithmic, symptom checklists, clinician interviews, etc.), formal use of the diagnostic criteria (DSM or ICD), outcome variables (e.g., depression, anxiety, schizophrenia, etc.), and performance metric type (e.g., AUC, RMSE, F-score, etc.) and values. Results Year of publication There was a notable increase in the number of studies focusing on AI use in psychological assessment contexts over time, particularly post-2023 (Fig. 2), from one peer-reviewed journal article on the topic in 2014 [57] to 129 in 2025 [58–61]. Three remaining studies were in press and scheduled for publication in 2026 [62–64]. Number of Peer-Reviewed Studies on the Use of AI in Psychological Assessment Contexts. Note. At the time of screening (17 November 2025), three studies were in press and scheduled for publication in 2026. Sample size, type, and origin A high-kurtosis distribution curve was observed across per-year average sample sizes over time (2014–2026), with 2020 as an outlier (M = 327,006), followed by 2024 (M = 209,845) and 2025 (M = 178,291). The average sample size across all reviewed studies was 133,438, with a range of 1 (minimum) [65] to 19,400,000 (maximum) [66]. Fourteen percent (n = 45) had samples of up to 100 [67–70], 28% (n = 88) had 101–500 [61, 71–73], 12% (n = 37) had 501–1,000 [57, 74–76], and 45% (n = 145) had a sample size of more than 1,000 [77–80]. Five studies did not specify their sample size [81–85]. Sample types are important to consider as they can highlight potential differences in the utility of AI assessment across populations and contexts and help establish generalisability. Amongst the sample types examined (Fig. 3), most studies recruited patients (n = 67; 21%) [70, 86–88], individuals from the general population (n = 51; 16%) [74, 89–91], or a combination of healthy individuals and patients (n = 45; 14%) [92–95]. Conversely, a minority of studies focused specifically on professionals (n = 5; 2%) [96–100], military personnel (n = 4; 1%) [76, 101–103], and men (n = 2; 1%) [104, 105]. Some of the ‘other’ sample types included athletes (n = 1) [106], community corrections prisoners (n = 1) [107], and individuals quarantined at home during the COVID–19 outbreak (n = 1) [108]. Distribution of Study Sample Types in the Included Literature. The reviewed studies demonstrated considerable diversity in their sample origins, with 40 countries being represented. Most studies had sample data originating in China (n = 78; 24%) [72, 80, 109, 110], United States (n = 68; 21%) [74, 75, 97, 111], South Korea (n = 15; 5%) [91, 112–114], India (n = 10; 3%) [96, 99, 115, 116], the United Kingdom (n = 9; 3%) [103, 117–119], Brazil (n = 9; 3%) [71, 120–122], and Germany (n = 8; 3%) [95, 123–125]. Some studies included international datasets (n = 44; 14%) [66, 126–128], but only one included data from the African continent [90]. Outcomes Most reviewed studies focused on clinical contexts (n = 279; 87%) [78, 117, 129, 130]. In this context, ‘clinical’ was defined as relating to outcomes that involved psychiatric or clinical psychological assessment and diagnosis (e.g., mood disorders, post-traumatic stress disorder (PTSD), schizophrenia spectrum disorders, and suicidal ideation). Some of the non-clinical outcomes included studies assessing personality factors (n = 10; 3%) [131–134], discrete emotions and emotional states (n = 3; 1%) [85, 135, 136], subjective wellbeing (n = 2; 1%) [137, 138], apathy (n = 1) [139], and fear of missing out (n = 1) [140]. Of the included studies, most focused on assessing depression (or a depressive disorder) as their primary outcome (n = 173; 54%) [78, 141–143], followed by anxiety (n = 44; 14%) [123, 144–146], suicidality (n = 38; 12%) [125, 147–149], stress (n = 26; 8%) [150–153], schizophrenia or psychosis (n = 25; 8%) [129, 154–156], PTSD or post-traumatic stress (n = 23; 7%) [57, 130, 157–159], bipolar disorders (n = 20; 6%) [73, 160–162], personality disorders (n = 7; 2%) [86, 163–165], and eating disorders (n = 4; 1%) [166–169]. An overview of the proportions of primary outcomes is provided in Fig. 4. Primary Outcomes Examined in the Included Literature. Note. ‘Other’ included outcomes such as Obsessive-Compulsive Disorder, loneliness, affective cognition, and apathy. AI algorithms A range of AI algorithms were in evidence across the reviewed studies, e.g., Decision Tree, Random Forest, ResNet, K-Nearest Neighbour, and Support Vector Machine. Most included studies (n = 246; 77%) [133, 170–172] analysed more than one type of AI algorithm, typically comparing their own or preferred model with others. As per the framework proposed by Mukhamediev et al. [173], the AI (machine learning) algorithms in the reviewed studies were categorised as supervised learning, unsupervised learning, semi-supervised learning, dimensionality reduction, reinforcement learning, and deep learning (neural networks) algorithms. Correspondingly, most studies were found to use a type of supervised learning algorithm (n = 241; 75%) [61, 77, 156, 157], followed by the use of deep learning (n = 150; 47%) [143, 174–176], unsupervised learning (n = 7; 2%) [79, 100, 177, 178], dimensionality reduction (n = 5; 2%) [62, 79, 171, 179], reinforcement learning (n = 1) [180], and semi-supervised learning (n = 1) [79] algorithms. Studies using deep learning models increased notably over the reviewed period, from none in 2014–2018 (except one in 2017) [70] to 66 in 2025 [143, 175, 181, 182]. Validity assessment Most reviewed studies utilised a combination of validation processes, with the most common being screening scales or symptom checklist-based convergent validity assessments (n = 182; 57%) [72, 79, 183, 184] or algorithmic cross-validation (n = 149; 47%) [69, 142, 151, 185]. Notably, majority of the clinical studies included in the review also relied on using screening scales or symptom checklists (n = 167; 60%) [72, 79, 183, 184] and/or algorithmic validation (n = 129; 46%) [69, 142, 165, 185] to assess the performance of their AI models, instead of using diagnoses provided by mental health professionals (n = 33; 12%) [65, 120, 166, 186]. Importantly, most clinical studies did not use either the Diagnostic and Statistical Manual of Mental Disorders (DSM) [187] or International Statistical Classification of Diseases and Related Health Problems (ICD) [188] to validate their AI algorithms (n = 197; 71%; Fig. 5) [161, 170, 171, 189], despite these being the most widely used criteria for mental health diagnosis [190]. Testament to the need for greater method rigour in this area, one study did not engage in any validation processes [100]. Use of DSM and ICD Diagnostic Criteria in Clinical Studies. Note. Most clinical studies used neither the DSM nor the ICD criteria for validation. The Area Under the Receiver Operating Characteristics Curve (AUC) [191], was selected as the performance measure for comparative analyses. AUC is a scalar summary of the ROC curve, which measures a classifier’s ability to discriminate between positive and negative classes; more specifically, it represents the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative instance [191]. AUC was selected as the primary comparison metric due to it being (a) the most commonly used primary performance metric across the included studies, and (b) the preferred validation measure in machine learning research, particularly over accuracy, error rate and error cost [191–193]. As mentioned earlier, AUC was the most commonly used primary validation measure (n = 192; 60%) [58–60, 68], followed by accuracy (n = 83; 26%) [84, 135, 162, 194], R2 (n = 11; 3%) [74, 178, 195, 196], precision (n = 6; 2%) [66, 197–199], F1-score (n = 5; 2%) [142, 200–203], RMSE (n = 4; 1%) [70, 116, 137, 161], and others (e.g., variance [104, 204, 205], Cohen’s kappa [168, 175, 206], and Matthews Correlation Coefficient (MCC) [207]). Two studies did not conduct any accuracy assessments [100, 208], and another did not provide specific values for its accuracy metric (MCC)—instead alluding to the values as being “better-than-chance accuracies but low” [207]. Accuracy trends A plot of the highest AUC values averaged across the last 11 years showed a marginally improving trend, with some reduction in variation over time (Fig. 6). The average (highest) AUC of AI-based assessment tools over the last 11 years was 86%, ranging from 50% (in 2025) [209] to 100% (in 2015 [68], 2024 [146, 210], and 2025 [58–60]). Mean Highest Reported AUC by Publication Year. Note. This is based on studies that included AUC values (n = 192; 60%). Because only one of the three studies scheduled for publication in 2026 included AUC values, the 2026 data were not included in the graph. Comparisons based on the highest yielded AUC suggested that studies using unsupervised learning models tended to be the most accurate in psychological assessment contexts (100% average highest AUC; n = 1) [68], followed by those using deep learning models (90%; n = 62) [143, 174, 176, 211], supervised learning models (85%; n = 155) [61, 77, 156, 157], and dimensionality reduction models (83%; n = 3) [62, 68, 171]. By virtue of having the greatest number of studies using supervised and deep learning models, both categories demonstrated the greatest variation (range) in their AUC-based accuracies, i.e., 50% [209]–100% [58, 60, 146, 210] and 66% [212]–100% [58, 59, 210], respectively. However, greater power is needed across AI types to enable more accurate and reliable AUC comparisons. Discussion This scoping review provides a comprehensive overview of empirical studies that examine the use of AI as a psychological assessment tool. To our knowledge, this is the first scoping review to specifically focus on mapping and systematising existing research on the use of AI in psychological assessment contexts. More broadly, this is also the first review to focus on the use of AI in psychological contexts that is comprehensive enough to incorporate over 300 peer-reviewed studies on the topic. The reviewed studies exhibited substantial variability in sample sizes, with no notable trend over time. Most studies recruited patients, individuals from the general population, or both. A significant majority of studies targeted clinical conditions, particularly mood and anxiety disorders, suicidality, and stress. Supervised learning and deep learning were the most widely used AI approaches, with most studies comparing multiple algorithms. Validation methods frequently included convergent validity via screening tools or algorithmic cross-validation, with limited direct reliance on DSM- or ICD-based diagnostic criteria. AUC was the predominant performance metric used, with studies using unsupervised and deep learning models yielding the highest average AUC. Based on the findings, AI can be considered a promising and increasingly studied psychological assessment tool. However, notable gaps and inconsistencies were also evident in the studies analysed in this review. For example, some algorithms performed well in one context but not another. For instance, the Ada chatbot had moderate agreement with prior diagnoses in adults (kappa = .64) but far poorer accuracy for paediatric cases (kappa = .40) [206]. Such patterns are evident in other studies, with notable variation as a function of age [144] and gender [213], likely indicating that a single machine learning approach may not perform comparably across demographically diverse samples. This consideration also underscores the importance of accounting for cultural and regional factors in the development, deployment, and data collection and interpretation of AI algorithms in psychological assessment contexts—particularly given that only one of the 320 reviewed studies included samples from the African continent. If AI models continue to be developed in cultural silos, with, for instance, a lack of input from healthcare users in African countries, they have the potential to exacerbate existing healthcare access barriers, biases and inequities. Despite unsupervised and deep learning models achieving the highest average AUC values and deep learning models increasingly outperforming linear models [58, 69, 214], most studies to date have relied on simpler, linear supervised learning models (whilst increasingly adopting deep learning). In this regard, it is important to consider that deep learning models rely on substantially larger datasets and greater computational resources and costs than alternative AI algorithms [215, 216]. These requirements may not be deliverable in mental healthcare contexts that already struggle to meet resource demands, particularly in LMICs. Beyond data and computational requirements, deep learning models depend heavily on “black box” mechanisms of information processing and consequently offer substantially reduced explainability, which likely diminishes patient and provider trust and acceptance [217]. So, whilst the literature and current review highlight deep learning models as delivering superior performance, a fundamental trade-off exists between an algorithm’s (a) performance, and (b) explainability, cost, computational efficiency, and clinician trust [215–217]. This trade-off underscores the critical nuances that warrant careful consideration when implementing AI technologies in mental healthcare contexts, particularly where practical deployment constraints and clinical acceptability may ultimately outweigh performance gains. Furthermore, whilst studies encompassed a range of clinical outcomes, personality disorders and conditions such as OCD have received limited attention. There were also limited attempts at directly comparing the performance of AI algorithms across specific types of disorders, e.g., generalised anxiety vs specific phobia, instead of depressive vs anxiety disorders more broadly. However, the most important limitation of the AI-based psychological assessment literature is its reliance on self-reported data (particularly self-reported diagnostic data) and on algorithmic cross-validation or convergent validity via symptom checklists rather than on construct validity assessed via formal DSM- or ICD-based clinician assessments. This pervasive pattern raises major concerns regarding the ostensible accuracy of the algorithms tested thus far. The use of validation and performance metrics other than AUC, or the absence of accuracy metrics [208], also raises concerns about the integrity of the validation processes used in some of these studies. Validation integrity concerns are critical in studies evaluating AI algorithms for psychological assessment, as the credibility of these studies and the associated technologies hinges on rigorous and conceptually sound validation practices. When algorithms are evaluated using proxy measures, such as symptom checklists or self-reported diagnoses (rather than direct assessments of the underlying constructs), the validity of the findings becomes questionable. This approach often conflates concurrent validity with true effectiveness, which would require demonstrating construct validity across diverse contexts and populations via adherence to diagnostic criteria (such as the DSM or ICD) and reliable psychological (or psychiatric) assessments by trained and experienced clinicians, along with predictive validity (i.e., using the algorithms to predict prospective outcomes). Proxy measures of the underlying construct, and a tendency to assess construct validity via concurrence with proxy measures, can be misleading and fail to predict decision-making accuracy or real-world performance (ecological validity), potentially making the algorithms appear valid only because they mimic flawed or incomplete benchmarks. Hence, future research on the use of AI in psychological assessment contexts might profitably focus on four key areas. One, it could focus on maximising transparency regarding the technical underpinnings of their AI creation and implementation, particularly data collection and interpretation processes. Second, it could focus on analysing and discussing the responsiveness and adaptability of AI algorithms, particularly across diverse demographics and mental health outcomes. Third, it could address some of the methodological limitations salient across the AI mental health assessment literature by using multimodal data (e.g., visual and aural) [67, 181], engaging in longitudinal and experimental validation, and ensuring consistent use of validated performance measures. Finally, future research should ensure adherence to validated diagnostic criteria (such as the DSM and the ICD), with reliance on clinician-assessed construct validity rather than self-reported diagnoses, algorithmic cross-validation, or symptom checklist-based convergent validity. Doing so would also facilitate a greater focus on establishing the clinical significance of the assessed AI algorithms, rather than merely statistical significance and discrimination. An emphasis on service integration pathways would further facilitate a focus on the clinical, rather than statistical or technical, significance of AI-based psychological assessment tools. Integration into existing mental healthcare systems is likely to be most feasible when AI tools are positioned as adjunctive assessment supports rather than stand-alone diagnostic substitutes. In well-resourced settings, this may involve triage support, symptom monitoring, prioritisation of referrals, or decision support embedded within stepped-care pathways. Conversely, in LMICs, where specialist access is often limited, implementation may need to prioritise low-cost, low-bandwidth, mobile-first, and linguistically adaptable tools that support community-based screening, task-shared care, or referral stratification. However, successful integration in such settings will depend on local validation, clinician and service-user involvement, infrastructure constraints, referral pathways for high-risk cases, and safeguards to prevent shifting the diagnostic burden onto under-resourced systems without adequate human oversight. Future research in this field should also move beyond simply acknowledging ethical concerns and instead operationalise established ethical AI frameworks throughout model development and deployment. In the context of psychological assessment, four priorities seem especially important: (a) using representative and culturally appropriate training and validation data; (b) routinely evaluating subgroup performance to detect demographic bias; (c) maintaining human oversight in high-stakes clinical decisions; and (d) establishing explicit lifecycle governance procedures for documentation, external validation, and post-deployment monitoring. This should include transparent reporting of intended use, data provenance, known limitations, and processes for auditing calibration drift or performance degradation across settings and populations. These issues are likely to be central to whether AI-based assessment tools are safe, equitable, and clinically useful. For more detailed implementation guidance, existing ethical and governance frameworks relevant to computational psychiatry and digital mental health, such as the Integrated Ethical Approach for Computational Psychiatry (IEACP) [218], the World Health Organization guidance on Ethics and Governance of Artificial Intelligence for Health [219, 220], and the NIST AI Risk Management Framework [221], may be used. The current review also had important limitations in its methodology that future reviews could mitigate by using more rigorous systematic and meta-analytical review methodologies and by focusing on statistically synthesising findings to engage in comparative analyses across different sample types, machine learning algorithms, input data types (e.g., genetic, neurological or self-report data), output data types, and performance metrics. One such limitation was the reliance on AUC as a performance metric to evaluate, summarise and compare the discriminatory ability of AI models in the studies included in this review. A model can yield acceptable AUC values whilst still being of limited clinical use, particularly when the outcome of interest is rare, and the cost of missed cases is high. AUC may be particularly misleading when applied to strongly imbalanced datasets [222]. In such contexts, where the number of negative cases substantially exceeds the number of positive cases (as may occur for low-base-rate outcomes such as suicidality or specific low-frequency diagnoses), seemingly favourable AUC [222], accuracy [223], and F1-scores [223], may overstate a model’s practical usefulness for identifying positive cases. Precision-recall approaches may thus provide a useful complement to AUC, as they focus on the proportion of true positives amongst positive predictions [222]. Alternatively, MCC can offer a more balanced summary of classification performance by accounting for true positives, true negatives, false positives, and false negatives (i.e., all cells of the confusion matrix) [223, 224]. For clinical applications, calibration assessment is also important to determine whether predicted probabilities correspond to observed risks and are therefore suitable for decision-making [225]. Moving forward, a multidimensional evaluation framework is therefore likely to be important for supporting the safe integration of AI into healthcare systems [226]. Relatedly, future trials and reviews could benefit from analysing and reporting complementary discrimination and calibration metrics rather than relying solely on AUC. Furthermore, excluding studies not published in English may have introduced bias in the review and likely increased the representation of countries and research groups publishing in English, particularly in high-income and Anglophone academic contexts. Although it is difficult to quantify, this risk is especially relevant in a field where local culture, language, symptom expression, and healthcare system structures may materially influence both the design and performance of AI-based assessment tools. Similarly, the exclusion of qualitative studies limited our ability to synthesise evidence on implementation feasibility, clinician and patient acceptability, trust, perceived harms, cultural fit, and workflow integration. These issues are clearly important and can be addressed by subsequent reviews on this topic. In summary, the findings suggest that AI holds promise as a psychological assessment tool and can help current healthcare systems better meet the increasing demand for mental healthcare, particularly in low-income countries and rural areas. However, greater attention must be paid to the specific types of AI being employed, as some may be less effective than others in varying population- or outcome-based assessment contexts. Methodological limitations need to be carefully considered when evaluating the effectiveness of such tools. Some of these limitations include lack of transparency regarding the creation, mechanistic processing and interpretability of AI models, reliance on self-report data, limited use of validated diagnostic criteria (DSM and ICD), use of questionable or no performance metrics, lack of contextual (particularly demographic and cultural) analyses of AI effectiveness, over-reliance on linear, supervised learning models, and lack of focus on clinical outcomes other than mood and anxiety disorders, suicidality and stress. Importantly, future research in this area must address these limitations whilst evaluating model performance across diverse samples, utilising multi-model data, emphasising clinical over statistical significance, and being ethically and culturally responsive and transparent. Supplementary information Data Extraction, Coding and Analysis Spreadsheet Acknowledgements We would like to thank the authors of the studies included in this review; without their efforts, this review would not have been possible. The review is a component of the Doctoral thesis of the first author at the University of Auckland. Whilst the University of Auckland provided ongoing support and guidance (e.g., through review workshops and consultations with a medical librarian), it did not influence the authors’ decisions or interpretations of the data. Appendix: Search strategy The following search strategy, extracted from Ovid, was used to obtain relevant records for this review. exp artificial intelligence/ or exp machine learning/ or exp deep learning/ or exp supervised machine learning/ or exp support vector machine/ or exp unsupervised machine learning/ or exp sentiment analysis/ exp artificial intelligence/ or exp machine learning/ or exp deep learning/ or exp supervised machine learning/ or exp support vector machine/ or exp unsupervised machine learning/ or exp sentiment analysis/ AI.ti,ab,kw. AI.ti,ab,kw. artificial intelligence.mp. artificial intelligence.mp. 1 or 2 or 3 1 or 2 or 3 Patient Outcome Assessment/ or Personality Assessment/ or Health Impact Assessment/ or Symptom Assessment/ or assessment.mp. Patient Outcome Assessment/ or Personality Assessment/ or Health Impact Assessment/ or Symptom Assessment/ or assessment.mp. exp Diagnosis, Computer-Assisted/ or exp Diagnosis-Related Groups/ or exp “Diagnosis, Dual (Psychiatry)”/ or exp Diagnosis/ or exp Diagnosis, Differential/ or diagnosis.mp. or exp Early Diagnosis/ exp Diagnosis, Computer-Assisted/ or exp Diagnosis-Related Groups/ or exp “Diagnosis, Dual (Psychiatry)”/ or exp Diagnosis/ or exp Diagnosis, Differential/ or diagnosis.mp. or exp Early Diagnosis/ 5 or 6 5 or 6 psychology.mp. or exp Psychology/ psychology.mp. or exp Psychology/ exp Psychiatry/ or psychiatry.mp. exp Psychiatry/ or psychiatry.mp. 8 or 9 8 or 9 4 and 7 and 10 4 and 7 and 10 limit 11 to humans limit 11 to humans limit 12 to yr = “2014 -Current” limit 12 to yr = “2014 -Current” Author contributions The first author (VD) was involved in the planning, data collection, screening, coding, analysis, interpretation, and manuscript writing aspects of the review. The third author (YG) co-screened the eligible studies and verified the transcription, coding, and analysis of the results. The fourth author (RN) assisted with data extraction. The second and fifth authors (NC and AS) provided supervisory oversight and input throughout the project, from its conception to the final revision of the corresponding manuscript. Funding No external research funding was received for this project. Competing interests There is no conflict of interest in this project. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Supplementary information The online version contains supplementary material available at https://doi.org/10.1038/s41398-026-04181-5. References