Use and Perceptions of Generative Artificial Intelligence Tools Among First-Year Medical Students: A Cross-Sectional Survey Authors: Stella Loizou, Vered Aharonson, Photos G Hajigeorgiou Published: 2026 ABSTRACT AbstractBackgroundArtificial intelligence (AI) tools, particularly large language models, are increasingly integrated into medical education. However, little is known about how early medical students perceive their educational value, reliability, and ethical implications.MethodsA cross-sectional survey was conducted among first-year students in the Undergraduate (MD) and Graduate-Entry (GEMD) Medical Programmes at the University of Nicosia, Cyprus. A total of 102 students participated across three cohorts: MD2030 (n = 44), MD2031 (n = 46), and GEMD2030 (n = 12). The questionnaire explored awareness and use of AI tools, perceived benefits and risks, trust in AI-generated medical information, ethical concerns, and attitudes toward AI in medical education. Quantitative data were analysed using descriptive and inferential statistics, while open-ended responses underwent sentiment analysis and topic modelling.ResultsAwareness of AI tools was nearly universal (93–100%), with ChatGPT use reported by 87–100% of students. Regular use ranged from 70.5% in MD2030 to 100% in GEMD2030. Students perceived AI as a valuable supplementary learning tool (mean Likert scores 4.07–4.22/5), particularly for improving understanding and efficiency. Trust in AI-generated medical information remained moderate (3.11–3.33/5), reflecting concerns about inaccuracies and ethical issues. Open-ended responses highlighted incorrect AI-generated information as a common concern. A significant difference in reported encounters with incorrect AI-generated content was observed between cohorts (χ2 = 10.73, p = 0.0047), with lower reporting among students exposed to AI-literacy training, suggesting an association between prior exposure to AI literacy activities and differences in how students report AI-generated errors.ConclusionAI tools are widely adopted by medical students and are perceived as enhancing rather than replacing traditional study methods. Structured AI literacy activities may be associated with differences in how students evaluate AI-generated content; however, further longitudinal and controlled studies are required to determine whether such educational experiences have a measurable impact on learners’ attitudes and practices.Keywords: artificial intelligence, medical education, student perceptions, AI-assisted learning, AI literacy Introduction Artificial intelligence (AI) technologies, particularly large language models and generative AI systems, have rapidly emerged as influential tools in higher education, with growing adoption in medical training environments. In medical education, platforms such as ChatGPT and other AI-assisted applications are increasingly used by students to support a wide range of learning activities, including summarizing lecture material, explaining complex concepts, generating practice questions, and assisting with written assignments.1 The widespread accessibility of these tools has created new opportunities for enhancing learning efficiency and personalization. At the same time, it has raised critical questions about whether AI serves primarily as a supplement to, or a potential replacement for, traditional study methods.2,3 Recent advances in generative AI have intensified interest in its role within medical education. Large language models are capable of producing human-like text, answering questions, and synthesizing information from multiple sources, making them attractive tools for rapid knowledge acquisition but also raising concerns about accuracy and reliability in clinical education contexts. These systems are also prone to generating inaccurate or misleading information, often referred to as “hallucinations”, which presents particular challenges in medicine, where accuracy and reliability are essential.4,5 This duality highlights the importance of understanding not only how students use AI tools, but also how they evaluate and trust AI-generated content. Despite the growing presence of AI technologies, the integration of AI literacy into medical curricula remains limited.6 Many students engage with AI tools independently, often without structured institutional guidance on appropriate use, limitations, or ethical considerations.1 This raises concerns about potential over-reliance on AI systems, reduced critical engagement with learning material, and the broader impact on the development of clinical reasoning and professional judgment. Consequently, there is an increasing need to explore whether AI enhances learning by supporting understanding and efficiency, or whether it risks displacing essential cognitive and analytical processes.2 Emerging evidence suggests that medical students frequently use AI tools and generally perceive them as beneficial for learning.7,8 However, studies also report concerns regarding the accuracy of AI-generated information, issues of trust, and ethical considerations related to academic integrity and responsible use.4,5,8 These findings indicate that while AI holds promise as an educational tool, its integration into medical education requires careful consideration of both pedagogical value and potential risks.5 An important and underexplored question is whether structured educational exposure to AI technologies can influence not only usage patterns but also students’ critical evaluation of AI-generated outputs. Educational interventions that promote critical evaluation of AI-generated content, such as identifying inaccuracies or biases, may strengthen AI literacy and encourage more reflective and responsible use.6 In particular, such training may influence not only students’ attitudes toward AI, but also how they recognise, interpret, and report errors in AI-generated information. However, empirical evidence on the effectiveness of such approaches remains limited, particularly in early stages of medical training. While several studies have examined medical students’ use of AI tools, few have explored the relationship between AI literacy exposure and students’ evaluation of AI-generated information. Furthermore, little is known about how first-year medical students perceive AI-generated inaccuracies and how these perceptions compare between undergraduate-entry and graduate-entry learners. The present study investigates the role of AI in medical education by examining awareness, usage patterns, and perceptions of AI tools among first-year medical students at the University of Nicosia Medical School. The objectives of this study were therefore to: (1) assess students’ awareness and use of AI tools for learning; (2) evaluate perceptions of the benefits, risks, and reliability of AI-generated information; (3) explore ethical considerations related to AI-assisted learning; and (4) explore whether differences in attitudes and reported experiences are associated with prior exposure to AI literacy activities. By addressing these aims, the study seeks to contribute to the evolving understanding of whether AI functions as an enhancement to traditional learning strategies or poses a risk of their displacement, and to inform the responsible integration of AI into medical education. Materials and Methods Study Design, Participants and Setting This study employed a cross-sectional survey design to examine medical students’ awareness, use, and perceptions of AI tools in medical education. The study was conducted at the University of Nicosia Medical School in Cyprus. Participants were first-year students of three cohorts enrolled in two different medical programs. Data were collected using an online questionnaire administered through the SurveyMonkey platform and a total of 102 students participated in the study. We received 44 responses from first-year students of a 6-year undergraduate medical program (MD2030) during academic year 2024–25, 46 responses from first-year students of a 6-year undergraduate medical program (MD2031) during academic year 2025–26 and 12 responses from first-year students enrolled in a 5-year graduate medical program (GEMD2030) during academic year 2025–26 (Table 1). Students in the MD2031 cohort had previously participated in a structured educational activity in which they were required to critically evaluate AI-generated text containing intentionally embedded factual errors and misleading statements. Students identified and corrected these inaccuracies through independent literature searching prior to completion of the survey, which may have been associated with differences in their evaluation of AI-generated content. Table 1Study ParticipantsCohortEligible StudentsResponsesResponse RateMD20301914423%MD20312204621%GEMD2030471226% Study Participants Additionally, the undergraduate-entry cohorts consist primarily of students entering medical school directly after secondary education. In contrast, students in the graduate-entry program are required to hold at least a bachelor’s degree prior to admission, resulting in a cohort with greater prior academic experience. Participation in the evaluation was voluntary and anonymous. Informed consent was obtained from all participants. The study received ethical approval from the Cyprus National Bioethics Committee, an independent national body that is not under the administrative control of any ministry. Survey Instrument and Validation The questionnaire was developed specifically for this study based on themes identified in the literature regarding AI use, trust, ethical concerns, and educational applications. It was reviewed by three faculty members with experience in medical education and AI-assisted learning to assess content validity and clarity. Although formal psychometric validation was not performed, the questionnaire underwent expert review for clarity and relevance. The survey consisted of five sections: Demographic information, including age, gender, academic background, preferred learning style, English language proficiency, and prior experience with AI tools. Demographic information, including age, gender, academic background, preferred learning style, English language proficiency, and prior experience with AI tools. Awareness and usage of AI tools, including familiarity with AI applications and frequency of use in academic study. Awareness and usage of AI tools, including familiarity with AI applications and frequency of use in academic study. Perceived benefits and limitations of AI-assisted learning, including reported advantages and challenges associated with AI use. Perceived benefits and limitations of AI-assisted learning, including reported advantages and challenges associated with AI use. Trust and ethical considerations related to AI-generated medical information, including students’ trust in AI outputs and concerns regarding reliability, bias, and potential over-reliance. Trust and ethical considerations related to AI-generated medical information, including students’ trust in AI outputs and concerns regarding reliability, bias, and potential over-reliance. Future perspectives, including students’ views on whether AI tools should be integrated into formal medical education. Future perspectives, including students’ views on whether AI tools should be integrated into formal medical education. Several questions allowed multiple responses, while two key questions—perceived usefulness of AI tools and trust in AI-generated medical information—were assessed using five-point Likert scales (1 = lowest rating, 5 = highest rating). Open-ended questions were also included to allow students to describe experiences with incorrect AI-generated information and to suggest improvements for AI tools in medical education. Data Collection and Statistical Analysis Students were invited to complete an anonymous online questionnaire hosted on SurveyMonkey (SurveyMonkey, San Mateo, California). The same instrument was administered in all three cohorts. Descriptive statistics were used to summarize demographic characteristics and survey responses. Categorical variables were reported as frequencies and percentages, while Likert-scale responses were summarized using means, standard deviations, and medians. Comparisons between the three cohorts were conducted to assess differences in awareness of AI tools, patterns of AI use, perceived usefulness of AI in studying, and trust in AI-generated medical information. Analysis of variance (ANOVA) was used to compare mean Likert-scale responses between cohorts, while chi-square tests of independence were used to examine differences in categorical variables. Although Likert-scale data are ordinal, mean scores are commonly analysed using ANOVA in medical education research when sample sizes are moderate and distributions are approximately symmetric. A p-value of less than 0.05 was considered statistically significant. NLP-Based Sentiment Analysis of Free-Text Responses Free-text responses to the two open-ended questions, Q17 and Q24, were exported from SurveyMonkey (SurveyMonkey Inc, San Mateo, California) into Microsoft Excel (Microsoft Corp, Redmond, Washington) files. For each response, the dataset included the student serial number, the response text, the completion date and time, and the corresponding cohort. Responses that were blank or that contained only “N/A” were removed. The analysis of the free-text responses followed standard natural language processing (NLP) procedures. The overall NLP workflow followed an analysis framework previously developed and applied by our research group to educational free-text responses and online discourse analysis, with adaptations appropriate to the present dataset.9–11 Sentiment analysis of the free-text responses was conducted in Python using the Natural Language Toolkit (NLTK), the Valence Aware Dictionary and sEntiment Reasoner (VADER).12,13 Minimal text preprocessing was applied prior to this analysis, in which comments were converted to string format and leading and trailing whitespace, as well as repeated internal whitespace, were removed. Punctuation and emphasis markers were retained, as these contribute to sentiment intensity in VADER. For each preprocessed response, VADER returned a compound sentiment score, between −1 (most negative) and 1 (most positive). The polarity classification used the standard VADER thresholds: positive for compound ≥ 0.05, negative for compound ≤ −0.05, and neutral for values between −0.05 and 0.05.11 Topic analysis was performed using standard natural language processing procedures.14 The minimally processed text used for sentiment analysis was further normalised by converting to lowercase, removing punctuation and non-alphabetic tokens, and eliminating common English stopwords (from the scikit-learn English stopwords corpus) while retaining negation terms. Tokens were lemmatised to reduce morphological variation, and a simple negation-tagging approach was applied (eg, “not accurate” → “accurate_NEG”) to preserve semantic context. The textual features extracted were unigrams (single words), bigrams (two-word phrases), and trigrams (three-word phrases), capturing both individual terms and short expressions. These features were weighted using Term Frequency–Inverse Document Frequency (TF-IDF), which highlights terms that are particularly informative within the dataset.15 Topic modelling was then applied using non-negative matrix factorisation (NMF), a matrix decomposition technique commonly used for unsupervised topic extraction in short-text corpora, which groups responses based on shared patterns of these weighted features and identifies underlying topics.16 Candidate models containing two to eight topics were evaluated separately for Q17 and Q24. Model selection considered reconstruction error, topic diversity, and the average number of topics assigned per response. The model with the highest composite internal score was selected, resulting in five topics for Q17 and six topics for Q24.3,14 The NMF model automatically identified latent topics within the responses. Each topic was characterised by a set of highest-weight terms and by the responses most strongly associated with that topic. To facilitate interpretation and reporting, two investigators reviewed the highest-weight terms together with representative responses from each topic and, where appropriate, combined conceptually similar topics into broader reporting themes. Thus, topics represent the direct computational output of the NLP model, whereas themes represent the higher-level concepts used to summarise the findings in the Results section. Statistical Tests Statistical analyses were performed on the two MD cohorts. Differences in sentiment between cohorts were assessed using the Mann–Whitney U-test. Differences in topic prevalence were evaluated using Fisher’s exact tests and chi-square tests of independence. Relationships between Q17 and Q24 responses were examined using Spearman correlation for sentiment scores and chi-square tests for dominant topic associations. The smaller GEMD cohort was included only in descriptive analyses. Results Participant Characteristics A total of 102 first-year medical students participated in the survey. The respondents comprised 44 students from the MD2030 cohort, 46 students from the MD2031 cohort, and 12 students from the GEMD2030 cohort. The age distribution differed between cohorts, reflecting the different entry routes to the program. Undergraduate-entry students (MD2030 and MD2031) were predominantly between 18 and 20 years of age, whereas the GEMD2030 cohort was older, with ages ranging from 21 to 25 years and a mean age of approximately 22.5 years. Gender distribution also differed between cohorts. Both undergraduate cohorts showed a female majority, while the graduate-entry cohort showed a slight male predominance (Table 2). Table 2Demographic Characteristics of ParticipantsVariableMD2030 (n=44)MD2031 (n=46)GEMD2030 (n=12)Age (years, mean ± SD)19.8 ± 3.119.0 ± 2.422.5 ± 1.3Age range17–3317–3021–25GenderFemale28 (63.6%)34 (73.9%)5 (41.7%)Male16 (36.4%)12 (26.1%)7 (58.3%)Academic backgroundHigh school entry36 (81.8%)39 (84.8%)0 (0%)Prior degree8 (18.2%)7 (15.2%)12 (100%)English proficiencyExcellent26 (59.1%)29 (63.0%)10 (83.3%)Good17 (38.6%)16 (34.8%)2 (16.7%)Fair1 (2.3%)1 (2.2%)0 (0%) Demographic Characteristics of Participants English language proficiency was high across all cohorts, with the majority of students reporting excellent or good academic English proficiency. As expected, all GEMD2030 students reported holding at least a bachelor’s degree prior to entry into medical school, whereas most undergraduate-entry students entered directly from secondary education (Table 2). Awareness and Use of AI Tools Awareness of AI tools designed for educational use was extremely high across all cohorts. 41 of 44 MD2030 students (93.2%), 43 of 46 MD2031 students (93.5%), and all 12 GEMD2030 students (100%) reported being aware of AI tools used in medical education. ChatGPT was the most commonly used AI tool across all cohorts. 42 MD2030 students (95.5%), 40 MD2031 students (87.0%), and all GEMD2030 students (100%) reported using ChatGPT for academic purposes. Other AI tools reported by students included Anki AI, Quillbot, and Canva AI, although these were used by smaller proportions of students (Table 3). Table 3Awareness and Use of AI ToolsVariableMD2030 (n=44)MD2031 (n=46)GEMD2030 (n=12)Aware of AI tools41 (93.2%)43 (93.5%)12 (100%)Used ChatGPT42 (95.5%)40 (87.0%)12 (100%)Used Anki AI12 (27.3%)11 (23.9%)6 (50.0%)Used Quillbot14 (31.8%)13 (28.3%)2 (16.7%)Used Canva AI5 (11.4%)6 (13.0%)3 (25.0%)Used MedPaLM2 (4.5%)2 (4.3%)0 (0%)Used AI before medical school28 (63.6%)30 (65.2%)12 (100%) Awareness and Use of AI Tools Frequency of AI Use Students reported frequent use of AI tools in their studies. Among MD2030 students, 31 of 44 students (70.5%) reported using AI tools at least weekly, including 12 students (27.3%) who reported daily use. Similarly, 34 of 46 MD2031 students (73.9%) reported using AI tools weekly or more frequently, with 11 students (23.9%) reporting daily use. In contrast, all GEMD2030 students (100%) reported using AI tools at least weekly, with 5 of 12 students (41.7%) reporting daily use. This suggests that graduate-entry students may integrate AI tools into their study routines more extensively than undergraduate-entry students (although statistical comparison was not performed due to small subgroup size) (Figure 1). Students reported using AI tools primarily for educational support activities. The most common uses included generating practice questions, answering medical questions, and summarizing lecture material. Other uses included assisting with coursework, supporting note-taking, and translating or simplifying complex material (Table 4). Frequency of AI Tool Use. Percentage of students in each cohort (MD2030, n = 44; MD2031, n = 46; GEMD2030, n = 12) reporting use of AI tools at least weekly. Frequent use was observed across all cohorts, with the highest proportion observed in the graduate-entry cohort (GEMD2030, 100%), compared with 70.5% in MD2030 and 73.9% in MD2031. Table 4Frequency of AI Use for StudyingFrequencyMD2030 (n=44)MD2031 (n=46)GEMD2030 (n=12)Daily12 (27.3%)11 (23.9%)5 (41.7%)Several times/week19 (43.2%)23 (50.0%)6 (50.0%)Once/week5 (11.4%)7 (15.2%)1 (8.3%)Monthly4 (9.1%)2 (4.3%)0 (0%)Never4 (9.1%)3 (6.5%)0 (0%)Weekly or more31 (70.5%)34 (73.9%)12 (100%) Frequency of AI Use for Studying Perceived Usefulness of AI Tools Students across all cohorts reported high perceived usefulness of AI tools for studying. On a five-point Likert scale, the mean usefulness scores were: MD2030: 4.07 ± 0.97 MD2030: 4.07 ± 0.97 MD2031: 4.22 ± 0.79 MD2031: 4.22 ± 0.79 GEMD2030: 4.17 ± 0.72 GEMD2030: 4.17 ± 0.72 The median score was 4 in all cohorts. Approximately 80–83% of students in each cohort rated AI usefulness as 4 or 5, indicating strong agreement that AI tools are beneficial for learning (Figure 2). Comparison of mean usefulness scores across the three cohorts using analysis of variance (ANOVA) showed no statistically significant differences (F = 0.34, p = 0.71), (Table 5). Students most frequently reported that AI tools enhanced their understanding of complex topics and saved time during studying. Other reported benefits included personalized explanations and improved retention of information. Perceptions of Usefulness and Trust in AI Tools (Mean Likert Scores). Mean Likert scale scores (1= Strongly Disagree; 5= Strongly Agree) for perceived usefulness and trust in AI tools across cohorts (MD2030, n = 44; MD2031, n = 46; GEMD2030, n = 12). Perceived usefulness was high across all cohorts (means 4.07–4.22), whereas perceived trust was lower (means 3.11–3.33), indicating a gap between perceived utility and confidence in AI outputs. No statistically significant differences were observed between cohorts for either measure (ANOVA, p > 0.05). Table 5Perceived Usefulness and Trust in AI Tools (Likert Scale)VariableMD2030MD2031GEMD2030p-valueUsefulness (mean ± SD)4.07 ± 0.974.22 ± 0.794.17 ± 0.720.71Median44495% CI intervals3.77–4.373.98–4.463.71–4.63Trust (mean ± SD)3.11 ± 0.753.20 ± 0.753.33 ± 0.490.63Median33395% CI intervals2.88–3.342.98–3.423.02–3.64 Perceived Usefulness and Trust in AI Tools (Likert Scale) Trust in AI-Generated Medical Information Despite the high perceived usefulness of AI tools, students reported moderate levels of trust in AI-generated medical information. The mean trust scores were: MD2030: 3.11 ± 0.75 MD2030: 3.11 ± 0.75 MD2031: 3.20 ± 0.75 MD2031: 3.20 ± 0.75 GEMD2030: 3.33 ± 0.49 GEMD2030: 3.33 ± 0.49 The median trust score was 3 in all cohorts. Statistical comparison using ANOVA demonstrated no significant difference between cohorts (F = 0.46, p = 0.63). These results suggest that although students consider AI tools useful, they remain cautious regarding the reliability of AI-generated information (Figure 2). Experience with Incorrect AI-Generated Information A majority of students reported having encountered incorrect or misleading information generated by AI tools; however, the frequency of these reports differed substantially between cohorts. Among MD2030 students, 37 of 44 students (84.1%) reported such experiences, compared with 24 of 46 students (52.2%) in MD2031 and 7 of 12 students (58.3%) in GEMD2030. Comparison between cohorts using a chi-square test of independence revealed a statistically significant difference (χ2 = 10.73, p = 0.0047). Notably, the MD2031 cohort, who had prior exposure to structured activities involving identification of errors in AI-generated content, reported fewer encounters with incorrect AI outputs than the MD2030 cohort (Figure 3) (Table 6). Encountered Incorrect AI-Generated Information. Percentage of students in each cohort (MD2030, n = 44; MD2031, n = 46; GEMD2030, n = 12) reporting having encountered incorrect or misleading AI-generated information. Reporting differed significantly between cohorts (χ2 = 10.73, p = 0.0047), with the lowest proportion observed in MD2031 (52.2%), which had prior exposure to structured AI literacy activities, compared with MD2030 (84.1%) and GEMD2030 (58.3%). Table 6Experience with AI Errors and Verification PracticesVariableMD2030 (n=44)MD2031 (n=46)GEMD2030 (n=12)p-valueEncountered incorrect AI information37 (84.1%)24 (52.2%)7 (58.3%)0.0047Cross-check AI information29 (65.9%)31 (67.4%)9 (75.0%)n.s.Do not always cross-check14 (31.8%)13 (28.3%)3 (25.0%)Never cross-check1 (2.3%)2 (4.3%)0 (0%) Experience with AI Errors and Verification Practices Verification of AI-Generated Information Most students reported verifying AI-generated information using other sources. 29 of 44 MD2030 students (65.9%), 31 of 46 MD2031 students (67.4%), and 9 of 12 GEMD2030 students (75.0%) reported routinely cross-checking information provided by AI tools (Table 6). Ethical Concerns and Attitudes Toward AI in Medical Education Students reported several concerns regarding the use of AI in medical education. The most frequently reported concerns included dependency on AI tools, potential reduction in critical thinking, and bias in AI-generated content. When asked whether AI tools enhance or replace traditional learning strategies, the majority of students indicated that AI enhances rather than replaces traditional learning methods. Specifically, 31 of 44 MD2030 students (70.5%), 33 of 46 MD2031 students (71.7%), and 9 of 12 GEMD2030 students (75.0%) selected the “enhance” option. Support for integrating AI tools into formal medical education was moderate. 22 of 44 MD2030 students (50.0%), 23 of 46 MD2031 students (50.0%), and 7 of 12 GEMD2030 students (58.3%) expressed support for such integration, while others were uncertain or opposed. Recommendation of AI Tools to Other Students A majority of respondents indicated that they would recommend AI tools to other medical students. 32 of 44 MD2030 students (72.7%), 33 of 46 MD2031 students (71.7%), and 9 of 12 GEMD2030 students (75.0%) reported that they would recommend AI tools for studying. Overall, the results demonstrate that AI tools are widely used by medical students and are perceived as highly useful learning aids, although students remain cautious regarding the reliability of AI-generated information and emphasize the importance of verifying AI outputs through other sources (Table 7). Table 7Attitudes Toward AI in Medical EducationVariableMD2030 (n=44)MD2031 (n=46)GEMD2030 (n=12)AI enhances learning31 (70.5%)33 (71.7%)9 (75.0%)AI replaces learning7 (15.9%)7 (15.2%)1 (8.3%)Neither6 (13.6%)6 (13.0%)2 (16.7%)Support AI in curriculum22 (50.0%)23 (50.0%)7 (58.3%)Recommend AI to others32 (72.7%)33 (71.7%)9 (75.0%) Attitudes Toward AI in Medical Education Free Text Comments Analysis Topic modelling identified five latent topics for Question 17 and six latent topics for Question 24. Table 8 summarises each NLP-derived topic by presenting its six highest-weight terms, one representative student response assigned to the topic, the corresponding human interpretation based on the highest-weight terms and representative responses, and the reporting theme used in the subsequent qualitative analysis. For Question 17, the five topics were consolidated into four reporting themes. Representative responses are included to illustrate the language associated with each topic identified by the NLP model. Table 8NLP-Derived Topics Identified from the Free-Text Responses to Questions 17 and 24NMF TopicHighest-Weight Topic Terms (NMF)Representative ResponseHuman Interpretation of the NMF Topic Based on Top ResponsesReporting ThemeIDCohortResponse17_1Information, book, topic, medical, fake, especially114959651256MD2031Incorrect information was given on book topics or chapters where I could find information from- or sometimes made-up books that had the topics I was looking for to readIncorrect, outdated or fabricated factual information and learning resources1. Incorrect factual information17_2Answer, question, generated, practice, different, structure114832345758MD2030In generated practice questions, it sometimes gives wrong option as right, but when asked for explanation, it fixes mistakes.Difficulties with AI-generated answers to educational questions and question structure2. Misleading explanations17_3Wrong, calculation, mathematical, math, formula, number114836423519MD2030On calculations or the answer to some multiple-choice questionsErrors involving numerical calculations, formulas or quantitative reasoning3. Calculation or numerical errors17_4Not, AI, NEG_correct, just, notes, specific114836306459MD2030Sometimes the answer is not accurate according to the lecture notes and the textbooks.Responses lacking specificity or consistency with teaching material4. Inappropriate or unclear AI responses17_5Incorrect, gave, chemical, solution, chemistry, structure114837020349MD2030When asked to generate a picture of a chemical compound, it gave me something entirely incorrect (incorrect structure, more of a picture than a chemical compound)Incorrect visual or structural scientific information1. Incorrect factual information24_1Improve, accuracy, explanation, information, provide, medical114959651256MD2031Improved accuracy and knowledge of medical curriculum in order to make generation of questions easier and explanation of processes better (being able to form links between)More accurate answers accompanied by better explanations1. Improved accuracy24_2Accurate, information, question, tool, medical, give114832516727MD2030Improve accuracy and giving possible answers to questions instead of just deciding on one thing that would be misleadingBetter explanations, transparency and verifiable information2. Improved explanation quality24_3Integration, medical, curricula, better, curriculum114969091833MD2031Better integration/involvement in med curriculumBetter alignment of AI tools with the medical curriculum and teaching activities3. Better integration with the medical curriculum24_4AI, use, tool, student, provide, source114835363794MD2030Specific guidelines that would prevent students from over-rely on AI, to ensure that the skill of critical thinking is not subduedAppropriate educational use of AI and guidance for students and instructors4. Safeguards against misinformation24_5Interface, user, improve, student, medical114834533139MD2030Improved interface specifically for medical students.Improvements to usability and interface design5. Improved user interface24_6Better, accuracy, source, medical, curriculum, integration114969062805MD2031Improved accuracy, provide resources of where information is gatheredBetter evidence-based information supported by reliable sources6. Clearer sources and referencesNotes: For each topic, the table presents the six highest-weight terms identified by the NMF model, one representative student response assigned to that topic, the corresponding human interpretation derived from the highest-weight terms and representative responses, and the broader reporting theme used to summarise the qualitative findings. NLP-Derived Topics Identified from the Free-Text Responses to Questions 17 and 24 Notes: For each topic, the table presents the six highest-weight terms identified by the NMF model, one representative student response assigned to that topic, the corresponding human interpretation derived from the highest-weight terms and representative responses, and the broader reporting theme used to summarise the qualitative findings. In Q17 responses, issues related to incorrect or misleading AI-generated information were the most frequently reported. The distribution of dominant topics differed significantly between the two MD cohorts (χ2 = 13.48, p = 0.0092), indicating variation in the types of errors reported. In contrast, Q24 responses showed consistent patterns across cohorts, with accuracy improvement and curriculum integration emerging as the most prominent suggestions. The distribution of dominant topics for Q24 did not differ significantly between cohorts (χ2 = 6.25, p = 0.3963), (Table 9). Table 9Summary of NLP-Based Analysis of Open-Ended ResponsesSurvey QuestionNLP MethodMain Themes IdentifiedCohort ComparisonStatistical ResultQ17. “Have you ever encountered incorrect or misleading information generated by AI tools? Please describe”.Topic modelling identified 5 topics, grouped into 4 broader themesIncorrect factual informationMisleading explanationsCalculation or numerical errorsInappropriate or unclear AI responsesThe distribution of dominant themes differed between the MD2030 and MD2031 cohortsχ2 = 13.48, p = 0.0092Q24. “How could AI tools be improved for medical education?”Topic modelling identified 7 topics, grouped into 6 broader themesImproved accuracyImproved explanation qualityBetter integration with the medical curriculumSafeguards against misinformationImproved user interfaceClearer sources and referencesThe distribution of dominant themes did not differ significantly between cohortsχ2 = 6.25, p = 0.3963Sentiment analysis of Q17 and Q24 responsesVADER sentiment analysisQ17 responses were more negative than Q24 responsesNo significant differences in sentiment were observed between the MD cohorts for either questionQ17 vs Q24: Wilcoxon signed-rank test, p < 0.001Q17 between cohorts: p = 0.7725Q24 between cohorts: p = 0.1077 Summary of NLP-Based Analysis of Open-Ended Responses Incorrect factual information Incorrect factual information Misleading explanations Misleading explanations Calculation or numerical errors Calculation or numerical errors Inappropriate or unclear AI responses Inappropriate or unclear AI responses Improved accuracy Improved accuracy Improved explanation quality Improved explanation quality Better integration with the medical curriculum Better integration with the medical curriculum Safeguards against misinformation Safeguards against misinformation Improved user interface Improved user interface Clearer sources and references Clearer sources and references Sentiment analysis indicated that responses were generally neutral to mildly negative. Q17 responses were on average more negative than Q24 responses, reflecting the problem-focused nature of this question. A paired analysis yielded a Wilcoxon signed-rank test, p < 0.001, indicating a significant difference in sentiment between Q17 and Q24 responses. However, differences in sentiment between the MD cohorts were not statistically significant for either question (Q17: p = 0.7725; Q24: p = 0.1077), (Table 9). Correlation analysis showed no significant association between Q17 and Q24 sentiment scores (Spearman ρ ≈ 0.11, p > 0.05), and no significant association between dominant topics across questions. This suggests that the types of AI errors reported did not strongly predict the types of improvements suggested. Discussion The present study examined awareness, use, and perceptions of AI tools among first-year medical students across three cohorts in a single medical school, including two undergraduate-entry cohorts (MD2030 and MD2031) and one graduate-entry cohort (GEMD2030). The results indicate that AI tools have already become a routine component of students’ study practices, with extremely high levels of awareness and widespread adoption across all cohorts. At the same time, students demonstrated a consistently cautious attitude toward the reliability of AI-generated information. These findings provide insight into how early-stage medical students currently engage with AI tools and highlight the potential importance of structured AI literacy training within medical education. A distinctive feature of this study is the comparison of cohorts with and without structured AI literacy exposure. A central finding of the study is the near-universal awareness of AI tools among students, with awareness exceeding 93% in the undergraduate-entry cohorts and reaching 100% in the graduate-entry cohort. This confirms that generative AI technologies are already deeply embedded in the learning environment of contemporary medical students. The dominant use of ChatGPT across all cohorts further reflects the accessibility and versatility of large language models for educational purposes. Students reported using AI tools primarily for activities such as generating explanations, answering medical questions, producing summaries of lecture material, and creating practice questions. These patterns are consistent with the view that AI tools are currently being used primarily as supplementary learning aids rather than as replacements for traditional study resources. Across all three cohorts, students reported high perceived usefulness of AI tools, with mean usefulness scores above 4 on a five-point scale. These results indicate that students view AI as a valuable tool for improving understanding of complex topics and facilitating efficient study practices. The most frequently reported benefits included enhanced understanding of complex concepts and time savings, suggesting that students perceive AI tools as particularly helpful for clarifying difficult material and organizing information. Despite this strong perception of usefulness, trust in AI-generated medical information was consistently more moderate, with mean trust scores around 3 on the same scale. This pattern, high usefulness but moderate trust, suggests that students recognize the limitations of AI tools and do not treat AI-generated outputs as authoritative sources of medical knowledge. Instead, students appear to view AI as a supportive cognitive tool that must be used critically and verified through other sources. Indeed, the majority of students reported that they routinely cross-check AI-generated information with other resources, indicating an awareness of the potential for inaccuracies or misleading responses. This interpretation is supported by the sentiment analysis of free-text responses, where students expressed more negative sentiment when describing incorrect AI outputs than when proposing improvements. This consistent shift suggests that students distinguish between evaluating AI limitations and engaging constructively with its potential uses, reflecting a critical approach to AI tools. The survey results also indicate that encountering incorrect or misleading AI-generated information is common among students, with a majority of respondents reporting such experiences. Interestingly, a statistically significant difference was observed between cohorts in the proportion of students reporting encounters with incorrect AI information. Students in the MD2031 cohort reported substantially fewer encounters with incorrect AI information compared with the MD2030 cohort. This difference is notable in light of an important pedagogical distinction between the two cohorts: MD2031 students had participated in coursework that involved analysing AI-generated text containing intentionally embedded errors and were required to identify and correct these errors through independent research. Analysis of free-text responses showed that differences between cohorts were evident in the types of errors reported (Q17), whereas suggested improvements (Q24) were broadly consistent across cohorts. This suggests that while students’ experiences of AI limitations may vary, their expectations for improvement are shared. Differences in error categorisation may reflect variation in prior exposure to AI-focused educational activities, which could influence how students recognise and interpret AI-generated inaccuracies. Although causal conclusions cannot be drawn, the findings are consistent with the possibility that structured AI literacy exposure may have been associated with how students recognise and interpret AI-generated errors. More specifically, exposure to structured exercises designed to highlight the limitations of AI systems may have improved students’ ability to critically assess AI-generated content or altered their interpretation of what constitutes an AI error. Such training may encourage students to approach AI tools more analytically, thereby reinforcing the importance of verification and critical evaluation. These findings support growing calls within medical education for the integration of AI literacy training into medical curricula. Alternative explanations should also be considered. Increasing public familiarity with generative AI tools, differences between successive student cohorts, variations in prior experience with AI, and ongoing improvements in AI models over time may all have contributed to the observed differences. Consequently, the association between prior AI literacy exposure and students’ reporting of AI-generated inaccuracies should be interpreted cautiously. Longitudinal or controlled intervention studies are needed to determine whether structured AI literacy activities independently influence students’ critical evaluation of AI-generated content. The graduate-entry cohort demonstrated several distinctive patterns. Graduate-entry students reported higher levels of prior exposure to AI tools and more frequent AI usage, with all students in this cohort reporting regular use of AI tools for studying. These differences likely reflect the greater academic and professional experience of graduate-entry students, who may be more accustomed to adopting digital tools as part of independent learning strategies. At the same time, perceived usefulness and trust in AI-generated information were broadly similar across cohorts, suggesting that attitudes toward AI are relatively consistent regardless of educational background. Students also expressed a range of ethical concerns regarding the use of AI in medical education, including potential over-reliance on AI tools, the possibility of reduced critical thinking, and concerns about bias or inaccuracies in AI-generated information. The prominence of these concerns indicates that students are aware of the potential risks associated with uncritical use of AI technologies. At the same time, most students indicated that AI tools enhance rather than replace traditional learning strategies, suggesting that students view AI as a complementary educational resource rather than a substitute for established learning methods. Analysis of free-text responses further indicated that the most prominent themes related to incorrect factual information, misleading explanations, and calculation errors. Suggested improvements focused on accuracy, curriculum integration, and explanation quality. Notably, no significant association was found between the types of errors reported and the types of improvements suggested, indicating that students’ expectations for AI tools are not simply reactive to specific negative experiences but reflect broader, shared educational priorities. Furthermore, the predominantly neutral to mildly negative sentiment scores, together with the observation that the most frequently identified themes in Question 24 concerned improving AI accuracy, curriculum integration, and evidence-based information rather than requests for additional functionality, suggest that students perceive AI as a valuable educational tool but expect improvements in its reliability and pedagogical alignment before its wider integration into medical education. Additionally, no statistically significant differences were observed between the two MD cohorts in either the distribution of the frequent Q24 themes or the sentiment scores. This consistency suggests that the identified students’ priorities, particularly greater accuracy and closer alignment with the medical curriculum, reflect recurring educational needs rather than cohort-specific opinions. These findings provide implications for the future integration of AI tools into medical education. Given the widespread use of AI tools by students, medical educators may need to move beyond policies that simply discourage AI use and instead focus on developing structured approaches to responsible AI engagement. Teaching students how to critically evaluate AI-generated information, recognize potential errors, and integrate AI tools appropriately into their learning processes may become an essential component of modern medical training. Limitations Several limitations of this study should be considered when interpreting the findings. First, the study was conducted within a single institution, which may limit the generalisability of the results to other medical schools or educational contexts. Patterns of AI use and student perceptions may vary depending on institutional policies, access to technology, and the extent to which AI tools are integrated into the curriculum. Second, the sample size of the graduate-entry cohort (GEMD2030) was relatively small compared with the undergraduate-entry cohorts. While this reflects the typical enrolment structure of graduate-entry programs, it limits the statistical power of comparisons involving this group and may have limited statistical power to detect potential differences between cohorts. Third, the study relied on self-reported data, which may be subject to response and recall bias. Students may overestimate or underestimate their use of AI tools, and reported perceptions may not fully reflect actual behaviour. In addition, variation in the interpretation of certain survey items, mainly those related to incorrect AI-generated information, may reflect differences associated with prior exposure. Fourth, the cross-sectional design precludes conclusions about causality. Although differences between cohorts were observed, particularly in relation to reported encounters with incorrect AI-generated content, it is not possible to determine whether these differences are directly attributable to prior exposure to AI-related coursework. Finally, the study focused on student perceptions and self-reported behaviours rather than objective measures of learning outcomes. As such, the findings do not provide direct evidence of the impact of AI use on knowledge acquisition, clinical reasoning, or academic performance. Future research should incorporate longitudinal and controlled study designs to examine these outcomes more directly. Our automated topic analysis employed data-driven criteria to determine the optimal number of topics, without external semantic validation. Although topics were reviewed against example responses to ensure interpretability, no formal inter-rater reliability assessment was conducted, which may limit reproducibility of thematic categorisation. Future work should incorporate independent human coding to further validate topic interpretation. Despite these limitations, this study provides a timely snapshot of how early-stage medical students engage with AI tools in their learning. The findings highlight the widespread adoption of AI, alongside a cautious awareness of its limitations, and suggest that structured educational approaches addressing AI use may support more critical and informed engagement with these technologies. Conclusions This study provides insight into how contemporary medical students perceive and use AI tools in their learning. Across all three cohorts, awareness and use of AI applications were widespread, with most students reporting regular use, particularly of large language models such as ChatGPT, to support their studies. AI tools were consistently perceived as highly useful for explaining complex concepts, generating practice questions, and organizing study material. However, trust in AI-generated medical information remained moderate, indicating awareness of the potential limitations and inaccuracies of these systems. The free-text analysis showed that students reported a range of AI-related errors, most commonly involving incorrect or misleading information, while proposed improvements focused on enhancing accuracy, explanation quality, and integration into the curriculum. Sentiment analysis indicated that responses describing AI limitations were more negative than those proposing improvements. These results suggest a constructive and reflective approach to AI use. No significant association was observed between the types of errors reported and the improvements suggested. The findings suggest that students are already incorporating AI into their learning strategies, regardless of formal instruction. At the same time, structured educational exposure, such as activities requiring critical evaluation of AI-generated content, may be associated with differences in how students interpret and assess AI outputs. This highlights the potential value of integrating AI literacy and critical appraisal skills into medical curricula. Importantly, students generally viewed AI as enhancing rather than replacing traditional learning approaches, positioning it as a complementary educational tool. Nonetheless, concerns regarding over-reliance and potential impacts on critical thinking were evident, underscoring the need for responsible and informed use. As AI technologies continue to evolve and become increasingly embedded in healthcare and education, medical schools should consider developing structured curricular approaches that support critical, ethical, and effective use of AI tools. Future research should explore the impact of AI-focused educational interventions on students’ engagement with AI and their ability to critically evaluate AI-generated medical information. Funding Statement There is no funding to report. Data Sharing Statement The data that support the findings of this study are available from the corresponding author upon reasonable request. Ethics Approval Informed consent to participate was obtained from all participants. The study received ethical approval from the Cyprus National Bioethics Committee (CNBC), (Ethics Approval Number: EEBK EΠ 2025.01.166). The CNBC is an independent body and is not subject to the administrative control of any ministry. Further information is available from the committee website (https://www.bioethics.gov.cy/moh/cnbc/cnbc.nsf/page12_en/page12_en?OpenDocument). Disclosure The authors report no conflicts of interest in this work. References Associated Data Data Availability Statement The data that support the findings of this study are available from the corresponding author upon reasonable request.