Artificial intelligence and academic writing questionnaire (AI-AWQ): development and validation among medical students’ experiences using exploratory factor analysis Authors: Laleh Khojasteh, Zahra Karimian, Elham Nasiri, Reza Kafipour, Amir Yousef Farahmandi Published: 2025 ABSTRACT IntroductionThis study aimed to develop and validate the Artificial Intelligence and Academic Writing Questionnaire (AI-AWQ) to assess participants’ perceptions with AI. The primary focus was to explore the factors that influence attitudes toward AI in educational settings.MethodsThis study utilized a mixed-methods approach to develop and validate the psychometric properties of the AI-AWQ. The questionnaire, consisting of 30 items rated on a 5-point Likert scale (1 = Never, 5 = Always), was administered to a sample of 252 medical and dental students at Shiraz University of Medical Sciences, Iran, who had taken the academic writing course during the 2023–2024 academic year. Data were analyzed using Exploratory Factor Analysis (EFA), with Varimax rotation employed to clarify the underlying factors.ResultsA total of 252 completed questionnaires were analyzed, of which 59.5% were from Iranian students and the remaining respondents were international students. The results of the exploratory factor analysis demonstrated satisfactory sampling adequacy (KMO = 0.930) and a significant Bartlett’s test of sphericity (P < 0.001), confirming the suitability of the data for factor analysis. Construct validity testing led to the extraction of five distinct factors—Perceived Effectiveness of AI, Ethical and Authenticity Concerns, AI-Supported Writing Process, AI Feedback and Writing Enhancement, and Affective and Motivational Impact—which together accounted for 77.99% of the total variance. The questionnaire demonstrated strong validity and reliability, with a Content Validity Index (CVI) of 0.903, a Content Validity Ratio (CVR) of 0.882, and an overall internal consistency confirmed by a Cronbach’s alpha of 0.883.ConclusionThe findings suggest that the AI-AWQ provides preliminary evidence of reliability and validity for measuring perceptions of AI, offering insights into the multi-faceted nature of AI and academic writing. This study contributes to understanding the factors shaping individuals’ views on AI in educational contexts and provides a foundation for further research.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12909-025-08288-z. Introduction The rapid advancement of artificial intelligence (AI) has brought about a significant transformation in various sectors, including academic writing [1–3]. However, while AI technologies, such as writing assistants, plagiarism detectors, and content generators, are increasingly integrated into professional writing practices, the curriculum in higher education has struggled to keep pace, particularly in regions where AI adoption lags [4]. As AI tools gain prominence in academic settings, universities have not yet fully integrated AI literacy into their writing programs, leaving students ill-equipped to engage with these tools ethically and effectively in their academic writing [3, 5, 6]. This lack of formal training not only hinders students’ ability to engage with AI tools effectively but also poses ethical challenges in academic writing, such as concerns about authorship, plagiarism, and maintaining academic integrity [7, 8]. AI tools, while enhancing writing quality and efficiency, can blur the lines between students’ original work and AI-assisted content, raising questions about the authenticity of academic writing [9]. Furthermore, the absence of a structured approach to teaching AI in writing can limit students’ opportunities to develop essential critical thinking and collaborative skills, which are typically fostered through peer review and instructor feedback [10, 11]. As AI tools continue to reshape the landscape of academic writing, universities must integrate AI literacy into their curricula to ensure that students are prepared not only to use these tools responsibly but also to navigate the ethical complexities of writing in an AI-driven world [12, 13]. Given these challenges, universities must adjust their curricula to better prepare students for an AI-driven world. The primary aim of this study is to develop and validate the Artificial Intelligence and Academic Writing Questionnaire (AI-AWQ), which assesses students’ perceptions, experiences, and ethical concerns regarding the use of AI tools in academic writing. By measuring students’ current understanding of AI tools and identifying gaps in their AI literacy, this study seeks to provide actionable insights for curriculum developers and educators, helping to bridge the gap in AI literacy and better prepare students for the demands of the workforce [12–14]. Ultimately, the study advocates for a more forward-thinking curriculum that integrates AI education to equip students with the skills necessary to excel in both their academic and professional careers in an increasingly digital, AI-driven society. In this study, the term ‘AI tools’ broadly refers to artificial intelligence applications used in academic writing, including AI-powered writing assistants (e.g., Grammarly, ChatGPT), plagiarism detectors, and content generators. Literature review This study draws on three key theoretical frameworks to understand the integration of AI tools in academic writing and their impact on students: the Technology Acceptance Model (TAM), Cognitive Load Theory (CLT), and the Community of Inquiry (CoI) framework. These theories offer a comprehensive lens through which to examine how students engage with AI tools in academic settings and the potential consequences of these interactions. TAM and AI writing assistants TAM, developed by Davis [15], has long been a foundational theory in understanding the factors that influence the adoption of new technologies. According to TAM, two primary determinants—perceived ease of use and perceived usefulness—play a central role in technology acceptance. In the context of AI writing assistants, such as Grammarly, ProWritingAid, and ChatGPT, TAM provides a framework for understanding how students perceive AI tools in their academic writing and how these perceptions influence their willingness to adopt and integrate such technologies into their writing practices. Recent studies applying TAM to AI in educational contexts shed important light on the factors driving AI adoption. Nguyen and Dieu [16], for example, explored ELT students’ perceptions of ChatGPT as an AI writing assistant and found that the students’ perceptions of the tool’s usefulness and ease of use were key drivers of adoption. This study aligns with TAM’s central proposition, demonstrating that when students find AI tools beneficial and easy to use, they are more likely to integrate them into their academic tasks. Similarly, research has extended TAM by incorporating elements of self-determination theory (SDT), with a particular focus on the role of intrinsic motivation—specifically, students’ confidence in using AI tools—in their adoption of these tools for academic writing. Studies have shown that students who feel more competent in using AI technologies are more likely to adopt and integrate these tools into their writing practices. For example, Xia, Chiu [17] explored how SDT can guide the design of inclusive AI education, highlighting that students’ intrinsic motivation, including their confidence in using AI, significantly influences their adoption of AI tools in learning environments. Similarly, Chiu and Chai [18] discussed how self-determination theory can inform sustainable curriculum planning for AI education, underscoring that students’ perceived competence in using AI tools drives their willingness to adopt and integrate them into their academic practices. These findings emphasize that students who feel more competent and autonomous in using AI tools are more likely to adopt and benefit from them in academic settings, particularly in writing tasks. In contrast, Papakonstantinidis, Kwiatek [19] introduced social influence as an additional factor within TAM, asserting that the opinions and behaviors of peers significantly affect students’ willingness to adopt AI tools. This dimension is particularly relevant in medical education, where peer-driven learning and collaborative practices are common. If students observe their peers successfully using AI writing tools, they are more likely to follow suit, influenced by social norms and peer validation [20]. While TAM has been successful in explaining the adoption of AI tools, recent studies have pointed out critical limitations, particularly the ethical concerns surrounding the use of AI tools, especially in academic contexts. Liu [21] and Peng, Ding [22], for example, highlighted the growing concerns regarding academic integrity, authorship, and plagiarism in the use of AI tools, such as writing assistants. They found that while AI tools, including Grammarly and ChatGPT, may be perceived as useful, students’ adoption intentions were significantly moderated by these ethical issues. This adds a layer of complexity to TAM, suggesting that in contexts where academic integrity is paramount—such as in medical writing—the perceived usefulness and ease of use may not be sufficient to drive adoption if students do not feel confident about using the tools ethically [1, 23]. Therefore, it is essential to consider both the functional benefits of AI tools and the ethical considerations in encouraging students to use AI technologies responsibly. Incorporating these dimensions into the questionnaire for this study is necessary to gain a comprehensive understanding of the adoption intentions of medical students regarding AI writing assistants. TAM provides the foundational framework for exploring how students perceive AI tools. However, it is essential to expand the inquiry also to consider confidence, social influence, and ethical considerations in the adoption process. Accordingly, several AI-AWQ items were directly informed by TAM constructs—particularly perceived usefulness (e.g., “AI tools help me complete writing tasks more efficiently”) and perceived ease of use (e.g., “I find it easy to use tools like Grammarly or ChatGPT in academic writing”). Items were also developed to reflect extended TAM factors, such as social influence and ethical confidence. Recent studies further support this approach. For instance, Abgaryan, Asatryan [24] applied TAM to Kazakhstani TEFL students’ intention to continue using AI tools, confirming that ease of use and peer support are critical to sustained adoption. Similarly, Marzuki, Widiati [25] found that perceived usefulness and social influence significantly predicted university students’ adoption of AI-powered writing tools, particularly when integrated into formal learning environments. Additionally, Dahri, Yahaya [26] demonstrated that ethical concerns, such as data privacy and academic integrity, moderated the relationship between perceived ease of use and adoption intention, reinforcing the need to extend TAM with contextual factors in educational settings. CLT and AI in academic writing CLT, developed by Sweller [27], posits that learning is most effective when cognitive load is optimized—neither underloading nor overloading the learner’s working memory. In the context of academic writing, CLT suggests that the use of AI writing assistants can help reduce extraneous cognitive load, thus allowing students to focus on higher-order tasks such as argument construction, idea generation, and critical analysis, rather than on repetitive aspects of writing like grammar and syntax [28, 29]. Recent studies in English as a Foreign Language (EFL) and English as a Second Language (ESL) settings have investigated how AI tools, such as Grammarly, ChatGPT, and other writing assistants, affect students’ cognitive load during writing tasks [2, 30, 31]. CLT posits that learners can only process a limited amount of information at once, so managing cognitive load is essential for effective learning [32]. AI tools can help manage this cognitive load, especially for writing tasks, by reducing the extraneous cognitive load associated with mechanics such as spelling and grammar, thereby enabling students to focus on more complex cognitive processes, such as critical thinking and content development [29, 33]. For instance, Yi and Kim [30] in their study on the impact of Computer-Assisted Language Learning (CALL) tools, including Grammarly and ChatGPT, on EFL learners’ writing skills, found that these tools significantly reduced cognitive overload during the writing process. By offering immediate feedback on grammar and syntax, students were able to allocate more mental resources to higher-order cognitive tasks, such as argument construction and idea development. This aligns with CLT, which emphasizes the importance of reducing extraneous cognitive load to allow for germane cognitive load—the mental effort needed to engage in meaningful learning and conceptual understanding [34]. Similarly, Werdiningsih, Marzuki [6] examined how AI-based feedback tools assist students in ESL writing, noting that tools like ChatGPT reduce the time spent on mechanical writing tasks, thereby providing students with more cognitive space to focus on developing coherent and sophisticated arguments. This study reinforced the notion that AI tools can support writing quality by helping master’s students organize their thoughts more efficiently and manage the complexities of academic writing in second-language settings. However, studies also point out the risks of cognitive dependency when students over-rely on AI tools. Anis and Khalid [35] argued that while tools like Grammarly and ChatGPT are beneficial in reducing cognitive load, they may lead to a situation where students fail to develop the independent writing and editing skills needed for academic success. Over-reliance on AI can hinder students’ ability to internalize the writing process, leading to a diminished capacity for critical self-reflection and independent problem-solving —key factors of academic writing [28, 36]. This concern aligns with Sweller’s [27] warning in CLT, where excessive support may lead to the development of cognitive dependency, thus inhibiting the development of students’ own critical thinking abilities. Moreover, Alharbi [37] explored the use of AI tools in EFL writing and found that while these tools significantly improved the mechanical accuracy of students’ writing, they sometimes led to cognitive overload in advanced tasks. This occurs because students attempt to rely on AI for more than just basic grammar correction, integrating AI suggestions in ways that conflict with their original ideas. In advanced writing tasks, this paradox of over-reliance can shift the cognitive burden from the student to the AI tool, further complicating the writing process rather than simplifying it [38]. The findings from these studies demonstrate that AI writing tools, such as Grammarly and ChatGPT, reduce cognitive load, enabling EFL and ESL students to focus on content rather than grammar. However, excessive reliance on these tools may hinder the development of critical writing skills, such as critical thinking and argumentation, which are essential for academic success, especially in fields like medical writing that require higher-order cognitive processes. Incorporating these findings into the current AI literacy questionnaire is essential. The questionnaire should explore how students perceive the usefulness and ease of use of AI tools like Grammarly and ChatGPT, but also assess whether students feel that their independent writing skills are being compromised. Furthermore, questions should be designed to gauge whether students feel that AI tools are helping them manage cognitive load by reducing mechanical tasks, while allowing them to focus on more complex aspects of writing. This will provide a comprehensive understanding of how AI tools are used in the medical writing context and whether they enhance or hinder the development of critical writing skills. In alignment with CLT principles, AI-AWQ items were constructed to assess both reduced extraneous load (e.g., grammatical assistance) and increased germane load (e.g., focus on idea development). Other items explore the potential for cognitive over-reliance on AI, a critical concern in educational contexts. These aspects are supported by recent work such as Shen and Cui [39], which emphasizes the importance of measuring both support and dependency when evaluating AI literacy. Further reinforcing this approach, Gkintoni et al. [40] empirically demonstrated that AI tools optimized for CLT significantly improved learning outcomes—but only when students maintained active engagement with the material. Similarly, Klingbeil, Grützner [41] found that over-reliance on AI-generated feedback led to diminished critical thinking skills among learners, underscoring the need for tools that balance automation with cognitive engagement. CoI framework and AI in education The CoI framework, developed by Garrison and Akyol [42], offers a comprehensive approach to understanding the learning process in online and blended environments. This framework emphasizes the importance of three interconnected types of presence in the learning process: social presence, cognitive presence, and teaching presence. Social presence refers to the ability of students to project themselves as genuine, emotionally engaged participants in the learning process [43, 44]. Cognitive presence is the extent to which students can engage in meaningful learning by constructing and validating their knowledge through inquiry and reflection [43, 45]. Teaching presence involves the design, facilitation, and direction of cognitive and social processes to achieve the intended learning outcomes [43, 46]. In the context of AI and education, AI tools—such as writing assistants like Grammarly and ChatGPT, as well as other AI-powered applications—play a role in shaping these presences in both positive and negative ways. The impact of AI on learning environments and student engagement is complex, as these tools may both support and challenge the balance between social, cognitive, and teaching presence. The role of social presence in AI-enhanced learning environments is a point of contention. Hew, Huang [47] have argued that while AI tools can facilitate cognitive presence and offer personalized feedback, they may diminish the social presence of students. AI-driven platforms, particularly in the context of writing, can reduce the level of interaction between students and their peers or instructors, which is vital in a collaborative learning environment. Syafii, Purwati [48] and Zou, Xie [49] emphasize that when students engage more with AI tools for writing, there is a risk that they may withdraw from peer feedback and teacher interaction, thus hindering the development of collaborative learning skills and the social aspects of academic discourse. However, some recent studies argue that AI tools can still support social presence, albeit in different ways. For example, Pereira, Komarlu [5] pointed out that AI-powered writing assistants can still promote collaboration by providing students with real-time feedback that they can share with peers for further discussion and refinement. They argue that AI tools can enhance social engagement if used to augment peer interactions, such as through shared writing platforms where students use AI to refine their drafts and then engage in collaborative editing and critique. The impact of AI tools on cognitive presence is perhaps the most significant [50]. Cognitive presence refers to the ability of students to engage meaningfully with the content, develop critical thinking skills, and engage in deeper levels of inquiry [43, 45]. Toribio [51] supports the notion that AI tools can enhance cognitive presence by reducing extraneous cognitive load, thus allowing students to focus on higher-order tasks, such as critical analysis, argumentation, and synthesis. AI tools like ChatGPT can assist students in generating ideas, organizing thoughts, and refining arguments, facilitating deeper engagement with the content [12, 23]. In contrast, Zhai, Wibowo [52] expressed concerns about the dangers of over-reliance on AI tools, which could ultimately diminish cognitive presence. If students depend too heavily on AI for content generation or structural organization, they may fail to develop their own problem-solving and analytical thinking skills [53]. This concern is particularly relevant for medical students, who are expected to engage in complex cognitive processes, such as clinical reasoning, research analysis, and synthesis of medical knowledge [54, 55]. AI tools that automate aspects of writing and content creation may reduce the opportunities for students to engage deeply with the material and develop critical academic skills [12]. Moreover, Hosler and Arend [56] suggested that the presence of a facilitator—a teacher or instructor—is critical for maintaining a high level of cognitive presence. While AI tools can provide feedback and guidance, they cannot fully replace the interactive discussions and scaffolded inquiry that occur in face-to-face or instructor-facilitated settings [57]. The integration of AI in writing courses must be coupled with effective teaching presence to ensure that cognitive presence is maintained at an optimal level [58]. Finally, teaching presence, which refers to the instructor’s role in designing and facilitating learning, is deeply impacted by the integration of AI tools. De Freitas, Agarwal [14] emphasized that teachers must adapt their instructional strategies when introducing AI-based tools into the classroom. This adaptation is particularly critical in medical education, where high-stakes writing tasks—such as clinical reports and research papers—require instructional guidance to ensure that students develop not only technical proficiency in writing but also the ability to think critically and engage ethically with AI-assisted technologies [59]. However, the effective use of AI tools for teaching presence depends on how well instructors are trained to integrate these tools into the curriculum. Celik, Dindar [60] found that instructors in regions with slow adoption of AI tools in education often lacked the training and support needed to incorporate AI into their teaching effectively. In such cases, teaching presence shifts from facilitating inquiry to guiding the use of AI tools, which can compromise the depth of students’ learning experiences [60]. The application of the CoI framework to AI-enhanced academic writing is highly relevant to the current study, particularly in understanding how AI tools shape the learning environment for medical students. The integration of AI tools must, therefore, be carefully managed to ensure that they do not replace the interactive and collaborative aspects of learning [47]. In medical education, where collaborative learning and critical thinking are essential, the integration of AI tools must, therefore, be carefully managed to ensure that they do not replace the interactive and collaborative aspects of learning [47]. This balance is vital to prevent potential risks, such as diminished social engagement or over-reliance on AI for cognitive tasks [52]. To reflect the CoI framework in AI-AWQ design, items were written to capture how students perceive AI’s influence on their interaction with peers (social presence), their capacity for reflection and critical thinking (cognitive presence), and the instructional guidance they receive (teaching presence). These constructs were considered essential to evaluating the pedagogical context surrounding AI use. Recent research supports this approach, including Wang, Wu [61], who found that AI-powered agents simulating co-learners in asynchronous online learning enhanced social presence through increased social awareness and interaction, while also improving cognitive presence via AI-generated feedback and shared notes. Similarly, Tankelevitch, Kewenig [62] demonstrated that AI tools prompting metacognitive questioning (e.g., encouraging reflection on AI suggestions) significantly deepened cognitive presence compared to non-interactive systems. These findings underscore the importance of intentionally integrating CoI principles into AI writing tools to foster meaningful learning interactions. Additionally, Annamalai [63] demonstrated that ethical concerns, such as data privacy and academic integrity, have a significant influence on students’ attitudes toward AI tool adoption, underscoring the need to address these factors in educational settings. While the integration of AI tools in education holds tremendous potential for enhancing students’ learning experiences and academic writing skills, a significant curriculum gap exists in many educational systems, particularly in regions where the adoption of AI literacy is slow [28]. This gap stems from the lack of structured instruction on AI tools and the ethical considerations that accompany their use. As highlighted by the CoI framework, the effective integration of AI requires thoughtful attention to social presence, cognitive presence, and teaching presence, all of which depend on well-structured curricular interventions. In the context of medical education, this gap becomes even more pressing, as AI tools are becoming increasingly integral to academic writing, clinical practice, and research development [64]. Addressing this gap by integrating AI literacy into the curriculum is not merely about introducing AI tools but also about preparing students to interact with these tools responsibly and thoughtfully. As the use of AI continues to expand in both academic and professional contexts, educational institutions must develop comprehensive AI literacy programs that equip students with the skills, ethical guidelines, and critical thinking abilities necessary for success in the AI-driven workforce. The objective of this study is to develop and validate the AI-AWQ, a tool designed to assess students’ perceptions, experiences, and ethical concerns regarding the use of AI tools in academic writing, with a focus on medical students. The study aims to address the curriculum gap in AI literacy, particularly in regions with slower AI adoption, by evaluating whether students possess the necessary skills to use AI tools responsibly. By assessing students’ self-perceived competence and ethical concerns, the study will provide actionable insights to inform curriculum revisions, ensuring that students are better equipped to meet the demands of an AI-driven workforce. Ultimately, the AI-AWQ will serve as a diagnostic tool to guide curriculum development, enabling universities to integrate AI literacy into their programs and equip students with the skills needed to interact with AI tools effectively and ethically, thus aligning education with the rapidly evolving technological landscape. Methods Study design This study employed a mixed-methods design to develop, validate, and assess the psychometric properties of AI-AWQ. The rationale for using a mixed-methods approach was to capitalize on the complementary strengths of both qualitative and quantitative methodologies, allowing for a more nuanced understanding of the research topic [65]. This comprehensive design included both qualitative and quantitative phases, ensuring rigorous validation of the research instrument. The qualitative phase involved expert review to assess the content validity of the AI-AWQ, ensuring that the questionnaire items accurately reflect the dimensions of AI usage in academic writing. The quantitative phase consisted of several stages, including pilot testing, Exploratory Factor Analysis (EFA), and an examination of the instrument’s reliability and construct validity. The primary objective of the study was to conduct a comprehensive psychometric evaluation of the AI-AWQ using 285 sample of medical students enrolled in an academic writing course. This allowed the study to investigate students’ perceptions and experiences with AI tools in academic writing, while also ensuring that the instrument was both reliable and valid. By utilizing EFA, the study aimed to identify underlying factors related to students’ engagement with AI tools in writing, thereby enhancing the theoretical and empirical understanding of AI’s role in second-language writing. This design was particularly suitable for developing a valid, culturally responsive questionnaire that could capture a broad range of experiences and concerns regarding AI use in academic writing, ultimately contributing valuable insights to the growing body of research in AI-assisted writing and medical education [66]. Participants The study involved three distinct groups of participants, all from Shiraz University of Medical Sciences (SUMS), Iran. Each group played a specific role in different phases of the study, contributing to the development and validation of AI-AWQ. Expert panel The expert panel consisted of 10 specialists selected for their expertise in relevant fields. This group comprised five specialists in English language education, two experts in e-learning, one expert in medical education, and two experts in Instrument psychometrics. This composition was deliberately designed to ensure the questionnaire’s validity across multiple dimensions. The English language experts provided crucial input on writing pedagogy and linguistic accuracy, the e-learning specialists evaluated the technological integration aspects, the medical educator ensured clinical relevance, and the measurement experts guaranteed psychometric robustness. The greater representation of English language specialists reflects the instrument’s primary focus on assessing writing assistance capabilities, which requires more profound expertise in language acquisition and composition than medical content knowledge at this stage of development. These experts provided valuable insights into the content validity of the questionnaire during the initial phase of development. Pilot study group A sample of 35 medical students participated in the pilot study. These students had recently completed the compulsory 3-unit academic writing course and had experience using AI tools in their academic writing. The pilot study aimed to assess the face validity and initial reliability of the questionnaire before its broader implementation. To ensure diversity, the pilot group included both male and female students across different academic years (preclinical and clinical phases). The participants also varied in their English language proficiency levels. They reported differing degrees of AI tool use, ranging from casual use of grammar checkers to more advanced use of AI-driven writing assistants, such as ChatGPT. This range allowed us to test the instrument’s clarity and relevance across varying learner profiles. Main study participants The study population consisted of all 285 medical and dentistry students enrolled in the compulsory academic writing course at SUMS during the 2023–2024 academic year. All participants were fifth-semester medical and dental students who had completed both general and specialized English courses and were enrolled in the academic writing course. Sampling The sampling strategy followed established recommendations for instrument validation studies, which suggest a sample size of 3 to 10 participants per questionnaire item. For the 30-item questionnaire, a minimum of eight participants per item (i.e., 240 students) was considered sufficient. However, to prevent potential sample loss, all 285 eligible students were invited to participate. After data collection, 252 questionnaires were fully completed, resulting in an 88% response rate. A census sampling method was employed to ensure that all students enrolled in the course had the opportunity to participate. Inclusion/exclusion criteria Eligible participants were fifth-semester medical and dental students who had completed both general and specialized English courses and were currently enrolled in the academic writing course. To qualify for inclusion, students needed to: (1) be enrolled at SUMS (2), have completed the academic writing course (3), participate voluntarily (4), have access to a device capable of using AI-based tools such as ChatGPT or Copilot, and (5) possess basic familiarity with AI-assisted writing platforms. Students who did not meet these criteria or left more than 20% of the questionnaire unanswered were excluded from the study. These criteria ensured that only participants capable of providing accurate and meaningful responses were included in the final analysis. Tools/Instruments The research tool used was a researcher-developed questionnaire designed to examine the use of Artificial intelligence in academic writing. The AI and Academic Writing Questionnaire (AI-AWQ) consists of 30 items on a 5-point Likert scale (Always = 5, Often = 4, Sometimes = 3, Rarely = 2, Never = 1), with a cut-off score of 3 (Appendix 1). To validate the tool, initial face validity was established using feedback from 35 students participating in English language courses at the university. Subsequently, after completing the course and the questionnaires, construct validity was assessed through EFA. Moreover, the reliability of the instrument was evaluated through the internal consistency of its items, as indicated by Cronbach’s alpha coefficient. The initial item pool was generated through a comprehensive literature review, focusing on constructs pertinent to AI-assisted academic writing. A panel of 10 experts in English language education reviewed the item pool and extracted the initial list of items. After the initial development of items, the next step was to conduct an expert review to validate them. Another panel of 10 experts was formed, comprising professionals in English language education, e-learning, medical education, and assessment tools. Their role was to evaluate the 65 preliminary items, focusing on their relevance, clarity, and consistency with theoretical frameworks. This review occurred in a group setting over two weeks, during which the experts collaboratively assessed the items. Face validity In this phase, face validity was assessed through pilot testing with a sample of five students, who provided qualitative feedback on item clarity and relevance. Two international students and three Iranian (Persian language students commented on grammatical issues and sentence simplification. Their suggested revisions were incorporated, and the revised items were subsequently re-approved by them. Content validity To determine the content validity of the questionnaire, the opinions of 10 experts were utilized, including five specialists in English language education, two in e-learning, one in medical education, and two in measurement tools. Content validity was assessed using the Content Validity Index (CVI) and the Content Validity Ratio (CVR). CVR This index helps determine whether an item on a questionnaire or test is essential for measuring the intended construct. The necessity of each item was assessed on three levels: Essential (3), Useful but not essential (2), and Not essential (1). Only items that received a score of 3 were included in the calculations. Considering that opinions were gathered from 10 experts, the minimum expected CVR is 0.625. CVI The Content Validity Index was also evaluated using three criteria —relevance, clarity, and simplicity —from the perspectives of 10 specialists. Experts were asked to express their opinions on a scale from 1 to 4 for each item. To calculate the CVI, only the scores of experts who rated an item with a score of 3 or 4 were divided by the total number of experts who rated the item. The expected minimum ratio for CVI is 0.790. Reliability Cronbach’s alpha test was used to determine the internal consistency among the questionnaire items. This statistical measure assesses how closely related a set of items is as a group, indicating the reliability of the scale. Construct validity This was assessed in the final stage using EFA. A summary of the steps for validating the instrument is presented in Fig. 1. Fig. 1Framework of development and validation of AI-AWQ Framework of development and validation of AI-AWQ Data collection procedures Given that the final exam for the compulsory academic writing course is always conducted using a pen-and-paper format, we opted to distribute a printed version of the questionnaire alongside the final exam prompt. This approach ensured that all medical and dentistry students had equal access and sufficient time to complete the questionnaire in a controlled and standardized environment. The students were informed that completing the questionnaire was entirely voluntary and had no bearing on their performance on the final exam. They were encouraged not to feel pressured about filling out the questionnaire during the examination period. After completing their essay exam, students were instructed to detach the questionnaire from their exam papers and place the completed, anonymous questionnaires on the desks at the back of the hall before leaving. The questionnaires contained no identifying information (e.g., names or student numbers), and submissions were made separately from the exam sheets to ensure that individual responses could not be traced back to specific students. This ensured both anonymity and confidentiality, allowing students to share their responses without fear of retribution. To facilitate their participation, students were provided with clear verbal and written instructions prior to the exam. The instructions emphasized that participation was voluntary, and opting out of the questionnaire would not impact their exam results. The questionnaire did not require any identifying information, ensuring full anonymity. Additionally, time would be allocated after the exam to allow students to complete the questionnaire without feeling rushed. This method ensured that all eligible students were given an equal opportunity to participate in the study, while also addressing ethical concerns related to voluntary participation and confidentiality. Ethical considerations Ethical approval for the study was obtained from the Research Ethics Committee of SUMS (Approval ID: IR.SUMS.REC.1403.299). The participants were informed about the objectives of the study and assured that their responses would remain anonymous and confidential. They were also informed that their decision to participate in the study would not influence their course grades or academic standing in any way. By conducting data collection in this manner, we aimed to minimize potential barriers to participation and ensure the study’s integrity while respecting the autonomy and privacy of the participants. Data analysis Data analysis was conducted using SPSS software, version 26, for Exploratory Factor Analysis. EFA was selected as an initial data reduction technique to identify the main factors’ structure, which is common practice in early questionnaire development. A confidence level of 95% and a permissible error rate of 0.05 were considered. To determine content validity, descriptive indices were used. Results Demographic characteristics Of the 285 students enrolled in the course, 252 participants completed the questionnaire in full and were included in the analysis (response rate: 88%). Among them, 87 were male (34.5%) and 165 were female (65.5%), with ages ranging from 20 to 23 years (mean = 21.67 ± 0.96). All participants were familiar with AI tools such as ChatGPT and Copilot and had prior experience using these tools for academic writing (Table 1). Table 1Demographic characteristics of participantsCharacteristicsGroupsFrequencyPercentageGenderFemale16565.50Male8734.50NationalityIran15059.50Other country (International)10240.50Field of studyMedicine17368.70Dentistry7931.30Familiar with AI tools (ChatGPT, Copilot, etc.)Familiar252100.0Not familiar00.00Age (Years)20 Y2811.121 Y8834.922 Y7429.423 Y6224.6Minimum ± SD21.67 ± 0.96Year of StudyFifth-semester medical and dental students Table 1Demographic characteristics of participantsCharacteristicsGroupsFrequencyPercentageGenderFemale16565.50Male8734.50NationalityIran15059.50Other country (International)10240.50Field of studyMedicine17368.70Dentistry7931.30Familiar with AI tools (ChatGPT, Copilot, etc.)Familiar252100.0Not familiar00.00Age (Years)20 Y2811.121 Y8834.922 Y7429.423 Y6224.6Minimum ± SD21.67 ± 0.96Year of StudyFifth-semester medical and dental students Demographic characteristics of participants Instrument development Step 1: identification and development of the initial item pool The development of the AI-AWQ (Appendix 1) began with an extensive literature review on AI applications in language learning, particularly academic writing. The review identified 65 key studies that explored the impact of AI tools, such as ChatGPT and Copilot, on writing skills, academic integrity, and learning behaviors. To ensure a comprehensive questionnaire, an initial pool of 65 items was developed by three subject-matter experts in EFL writing and educational technology, incorporating diverse perspectives. These items addressed topics such as students’ perceived competence and motivation to use AI tools (informed by Self-Determination Theory/SDT), as well as the cognitive load and ethical concerns related to AI’s impact on writing (drawing on CLT and CoI framework). These theories provided a more nuanced exploration of AI’s influence on students’ writing processes, ensuring the questionnaire comprehensively addressed both practical and ethical dimensions. Step 2: expert review and questionnaire refinement Following the initial item development, the next phase involved an expert review to ensure the validity of the items. Ten expert panels were tasked with reviewing the 65 initial items, providing feedback on their relevance, clarity, and alignment with the theoretical constructs. The review took place in a group session spanning two weeks, during which the experts collaboratively evaluated the items. During this period, they discussed the content and structure of the items, identifying redundancies, ambiguities, and areas that needed further refinement. The expert panel employed an iterative process, analyzing the items based on content similarities and their relevance to the research objectives. This led to several rounds of revisions, which ultimately reduced the item pool from 65 to 30 items. The experts also assessed whether each item effectively captured the theoretical constructs, ensuring the questionnaire was both valid and comprehensive. The review process, which consisted of three sessions over two weeks, was crucial in refining the questionnaire to ensure its relevance and alignment with the study’s objectives. During this process, the expert panel also recommended revising the questionnaire’s response format. Initially, the items had varying scales that could potentially confuse respondents. To standardize the responses, a uniform 5-point Likert scale was introduced, with ratings ranging from Always [5] to Never [1]. This adjustment aimed to enhance response consistency and improve the interpretability of the data [67]. In addition to these revisions, eight items were modified to improve clarity. For example, an item initially stated: “AI tools have significantly improved my syntactic structures and grammatical accuracy.” This phrasing was deemed too technical, especially for students without extensive linguistic expertise. It was revised to: “AI tools have significantly improved my grammar and sentence structure in my writing.” This rewording ensured that the statement was accessible to all participants, particularly those with English as a second language, thereby minimizing any potential confusion and enhancing the reliability of the responses [68]. Furthermore, some items were removed for redundancy or irrelevance. A total of 15 items were identified as repetitive or overlapping with other items that assessed similar constructs. For instance, one item—“I am concerned that using AI tools in my writing may lead to negative perceptions or judgments, which could affect how my work is viewed in academic or professional settings”—was found to overlap with other items concerning academic integrity and authenticity of AI-assisted writing. By removing such redundant items, we minimized respondent fatigue and ensured that participants could focus on the most pertinent questions, thus enhancing response quality and reducing cognitive overload [67]. These refinements and item reductions ensured that the questionnaire was not only concise but also culturally relevant and cognitively manageable for students, which are key factors in maintaining high-quality data collection in questionnaire-based research [69]. Step 3: face and content validity Face validity To assess the face validity of the questionnaire, the initial version (derived from expert input) was provided to 35 medical students at SUMS. The feedback collected from these students was invaluable for refining the questionnaire. The following revisions were madeGrammatical Corrections: Eight items were identified as needing grammatical adjustments.Clarity Improvements: Five items were rewritten to improve clarity and simplify language. For instance, technical terminology was reduced to ensure that students could easily understand the questions.Item Appropriateness: No items were found to be inappropriate or requiring removal. Grammatical Corrections: Eight items were identified as needing grammatical adjustments. Grammatical Corrections: Eight items were identified as needing grammatical adjustments. Clarity Improvements: Five items were rewritten to improve clarity and simplify language. For instance, technical terminology was reduced to ensure that students could easily understand the questions. Clarity Improvements: Five items were rewritten to improve clarity and simplify language. For instance, technical terminology was reduced to ensure that students could easily understand the questions. Item Appropriateness: No items were found to be inappropriate or requiring removal. Item Appropriateness: No items were found to be inappropriate or requiring removal. Content validity Content validity was assessed using the CVI and the CVR. The expert panel evaluated the relevance, simplicity, and clarity of each itemCVI: The Content Validity Index for each item was calculated, with all items demonstrating values over 80%, indicating good content validity [70].CVR: The Content Validity Ratio ranged from 0.60 to 1.00, confirming that the items were considered appropriate by the expert panel. CVI: The Content Validity Index for each item was calculated, with all items demonstrating values over 80%, indicating good content validity [70]. CVI: The Content Validity Index for each item was calculated, with all items demonstrating values over 80%, indicating good content validity [70]. CVR: The Content Validity Ratio ranged from 0.60 to 1.00, confirming that the items were considered appropriate by the expert panel. CVR: The Content Validity Ratio ranged from 0.60 to 1.00, confirming that the items were considered appropriate by the expert panel. Reliability The internal consistency of the items was initially assessed in the pilot phase using Cronbach’s alpha with 40 participants. The Cronbach’s alpha value of 0.949 indicated strong reliability for the instrument. Following the full distribution of the questionnaire, the reliability was re-evaluated with a larger sample of 252 participants, and the final reliability value was 0.883, which remains high and supports the consistency of the questionnaire’s results. Additionally, the reliability of subcomponents was also checked, confirming the robustness of the questionnaire’s individual dimensions. Table 2 below summarizes the content validity (CVI/CVR), as well as the overall reliability of the AI-AWQ:Table 2Characteristics of content validity and reliability of AI-AWQItemsCVRCVIReliability (Cronbach’s Alpha)RelevanceSimplicityClarityIf Item deletedQ10.901.000.900.900.875Q21.001.000.900.900.883Q31.001.000.900.800.879Q41.001.000.900.900.879Q50.800.900.800.900.876Q60.900.900.700.900.880Q70.801.000.800.900.880Q80.901.000.800.900.877Q90.801.000.800.900.877Q100.900.800.800.800.878Q111.000.800.800.800.892Q121.001.001.000.800.886Q130.901.000.801.000.878Q141.000.900.800.900.887Q151.001.001.001.000.878Q160.901.001.001.000.890Q170.901.001.001.000.889Q180.901.001.001.000.879Q190.900.800.801.000.878Q200.901.000.701.000.878Q210.901.000.801.000.879Q220.900.900.900.900.883Q230.700.700.700.800.879Q241.000.700.700.800.874Q250.801.001.001.000.874Q260.800.700.600.700.874Q271.001.001.000.900.874Q280.800.700.700.600.872Q290.901.000.800.700.883Q300.901.000.800.700.875Total ScoreCVR Total = 0.9030.9270.8400.8800.883CVI Total = 0.882Reliability Total = 0.883 Table 2Characteristics of content validity and reliability of AI-AWQItemsCVRCVIReliability (Cronbach’s Alpha)RelevanceSimplicityClarityIf Item deletedQ10.901.000.900.900.875Q21.001.000.900.900.883Q31.001.000.900.800.879Q41.001.000.900.900.879Q50.800.900.800.900.876Q60.900.900.700.900.880Q70.801.000.800.900.880Q80.901.000.800.900.877Q90.801.000.800.900.877Q100.900.800.800.800.878Q111.000.800.800.800.892Q121.001.001.000.800.886Q130.901.000.801.000.878Q141.000.900.800.900.887Q151.001.001.001.000.878Q160.901.001.001.000.890Q170.901.001.001.000.889Q180.901.001.001.000.879Q190.900.800.801.000.878Q200.901.000.701.000.878Q210.901.000.801.000.879Q220.900.900.900.900.883Q230.700.700.700.800.879Q241.000.700.700.800.874Q250.801.001.001.000.874Q260.800.700.600.700.874Q271.001.001.000.900.874Q280.800.700.700.600.872Q290.901.000.800.700.883Q300.901.000.800.700.875Total ScoreCVR Total = 0.9030.9270.8400.8800.883CVI Total = 0.882Reliability Total = 0.883 Characteristics of content validity and reliability of AI-AWQ Based on the results in Table 2, all items demonstrate adequate content validity, with a CVI of 0.882 and a CVR of 0.903 reported. The overall reliability was confirmed with a Cronbach’s alpha of 0.883. Except for items 10, 16, and 17, removing any other item resulted in a decrease in the reliability value, indicating the appropriateness of the remaining items. Step 4: construct validity – EFA Given that the AI-AWQ is a researcher-developed instrument (Appendix 1), it was essential to assess its construct validity using EFA. Exploratory Factor Analysis was used to reduce the number of variables and extract the main meaningful latent factors of the data [71]. For the rotation technique, Varimax rotation was applied. This is an orthogonal rotation method that simplifies the factor structure by maximizing the variance of the squared loadings across factors, resulting in factors that are uncorrelated and easier to interpret. Two criteria were used to determine the number of factors to retain: Eigenvalue greater than 1 rule (Kaiser criterion): Factors with eigenvalues exceeding 1 were retained, as they explain more variance than an individual observed variable.Scree plot inflection point: The plot of eigenvalues was visually inspected to identify the point where the curve flattens, indicating the optimal number of factors to keep [72]. Eigenvalue greater than 1 rule (Kaiser criterion): Factors with eigenvalues exceeding 1 were retained, as they explain more variance than an individual observed variable. Eigenvalue greater than 1 rule (Kaiser criterion): Factors with eigenvalues exceeding 1 were retained, as they explain more variance than an individual observed variable. Scree plot inflection point: The plot of eigenvalues was visually inspected to identify the point where the curve flattens, indicating the optimal number of factors to keep [72]. Scree plot inflection point: The plot of eigenvalues was visually inspected to identify the point where the curve flattens, indicating the optimal number of factors to keep [72]. A. Initial Criteria Before conducting the EFA, the suitability of the data was assessed using two key tests: Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy and Bartlett’s Test of Sphericity (Table 3).KMO Measure: The KMO statistic was 0.930, which is considered excellent and indicates that the sample size is sufficient for EFA [73].Bartlett’s Test of Sphericity: This test was significant (p < 0.001), confirming that the correlations between items were sufficiently strong to proceed with EFA [71].Table 3KMO and bartlett’s testKaiser-Meyer-Olkin Measure of Sampling Adequacy.0.930Bartlett’s Test of Sphericity N = 252Approx. Chi-Square7237.476df435Sig.< 0.001 KMO Measure: The KMO statistic was 0.930, which is considered excellent and indicates that the sample size is sufficient for EFA [73]. KMO Measure: The KMO statistic was 0.930, which is considered excellent and indicates that the sample size is sufficient for EFA [73]. Bartlett’s Test of Sphericity: This test was significant (p < 0.001), confirming that the correlations between items were sufficiently strong to proceed with EFA [71]. Bartlett’s Test of Sphericity: This test was significant (p < 0.001), confirming that the correlations between items were sufficiently strong to proceed with EFA [71]. Table 3KMO and bartlett’s testKaiser-Meyer-Olkin Measure of Sampling Adequacy.0.930Bartlett’s Test of Sphericity N = 252Approx. Chi-Square7237.476df435Sig.< 0.001 KMO and bartlett’s test These results indicate that the data were suitable, and the sample size of 252 participants was adequate for factor extraction (Table 3). B. Item Fit and EFA criteria The Exploratory Factor Analysis as the extraction method, and the results confirmed the construct validity of the questionnaire. Items were grouped based on their commonality values, which measure the variance shared by items in relation to their respective factors. Items commonality: After refining the initial item pool and reducing the number of items, the next crucial step was to evaluate the commonality of each item in the AI-AWQ. This stage focused on examining how well each item correlated with the underlying constructs they were meant to measure. Item commonality refers to the extent to which each item shares variance with the other items in the questionnaire. In the current study, a thorough EFA was conducted to assess the commonalities of the 30 items retained in the final version of the AI-AWQ (Appendix 1). As detailed in Table 4 (below), the commonality values for each item in the questionnaire ranged from 0.702 to 0.895, which confirms that all items met or exceeded the expected threshold of 0.5. This strong commonality indicates that the items have adequate internal consistency and collectively measure the core constructs related to AI tools and academic writing, without significant redundancy or irrelevant variance. Table 4Items commonalities of AI-AWQItemsExtractionItemsExtractionItemsExtractionQ10.732Q110.801Q210.594Q20.816Q120.767Q220.706Q30.660Q130.693Q230.907Q40.641Q140.768Q240.665Q50.844Q150.767Q250.733Q60.581Q160.830Q260.801Q70.655Q170.715Q270.784Q80.827Q180.656Q280.796Q90.696Q190.784Q290.722Q100.646Q200.799Q300.659Extraction Method: Exploratory Factor Analysis. Expected value: Item communalities > 0.5 Table 4Items commonalities of AI-AWQItemsExtractionItemsExtractionItemsExtractionQ10.732Q110.801Q210.594Q20.816Q120.767Q220.706Q30.660Q130.693Q230.907Q40.641Q140.768Q240.665Q50.844Q150.767Q250.733Q60.581Q160.830Q260.801Q70.655Q170.715Q270.784Q80.827Q180.656Q280.796Q90.696Q190.784Q290.722Q100.646Q200.799Q300.659Extraction Method: Exploratory Factor Analysis. Expected value: Item communalities > 0.5 Items commonalities of AI-AWQ Extraction Method: Exploratory Factor Analysis. Expected value: Item communalities > 0.5 This result aligns with the recommendations of Cooper [72], who emphasize that Items commonalities should ideally be greater than 0.5. Furthermore, as noted by Schipolowski and Böhme [74], achieving high commonality scores reflects construct validity, indicating that the instrument effectively captures the conceptual dimensions of AI-AWQ. Total variance explained: In Exploratory Factor Analysis, one of the key indicators is the eigenvalue, which determines how many factors the questionnaire can be classified into. As shown in Table 5, the 30 items of the questionnaire explain 77.996% of the variance, with five factors extracted. This indicates that the AI-AWQ accounts for approximately 78% of the total construct being measured. The total variance explained by the five factors indicates a solid fit for the model, supporting the multi-dimensional nature of the instrument. Table 5Total variance explained of AI-AWQFactorsInitial EigenvaluesExtraction Sums of Squared LoadingsRotation Sums of Squared LoadingsTotal% of VarianceCumulative %Total% of VarianceCumulative %Total% of VarianceCumulative %111.36337.87637.87611.12737.09037.0905.74019.13319.13326.05920.19658.0725.77919.26356.3535.42518.08537.21832.6518.83766.9092.3707.90264.2554.24514.14851.36642.0516.83673.7461.7575.85670.1113.33211.10762.47351.2754.25077.9961.0133.37573.4863.30411.01373.48660.5341.77979.77570.4991.66381.438Extraction Method: Exploratory Factor Analysis Table 5Total variance explained of AI-AWQFactorsInitial EigenvaluesExtraction Sums of Squared LoadingsRotation Sums of Squared LoadingsTotal% of VarianceCumulative %Total% of VarianceCumulative %Total% of VarianceCumulative %111.36337.87637.87611.12737.09037.0905.74019.13319.13326.05920.19658.0725.77919.26356.3535.42518.08537.21832.6518.83766.9092.3707.90264.2554.24514.14851.36642.0516.83673.7461.7575.85670.1113.33211.10762.47351.2754.25077.9961.0133.37573.4863.30411.01373.48660.5341.77979.77570.4991.66381.438Extraction Method: Exploratory Factor Analysis Total variance explained of AI-AWQ Extraction Method: Exploratory Factor Analysis As evident from the Scree Plot (Fig. 2), five factors were retained, with the eigenvalue being above 1, suggesting that the model captures the essential dimensions of the construct. Fig. 2Scree plot of AI-AWQ Scree plot of AI-AWQ The Scree plot visually confirms the extraction of five factors, with the eigenvalue of each factor being above 1, validating the number of factors retained. Rotated factor matrix: The Rotated factor Matrix (Table 6) illustrates the internal correlations of the items within each factor. Items that load highly on a particular factor are considered strong indicators of that factor. As shown in Table 6, all items exhibit factor loadings above the 0.40 threshold, with most substantially exceeding this level. This indicates that each item contributes meaningfully to its respective factor (Table 6). Based on the results from Table 6, the following factors and their associated items have been identified (Appendix 1)Perceived Effectiveness of AI: Items 11, 12, 14, 16, 17, 22, and 29 load highly on this factor, reflecting students perceived effectiveness of AI.Ethical and Authenticity Concerns: Items 24, 25, 26, 27, 28, and 30 form this factor, emphasizing the ethical aspects of AI.AI-Supported Writing Process: Items 1, 2, 3, 4, 5, 6, 7 and 9, which focus on the AI-Supported Writing Process.AI Feedback and Writing Enhancement: Items 8, 10, 13, and 15 address AI Feedback and Writing Enhancement.Affective and Motivational Impact: Items 18, 19, 20, 21, and 23 reflect users’ experiences with Affective and Motivational aspects of AI which collectively form this final factor.Table 6Rotated factor matrix of AI-AWQItemsConstruct validity and Exploratory Factor Analysis (EFA)Reliability of ComponentsPerceived EffectivenessEthical and AuthenticityAI-Supported WritingFeedback and EnhancementAffective and MotivationQ110.846−0.1600.091−0.069−0.217α = 0.921Q120.860−0.0960.1240.0430.023Q140.849−0.0790.1540.001−0.132Q160.843−0.2370.187−0.006−0.167Q170.809−0.1410.136−0.013−0.150Q220.587−0.1510.138−0.3840.415Q290.5640.3380.319−0.432−0.044Q24−0.1130.6930.1670.2590.278α = 0.940Q25−0.0400.7870.2150.1090.233Q26−0.2550.7630.0990.2880.250Q27−0.1050.8160.1530.1750.230Q28−0.1630.7350.2080.3030.307Q30−0.1260.7370.1490.2100.185Q01−0.3170.4240.4990.4110.185α = 0.871Q020.452−0.0090.735−0.260−0.055Q030.0520.1980.7780.1120.020Q040.2040.1360.762−0.014−0.006Q05−0.4110.4030.5250.4650.147Q060.1790.1090.729−0.073−0.009Q070.2720.0590.758−0.0520.028Q09−0.3570.4220.5290.3090.122Q08−0.4660.4020.4220.4760.208α = 0.841Q100.1570.3040.0020.7160.127Q130.0170.350−0.0600.7350.159Q15−0.1720.459−0.0470.7030.174Q18−0.1780.307−0.0780.2110.693α = 0.906Q19−0.2720.2990.0480.2300.752Q20−0.3290.3890.0080.3030.668Q210.1620.2740.118−0.1220.681Q23−0.3740.388−0.0870.3400.702Reliability of Factors:• Perceived Effectiveness of AI (α = 0.921)• Ethical and Authenticity Concerns (α = 0.940)• AI-Supported Writing Process (α = 0.871)• AI Feedback and Writing Enhancement (α = 0.841)• Affective and Motivational Impact (α = 0.906)The bold values represent the main factor loadings of the items, which are greater than 0.5, while the italicized values indicate acceptable loadings that range between 0.4 and 0.5 Perceived Effectiveness of AI: Items 11, 12, 14, 16, 17, 22, and 29 load highly on this factor, reflecting students perceived effectiveness of AI. Perceived Effectiveness of AI: Items 11, 12, 14, 16, 17, 22, and 29 load highly on this factor, reflecting students perceived effectiveness of AI. Ethical and Authenticity Concerns: Items 24, 25, 26, 27, 28, and 30 form this factor, emphasizing the ethical aspects of AI. Ethical and Authenticity Concerns: Items 24, 25, 26, 27, 28, and 30 form this factor, emphasizing the ethical aspects of AI. AI-Supported Writing Process: Items 1, 2, 3, 4, 5, 6, 7 and 9, which focus on the AI-Supported Writing Process. AI-Supported Writing Process: Items 1, 2, 3, 4, 5, 6, 7 and 9, which focus on the AI-Supported Writing Process. AI Feedback and Writing Enhancement: Items 8, 10, 13, and 15 address AI Feedback and Writing Enhancement. AI Feedback and Writing Enhancement: Items 8, 10, 13, and 15 address AI Feedback and Writing Enhancement. Affective and Motivational Impact: Items 18, 19, 20, 21, and 23 reflect users’ experiences with Affective and Motivational aspects of AI which collectively form this final factor. Affective and Motivational Impact: Items 18, 19, 20, 21, and 23 reflect users’ experiences with Affective and Motivational aspects of AI which collectively form this final factor. Table 6Rotated factor matrix of AI-AWQItemsConstruct validity and Exploratory Factor Analysis (EFA)Reliability of ComponentsPerceived EffectivenessEthical and AuthenticityAI-Supported WritingFeedback and EnhancementAffective and MotivationQ110.846−0.1600.091−0.069−0.217α = 0.921Q120.860−0.0960.1240.0430.023Q140.849−0.0790.1540.001−0.132Q160.843−0.2370.187−0.006−0.167Q170.809−0.1410.136−0.013−0.150Q220.587−0.1510.138−0.3840.415Q290.5640.3380.319−0.432−0.044Q24−0.1130.6930.1670.2590.278α = 0.940Q25−0.0400.7870.2150.1090.233Q26−0.2550.7630.0990.2880.250Q27−0.1050.8160.1530.1750.230Q28−0.1630.7350.2080.3030.307Q30−0.1260.7370.1490.2100.185Q01−0.3170.4240.4990.4110.185α = 0.871Q020.452−0.0090.735−0.260−0.055Q030.0520.1980.7780.1120.020Q040.2040.1360.762−0.014−0.006Q05−0.4110.4030.5250.4650.147Q060.1790.1090.729−0.073−0.009Q070.2720.0590.758−0.0520.028Q09−0.3570.4220.5290.3090.122Q08−0.4660.4020.4220.4760.208α = 0.841Q100.1570.3040.0020.7160.127Q130.0170.350−0.0600.7350.159Q15−0.1720.459−0.0470.7030.174Q18−0.1780.307−0.0780.2110.693α = 0.906Q19−0.2720.2990.0480.2300.752Q20−0.3290.3890.0080.3030.668Q210.1620.2740.118−0.1220.681Q23−0.3740.388−0.0870.3400.702Reliability of Factors:• Perceived Effectiveness of AI (α = 0.921)• Ethical and Authenticity Concerns (α = 0.940)• AI-Supported Writing Process (α = 0.871)• AI Feedback and Writing Enhancement (α = 0.841)• Affective and Motivational Impact (α = 0.906)The bold values represent the main factor loadings of the items, which are greater than 0.5, while the italicized values indicate acceptable loadings that range between 0.4 and 0.5 Rotated factor matrix of AI-AWQ Reliability of Factors: • Perceived Effectiveness of AI (α = 0.921) • Ethical and Authenticity Concerns (α = 0.940) • AI-Supported Writing Process (α = 0.871) • AI Feedback and Writing Enhancement (α = 0.841) • Affective and Motivational Impact (α = 0.906) The bold values represent the main factor loadings of the items, which are greater than 0.5, while the italicized values indicate acceptable loadings that range between 0.4 and 0.5 The full Rotated Factor Matrix, including all detailed loadings, is presented in Table 6; Fig. 3. Fig. 3Conceptual map of the model derived from factor loading Conceptual map of the model derived from factor loading Most items demonstrated factor loadings above 0.40, indicating satisfactory construct validity. No item exceeded the cross-loading threshold of 0.30, suggesting that items were appropriately aligned with their respective factors. Items 1 and 8 showed moderate loadings across multiple domains but were retained due to their conceptual relevance and acceptable discrimination among factors (Appendix 1). After determining the factors and their respective sub-items, the reliability of the tool was also examined for each domain. The reliability for the domains is as follows: Perceived Effectiveness of AI (0.921), Ethical and Authenticity Concerns (0.940), AI-Supported Writing Process (0.871), AI Feedback and Writing Enhancement (0.841), and Affective and Motivational Impact (0.906). Since our research sample included both Iranian students (whose primary language is Persian) and international students (with diverse cultural and linguistic backgrounds) who participated in this training course, we re-examined the reliability values separately for each of these two groups as well (see Table 7). Table 7Reliability of the instrument by participants’ nationality characteristicsFactorsAll students (N = 252)Iranian students (N = 150)International students (N = 102)• Perceived Effectiveness of AI0.9210.7910.800• Ethical and Authenticity Concerns0.9400.9270.879• AI-Supported Writing Process0.8710.9290.932• AI Feedback and Writing Enhancement0.8410.7210.817• Affective and Motivational Impact0.9060.8600.920Total reliability0.8830.9370.913 Table 7Reliability of the instrument by participants’ nationality characteristicsFactorsAll students (N = 252)Iranian students (N = 150)International students (N = 102)• Perceived Effectiveness of AI0.9210.7910.800• Ethical and Authenticity Concerns0.9400.9270.879• AI-Supported Writing Process0.8710.9290.932• AI Feedback and Writing Enhancement0.8410.7210.817• Affective and Motivational Impact0.9060.8600.920Total reliability0.8830.9370.913 Reliability of the instrument by participants’ nationality characteristics Based on Table 7, and given that the number of items remained constant across both groups, the differences in the reliability of some factors may be attributed to factors such as nationality, cultural and linguistic diversity, prior knowledge of English, and other related variables. Furthermore, Composite Reliability (CR) and Average Variance Extracted (AVE) were computed for each subscale of the AI-AWQ to provide additional evidence of the instrument’s reliability and construct validity. Table 8 presents the values of CR and AVE for the latent constructs. As shown, all CR values exceed the recommended threshold of 0.70, indicating satisfactory internal consistency and reliability across the measurement model. Specifically, Perceived Effectiveness of AI (CR = 0.911, AVE = 0.601) and Ethical and Authenticity Concerns (CR = 0.889, AVE = 0.572) demonstrate strong reliability and convergent validity, as both CR and AVE values are above the acceptable limits (CR > 0.70, AVE > 0.50). Although AI-Supported Writing Process (CR = 0.870, AVE = 0.456), AI Feedback and Writing Enhancement (CR = 0.757, AVE = 0.443), and Affective and Motivational Impact (CR = 0.827, AVE = 0.491) show AVE values slightly below 0.50, their CR values remain above 0.70. This suggests that, despite minor concerns regarding convergent validity, the constructs still exhibit acceptable levels of composite reliability. Overall, the measurement model demonstrates adequate reliability and convergent validity for further structural analysis.Table 8Composite reliability and average variance extractedFactorsCRAVE• Perceived Effectiveness of AI0.9110.601• Ethical and Authenticity Concerns0.8890.572• AI-Supported Writing Process0.8700.456• AI Feedback and Writing Enhancement0.7570.443• Affective and Motivational Impact0.8270.491 Expected Index≥ 0.70≥ 0.50 Table 8Composite reliability and average variance extractedFactorsCRAVE• Perceived Effectiveness of AI0.9110.601• Ethical and Authenticity Concerns0.8890.572• AI-Supported Writing Process0.8700.456• AI Feedback and Writing Enhancement0.7570.443• Affective and Motivational Impact0.8270.491 Expected Index≥ 0.70≥ 0.50 Composite reliability and average variance extracted Finaly Table 9 presents the inter-item correlations for all 30 items of the AI-AWQ, providing an overview of the relationships within and across subscales (Table 9). Table 9Average of inter-item correlationsQ1Q2Q3Q4Q5Q6Q7Q8Q9Q10Q11Q12Q13Q14Q15Q16Q17Q18Q19Q20Q21Q22Q23Q24Q25Q26Q27Q28Q29Q30Q11Q20.1291Q30.4930.5531Q40.3560.6740.6371Q50.7970.0540.5310.3661.Q60.3200.6780.5960.5720.3001Q70.2640.6670.5920.6670.2690.6551Q80.778− 0.0590.4660.2560.8560.2250.1831Q90.7250.1090.5060.3810.7670.3210.3460.7281Q100.403− 0.1170.1510.0790.401− 0.0260.0380.3990.34710.034Q11− 0.3700.4780.0760.229− 0.4230.2080.288− 0.476− 0.4020.0341Q12− 0.2170.4490.1400.248− 0.3010.2290.314− 0.348− 0.2690.1250.73210.013Q130.440− 0.2270.0730.0060.467− 0.029− 0.0560.4800.3510.661− 0.1550.0131Q14− 0.2460.5230.1470.262− 0.3120.2590.344− 0.387− 0.2910.0730.7820.769− 0.0341Q150.546− 0.3030.131− 0.0210.581− 0.070− 0.0930.6070.4840.612− 0.314− 0.1480.723− 0.1921Q16− 0.3090.5290.1560.261− 0.3750.2750.338− 0.437− 0.3220.0320.8090.767− 0.1300.785− 0.2781Q17− 0.2870.4650.1060.276− 0.3460.2280.323− 0.426− 0.3020.0940.7490.715− 0.0680.709− 0.2730.7741Q180.370− 0.2540.038− 0.0780.362− 0.061− 0.0980.4340.3010.331− 0.364− 0.1590.331− 0.2810.423− 0.333− 0.3051Q190.465− 0.1690.1200.0590.467− 0.046− 0.0170.5250.4190.308− 0.447− 0.2500.377− 0.3530.478− 0.406− 0.3590.7181Q200.519− 0.2550.0900.0060.509− 0.007− 0.0350.5900.4300.341− 0.509− 0.3290.498− 0.3750.597− 0.484− 0.4350.7050.7811Q210.2120.1670.1140.1500.1520.1800.1690.1690.1660.129− 0.0430.1310.1070.0230.129− 0.0360.0410.4840.5200.4951Q22− 0.2530.4310.0970.192− 0.3130.1650.290− 0.369− 0.255− 0.2110.4720.562− 0.2470.487− 0.3940.4900.4420.0390.012− 0.1070.4161Q230.486− 0.3690.075− 0.1220.520− 0.113− 0.1360.5980.4500.406− 0.564− 0.3170.491− 0.4560.611− 0.553− 0.5050.7480.8300.8370.455− 0.1141Q240.602− 0.0310.3110.1350.5690.1530.1490.5750.5630.417− 0.284− 0.1160.459− 0.1590.541− 0.264− 0.2050.5050.5460.5560.323− 0.1420.5551Q250.5270.0750.3390.3360.5210.1960.2050.5270.5000.371− 0.203− 0.0700.379− 0.1110.463− 0.231− 0.1620.4330.4400.5070.372− 0.0480.5010.6811Q260.602− 0.1320.2630.1160.6580.0980.0350.6460.5700.437− 0.391− 0.2630.507− 0.2960.626− 0.424− 0.3480.5060.5450.6430.301− 0.2550.6760.7000.7601Q270.566− 0.0010.2930.2020.5680.1980.1190.5860.5230.342− 0.253− 0.1360.437− 0.1360.569− 0.288− 0.2270.4560.5150.5560.338− 0.1300.5610.7250.7410.7731Q280.623− 0.0250.3660.2030.6570.1730.1830.6720.5840.456− 0.318− 0.1670.529− 0.2140.609− 0.338− 0.2540.4980.5660.6480.384− 0.1760.6570.7220.7390.7860.7871Q29− 0.0630.6100.2890.407− 0.1350.3710.414− 0.2140.044− 0.1320.4930.482− 0.2300.512− 0.2620.4760.467− 0.144− 0.173− 0.2160.2500.477− 0.2800.1220.2590.0010.1550.0651Q300.568− 0.0080.2650.1740.5460.1640.1080.5610.5220.382− 0.271− 0.1710.435− 0.1730.546− 0.288− 0.1890.3820.4370.5170.349− 0.1820.5250.6480.6570.6990.7340.7140.1371 Table 9Average of inter-item correlationsQ1Q2Q3Q4Q5Q6Q7Q8Q9Q10Q11Q12Q13Q14Q15Q16Q17Q18Q19Q20Q21Q22Q23Q24Q25Q26Q27Q28Q29Q30Q11Q20.1291Q30.4930.5531Q40.3560.6740.6371Q50.7970.0540.5310.3661.Q60.3200.6780.5960.5720.3001Q70.2640.6670.5920.6670.2690.6551Q80.778− 0.0590.4660.2560.8560.2250.1831Q90.7250.1090.5060.3810.7670.3210.3460.7281Q100.403− 0.1170.1510.0790.401− 0.0260.0380.3990.34710.034Q11− 0.3700.4780.0760.229− 0.4230.2080.288− 0.476− 0.4020.0341Q12− 0.2170.4490.1400.248− 0.3010.2290.314− 0.348− 0.2690.1250.73210.013Q130.440− 0.2270.0730.0060.467− 0.029− 0.0560.4800.3510.661− 0.1550.0131Q14− 0.2460.5230.1470.262− 0.3120.2590.344− 0.387− 0.2910.0730.7820.769− 0.0341Q150.546− 0.3030.131− 0.0210.581− 0.070− 0.0930.6070.4840.612− 0.314− 0.1480.723− 0.1921Q16− 0.3090.5290.1560.261− 0.3750.2750.338− 0.437− 0.3220.0320.8090.767− 0.1300.785− 0.2781Q17− 0.2870.4650.1060.276− 0.3460.2280.323− 0.426− 0.3020.0940.7490.715− 0.0680.709− 0.2730.7741Q180.370− 0.2540.038− 0.0780.362− 0.061− 0.0980.4340.3010.331− 0.364− 0.1590.331− 0.2810.423− 0.333− 0.3051Q190.465− 0.1690.1200.0590.467− 0.046− 0.0170.5250.4190.308− 0.447− 0.2500.377− 0.3530.478− 0.406− 0.3590.7181Q200.519− 0.2550.0900.0060.509− 0.007− 0.0350.5900.4300.341− 0.509− 0.3290.498− 0.3750.597− 0.484− 0.4350.7050.7811Q210.2120.1670.1140.1500.1520.1800.1690.1690.1660.129− 0.0430.1310.1070.0230.129− 0.0360.0410.4840.5200.4951Q22− 0.2530.4310.0970.192− 0.3130.1650.290− 0.369− 0.255− 0.2110.4720.562− 0.2470.487− 0.3940.4900.4420.0390.012− 0.1070.4161Q230.486− 0.3690.075− 0.1220.520− 0.113− 0.1360.5980.4500.406− 0.564− 0.3170.491− 0.4560.611− 0.553− 0.5050.7480.8300.8370.455− 0.1141Q240.602− 0.0310.3110.1350.5690.1530.1490.5750.5630.417− 0.284− 0.1160.459− 0.1590.541− 0.264− 0.2050.5050.5460.5560.323− 0.1420.5551Q250.5270.0750.3390.3360.5210.1960.2050.5270.5000.371− 0.203− 0.0700.379− 0.1110.463− 0.231− 0.1620.4330.4400.5070.372− 0.0480.5010.6811Q260.602− 0.1320.2630.1160.6580.0980.0350.6460.5700.437− 0.391− 0.2630.507− 0.2960.626− 0.424− 0.3480.5060.5450.6430.301− 0.2550.6760.7000.7601Q270.566− 0.0010.2930.2020.5680.1980.1190.5860.5230.342− 0.253− 0.1360.437− 0.1360.569− 0.288− 0.2270.4560.5150.5560.338− 0.1300.5610.7250.7410.7731Q280.623− 0.0250.3660.2030.6570.1730.1830.6720.5840.456− 0.318− 0.1670.529− 0.2140.609− 0.338− 0.2540.4980.5660.6480.384− 0.1760.6570.7220.7390.7860.7871Q29− 0.0630.6100.2890.407− 0.1350.3710.414− 0.2140.044− 0.1320.4930.482− 0.2300.512− 0.2620.4760.467− 0.144− 0.173− 0.2160.2500.477− 0.2800.1220.2590.0010.1550.0651Q300.568− 0.0080.2650.1740.5460.1640.1080.5610.5220.382− 0.271− 0.1710.435− 0.1730.546− 0.288− 0.1890.3820.4370.5170.349− 0.1820.5250.6480.6570.6990.7340.7140.1371 Average of inter-item correlations Inter-Item correlation table Based on Table 9, The overall pattern of inter-item correlations revealed that most items within the same subscale exhibited moderate to high positive correlations (approximately 0.3 to 0.8), indicating good internal consistency. Some items showed negative correlations with items from other subscales, which is expected given the conceptual differences between constructs. Within-subscale consistency was particularly evident: items belonging to the same factor (e.g., Perceived Effectiveness of AI or AI-Supported Writing Process) generally correlated more strongly with each other than with items from other subscales. This pattern supports convergent validity and aligns well with the factor structure extracted through the EFA. Moderate correlations (0.3–0.5) were observed across some subscales, notably for items 14, 18, 21, and 22. This aligns with previous EFA results, where these items displayed moderate cross-loadings, reflecting partial conceptual overlap between related constructs, such as motivation and feedback. Overall, the inter-item correlation pattern corroborates the Cronbach’s alpha and Composite Reliability (CR) findings, confirming that all five factors demonstrate strong internal consistency. High correlations within subscales (≥ 0.6–0.8) justify retaining items even when AVE values for certain subscales are slightly below 0.50, as these items meaningfully contribute to their respective constructs. Finally, item 29 showed generally weak correlations with other items (ranging from ~ 0.001 to 0.61), suggesting it may behave differently across subscales and could be a candidate for refinement in future research. Discussion Psychometric characteristics of AI-AWQ Based on the findings of the exploratory factor analysis, five factors were identified: Perceived Effectiveness of AI, Ethical and Authenticity Concerns, AI-Supported Writing Process, AI Feedback and Writing Enhancement, and Affective and Motivational Impact. Together, these factors accounted for approximately 78% of the total variance, indicating a satisfactory model fit for the construct of AI-assisted academic writing. The subsequent sections examine the psychometric properties of the instrument and its underlying constructs. Essentially, a value above 0.50 for total variance is considered acceptable; therefore, the obtained value can be evaluated as satisfactory. Additionally, the Kaiser-Meyer-Olkin (KMO) measure assesses the sampling adequacy for factor analysis, indicating whether the correlations among variables are sufficiently strong for reliable factor extraction. In this study, the KMO value was 0.930, which falls within the “excellent” range (0.90–1.00) and confirms that the sample is highly adequate [72]. Furthermore, Bartlett’s Test of Sphericity was significant, indicating that the correlation matrix differs significantly from an identity matrix and that factor analysis is appropriate. Together, these results demonstrate that the collected sample is sufficient and suitable for conducting factor analysis. Also, communalities of the AI-AWQ items showed most items have extraction values above 0.50 [75], indicating that a substantial portion of each item’s variance is explained by the extracted factors. The values range from 0.581 (Q6) to 0.907 (Q23), demonstrating that all items are well-represented by the factor solution. These results confirm that the factor model adequately captures the underlying constructs of the questionnaire. Factor loadings indicate the extent to which each item correlates with the extracted factor, reflecting how well the item represents the underlying construct. Loadings range from − 1 to + 1, with higher positive values indicating stronger relationships. According to Hair et al. (2010), and Comrey et al. (1992), loadings ≥ 0.30 are considered minimally acceptable, ≥ 0.50 are good, and ≥ 0.70 are excellent [75, 76]. In the current study, most items had loadings between ~ 0.500 and 0.860, demonstrating that they are well represented by their respective factors. Only one item (Q08 = 0.476) was slightly below 0.50, but still close to the good threshold, indicating it reasonably contributes to its factor. Overall, these results suggest that the items have strong construct validity and effectively measure the intended factors. The content validity of the questionnaire was assessed using both the CVR and the CVI. The overall CVR value (0.903) exceeded the minimum threshold recommended by Lawshe (1975), confirming that all items were essential and relevant according to expert judgment. Given that the number of experts was 10, the minimum acceptable CVR value is 0.625 [77]. In addition, the CVI was evaluated based on the recommendations of Waltz and Bausell (1981), with a minimum acceptable value of 0.79. The mean CVI in the present study (0.882) also surpassed this threshold, indicating satisfactory clarity, simplicity, and relevance across all items [78]. The overall reliability coefficient (Cronbach’s alpha = 0.883) confirmed strong internal consistency among the items. A Cronbach’s alpha value closer to 1 indicates higher internal consistency and stronger reliability of the instrument [79]. Examination of the “if item deleted” column showed that removing any individual item would not meaningfully increase the alpha value, suggesting that all items contributed positively to the reliability of the instrument. Notably, in comparison between Iranian and international students, Iranian students showed slightly higher overall reliability (α = 0.937) compared to international students (α = 0.913), though both values indicate strong internal consistency. Examining individual factors, reliability was generally high across groups, particularly for AI-Supported Writing Process and Ethical and Authenticity Concerns. However, some differences emerged: Iranian students exhibited higher consistency in Affective and Motivational Impact, while international students showed greater reliability in Perceived Effectiveness of AI, Ethical and Authenticity Concerns, and AI Feedback and Writing Enhancement. These variations may stem from cultural and linguistic diversity, differing prior knowledge and experiences with AI, and distinct interpretations of ethical issues. Additionally, the larger and more linguistically homogeneous Iranian group (N = 150) compared to the international group (N = 102) may contribute to these patterns, reflecting both higher consistency in some areas and greater variability in others due to diverse educational backgrounds. Of course reliability can be influenced by sample size and variability [80]. Research by Ng, Wu [81] suggests that such reliability issues could stem from the evolving nature of AI technologies and the diversity of individual experiences with these tools. Additionally, our results are consistent with studies by Aysolmaz, Müller [82], which emphasize that public perceptions of AI are shaped by a combination of concern, potential benefits, and ethical considerations. The emergence of AI-powered writing tools has been well-documented, with research such as Khalifa, Albadawy [83] demonstrating that AI can aid in improving the writing process. However, as pointed out in Al-Sofi [58], the adoption of AI-powered writing tools like ChatGPT raises significant ethical concerns regarding authorship and academic integrity. To ensure the validity and reliability of measurement instruments, it is recommended to compute not only Cronbach’s Alpha (CA) but also Composite Reliability (CR) and Average Variance Extracted (AVE). While CA assesses the internal consistency of the items, CR provides a more stringent evaluation of construct reliability, especially in reflective measurement models [79, 84, 85]. AVE, on the other hand, measures the amount of variance captured by a construct in relation to the variance due to measurement error. Together, these indices offer a comprehensive assessment of both convergent validity and internal consistency, thereby enhancing the confidence in the instrument’s ability to accurately measure the intended constructs in environmental health assessments. In psychometric studies, to evaluate convergent validity and internal consistency of constructs, it is recommended that the Composite Reliability (CR) should be greater than 0.70, and the Average Variance Extracted (AVE) should exceed 0.50. These values indicate satisfactory internal consistency and construct validity of measurement instruments [86]. In this research, all five factors demonstrated acceptable to excellent CR, ranging from 0.757 to 0.911, all exceeding the recommended threshold of 0.70 [79, 84]. This finding is consistent with the results of studies such as Ng et al. [81], who highlighted the reliability of scales used to measure attitudes toward AI in educational settings. The results of the psychometric analysis, presented in this research, show that the CR values for all five factors exceed the recommended threshold of 0.70, indicating satisfactory internal consistency across the constructs. Specifically, Perceived Effectiveness of AI (CR = 0.911) and Ethical and Authenticity Concerns (CR = 0.889) demonstrate very strong reliability, suggesting that the items within these constructs consistently measure the underlying latent variables. Regarding AVE, two factors, Perceived Effectiveness of AI (0.601) and Ethical and Authenticity Concerns (0.572), exceed the recommended cutoff of 0.50, confirming good convergent validity. The remaining three factors—AI-Supported Writing Process (0.456), AI Feedback and Writing Enhancement (0.443), and Affective and Motivational Impact (0.491)—fall slightly below the 0.50 threshold. While their CR values indicate acceptable internal consistency, the lower AVE values suggest that these constructs explain slightly less than 50% of the variance in their respective items, highlighting areas where future refinement of the measurement items may improve convergent validity. Overall, the instrument demonstrates adequate reliability and generally acceptable convergent validity, with the strongest psychometric properties observed in Perceived Effectiveness of AI and Ethical and Authenticity Concerns. These results provide confidence in the suitability of the instrument for subsequent structural analyses and support its use in evaluating perceptions of AI-supported academic writing. The slightly lower AVE values observed for AI-Supported Writing Process, AI Feedback and Writing Enhancement, and Affective and Motivational Impact can be attributed to the presence of some items with relatively low factor loadings within these constructs. For instance, in the AI-Supported Writing Process factor, three items have loadings below 0.55, while AI Feedback and Writing Enhancement contains an item with a loading of 0.476. Since AVE is computed as the average of squared factor loadings, the lower loadings substantially reduce the AVE even if the CR remains above 0.7. This explains why AVE for these constructs falls slightly below the recommended 0.50 threshold, highlighting potential areas for item refinement to enhance convergent validity. Importantly, as an essential next step for construct validation, confirmatory factor analysis (CFA) using fit indices such as χ2/df, CFI, and RMSEA should be conducted to fully confirm the construct of this instrument before its widespread adoption. Construct domain The results of the exploratory factor analysis revealed a five-factor structure that collectively explained 78% of the total variance, indicating an adequate representation of the underlying dimensions of AI-assisted writing perceptions. The first factor, Perceived Effectiveness of AI, had the highest eigenvalue (11.363) and accounted for 37.88% of the total variance. This suggests that participants primarily associated AI tools with their effectiveness in enhancing writing performance and overall productivity. The second factor, Ethical and Authenticity Concerns, explained an additional 20.20% of the variance, reflecting respondents’ awareness of ethical implications and authenticity issues related to AI-generated content. The third factor, AI-Supported Writing Process, contributed 8.84% of the variance, emphasizing the role of AI in assisting with various stages of the writing process. The fourth factor, AI Feedback and Writing Enhancement, accounted for 6.84% of the variance, indicating the significance of AI-generated feedback in improving writing quality. Finally, the fifth factor, Affective and Motivational Impact, explained 4.25% of the variance, highlighting the emotional and motivational influences of AI tools on users’ engagement and confidence in writing. Overall, these five factors demonstrate that users’ perceptions of AI in writing encompass not only functional effectiveness but also ethical, cognitive, and affective dimensions. These findings align with existing research, which highlights the multi-faceted nature of public perceptions of AI and its impact on various sectors [81, 87]. Perceived effectiveness of AI The first factor reflecting Perceived Effectiveness of AI, with high factor loadings ranging from 0.564 to 0.846, demonstrates students’ perceptions of AI’s practical utility and effectiveness in supporting their academic writing tasks. These findings are consistent with previous literature, which indicates that AI’s increasing integration into industries such as healthcare and education raises both opportunities and concerns [88]. In education, AI tools demonstrate significant potential to enhance academic writing effectiveness through various support mechanisms, including grammar improvement, structural organization, and adherence to disciplinary standards [52, 83]. From a theoretical standpoint, CLT and TAM provide frameworks for understanding this factor. According to TAM, the perceived effectiveness of AI tools can lead students to adopt these technologies based on their utility and ease of use in improving writing outcomes. Additionally, CLT suggests that AI tools can reduce extraneous cognitive load by handling routine writing tasks, allowing students to focus on higher-order thinking processes such as argumentation, critical analysis, and content development [52, 53]. This factor, reflects students’ recognition of AI’s practical benefits in their academic work. Our findings align with recent studies that suggest familiarity with AI technologies significantly influences perceptions of their effectiveness. For example, Kumar, Haque [89] highlighted that AI tools, when integrated into education, enhance the efficiency of teaching and learning, though their effectiveness depends on the user’s familiarity with these technologies. Similarly, Mon, Wasfi [90] found that AI tools improve personalized learning by adapting to students’ individual needs, which significantly increases their engagement. While the Perceived Effectiveness of AI factor demonstrates students’ recognition of AI’s practical benefits, these findings also highlight a tension between technological progress and its societal implications, as over-reliance on AI tools may lead students to become less autonomous, undermining their ability to engage critically with the writing process [91]. Similar concerns are raised in the literature, where the impact of AI tools on student autonomy and critical thinking has been widely debated. For instance, Henno [88] and others emphasize that while AI can enhance the efficiency of educational processes, its overuse may disrupt students’ ability to think independently, leading to a decline in critical thinking skills. Similarly, Abgaryan, Asatryan [24] discuss how AI’s role in personalized education might foster dependency, ultimately limiting students’ capacity to engage meaningfully with the learning material. Strasser [92] also highlights the importance of balancing AI’s assistive capabilities with the development of creative engineering skills to ensure that students do not become overly reliant on technology. These findings, alongside Newton and Newton [93] warning about the potential stifling of creative thinking in a highly digitized world, underscore the broader implications of AI in education: while technology offers unprecedented support, as reflected in students’ high perceived effectiveness ratings, it also necessitates careful integration to avoid undermining essential cognitive and creative skills. Ethical and authenticity concerns The Ethical and Authenticity Concerns factor highlights growing concerns about the ethical implications of AI, particularly issues related to academic integrity, authenticity, privacy, algorithmic bias, and accountability. This factor, reflects students’ strong awareness of ethical challenges associated with AI use in academic writing. Our findings support research that underscores the importance of ethical governance in AI deployment, such as the work by Rosenstrauch, Mangla [94], which emphasizes the need for transparency and fairness in AI systems to mitigate biases and ensure equitable outcomes. Studies also show that AI technologies must be developed with accountability in mind, as discussed by Chaudhry [95], who argues for more robust frameworks that address bias and discrimination within AI systems. From a CoI perspective, ethical concerns are crucial in maintaining teaching presence. AI in education must be deployed with careful attention to the ethical frameworks that govern student-teacher interactions and academic integrity [96]. Chaudhry [95] further argues that transparency in AI is not only a matter of ethical responsibility but also an enabler of accountability, interpretability, and safety in educational contexts. Without these frameworks, AI’s potential to transform education could be undermined by issues related to trust and authenticity in student work. AI-supported writing process This factor reflects the growing role of AI in transforming educational practices, particularly in supporting students’ writing activities and fostering personalized learning environments. Studies have emphasized both the potential and challenges of integrating AI into learning, particularly through tools like ChatGPT. AI can assist in academic writing by aiding with tasks such as brainstorming, drafting, and proofreading, which can significantly enhance writing efficiency [58]. This factor, with factor loadings ranging from 0.499 to 0.778 and reliability of α = 0.871, captures students’ perceptions of how AI tools support various stages of the writing process. However, concerns about the dehumanization of education and data privacy risks persist [97]. The widespread use of AI raises ethical questions about authorship and privacy, particularly when it involves the collection of personal data. For example, research by Irfan, Aldulaylan [98] found that students expressed heightened concern about data privacy when using AI tools, especially when their academic work could be stored or accessed by external entities. Similarly, Kumar, Haque [89] discuss how the widespread use of AI in education raises ethical challenges, particularly in maintaining transparency and accountability regarding AI’s impact on student learning. The CoI framework offers an insightful lens to analyze this issue. While AI tools may enhance cognitive presence by improving writing efficiency and supporting the writing process, their overuse may reduce social presence as students engage less in peer collaboration and instructor feedback [96]. This shift challenges traditional models of collaborative learning, which are foundational to higher-order cognitive engagement and critical thinking in education. A study by Thong, Butson [99] found that students using ChatGPT extensively reported a decline in their participation in collaborative learning activities, highlighting the potential drawbacks of AI’s role in education. While AI can support the writing process and personalized learning, it may inadvertently diminish the sense of community and interpersonal engagement that are crucial for fostering critical thinking and academic growth. AI feedback and writing enhancement The Feedback and Writing Enhancement factor reflects the growing interaction between students and AI technologies, particularly through tools like ChatGPT, which are used for providing feedback and enhancing academic writing. This factor, captures students’ experiences with AI-generated feedback and its role in improving their writing. These tools have shown significant potential to enhance writing productivity, as highlighted by Bahammam, Trabelsi [100], who discuss the efficiency gains in scientific writing through AI-assisted drafting and peer review. However, concerns have been raised that excessive reliance on AI may hinder the development of critical writing skills, as students may become overly dependent on these tools. This concern is supported by Rosenstrauch, Mangla [94], who argue that AI feedback, while enhancing efficiency, may reduce students’ need for deep critical engagement with their work. This concern aligns with CLT, which posits that while AI tools reduce extraneous cognitive load, they can inadvertently promote cognitive dependency [27]. Moreover, TAM suggests that students might perceive AI feedback as useful and easy to use, leading them to adopt these tools. However, this could reduce their engagement with manual writing and revision processes, diminishing the learning experience that comes from critical reflection and self-editing. This concern about over-dependence is particularly relevant in light of the ethical implications of using AI to generate or modify academic content without developing the necessary critical engagement with the material, as discussed by McGrath, Cooper [7]. They highlight the risk of eroding academic integrity and authentic learning when students rely too heavily on AI-generated content. Affective and motivational impact This factor reflects the emotional and motivational responses of students toward the use of AI tools in academic writing. The items within this construct capture how AI integration influences students’ writing confidence, creativity, anxiety levels, and perceptions of originality. Specifically, items such as “AI tools have saved me time” and “I feel more creative when generating ideas” emphasize the motivational enhancement and increased efficiency experienced by students when using AI in their writing process. The statement “My confidence in my writing abilities has increased due to using AI tools” further indicates that exposure to AI-assisted writing can foster self-efficacy and reduce apprehension toward academic writing tasks. At the same time, the inclusion of the item “I am concerned that using AI tools may compromise the originality of my academic work” introduces a critical emotional dimension — it acknowledges the ambivalence some students feel, balancing between the motivational benefits of AI and ethical or authenticity concerns. This highlights that, while AI tools generally promote positive affect and motivation, a subset of students remains cautious about potential threats to originality and academic integrity. Overall, the Affective and Motivational Impact factor suggests that AI tools can play a dual role: they empower students emotionally and creatively, while also eliciting ethical reflection about responsible use. This combination indicates that emotional engagement is a key mediator in how students adopt and perceive AI in academic contexts. Furthermore, studies utilizing network analysis have elucidated the complex interconnections between affective states and motivational traits, highlighting their collective influence on error monitoring and adaptive functioning [101]. In fact, the final factor (Affective and Motivational Impact) appears to reflect not only students’ positive beliefs regarding the usability and benefits of AI tools but also their underlying concerns about relying on such technology. For instance, while many students reported feeling more confident, creative, and less anxious when using AI tools, some simultaneously expressed apprehension that AI might compromise the originality or authenticity of their academic work. A critical analysis of AI-generated versus human-authored texts reveals that while AI can produce grammatically correct content, it often lacks the depth of analysis and creativity found in human writing [102]. This concern is echoed by Atkinson and Barker [103], who argue that while AI can assist in creative processes, it lacks the nuanced understanding and originality of human creators. CLT helps explain this concern, suggesting that while AI tools can reduce extraneous cognitive load (helping with structure and syntax), they may not engage students in the critical thinking required for high-quality writing. In addition, TAM posits that students’ adoption of AI tools often stems from perceived usefulness, as evidenced by the high reliability of the Perceived Effectiveness factor, but the lack of AI-generated creativity may hinder their long-term academic engagement [15]. This concern is further explored by Granda, Inzhivotkina [104], which emphasizes the need for AI tools to foster engagement beyond mere efficiency, encouraging deeper cognitive involvement. Jaakkola, Henno [105] also point out that while AI has revolutionized educational practices, it is essential to balance its use to ensure that it enhances—not replaces—critical thinking and creativity. Limitations and future research Given the limited exposure to artificial intelligence (AI) technologies, future research should include more diverse demographic groups to gain a deeper understanding of how different populations perceive AI. Although the subscales of the questionnaire demonstrated strong and acceptable reliability, since this instrument was used for the first time, it is essential that future studies conduct revalidation to ensure greater confidence in its application. Moreover, the cultural context of this study—conducted at a medical sciences university in Iran—may have influenced participants’ perceptions of AI tools, particularly regarding trust in technology, educational norms, and ethical considerations. Another point to note is that our sample consisted solely of medical and dental students. We acknowledge that the AI-AWQ demonstrated initial validity within this specific cultural and disciplinary context. Therefore, caution should be exercised when generalizing its applicability to broader student populations. Future research should examine the validity of the AI-AWQ across diverse cultural contexts to assess its generalizability and reduce potential cultural bias. Since this is a newly developed instrument, only EFA was employed in the present study. Future studies should consider using Maximum Likelihood EFA or Confirmatory Factor Analysis (CFA) to achieve stronger construct validation. Furthermore, due to limited methodological expertise, parallel analysis and the Minimum Average Partial (MAP) test were not performed in this study. We acknowledge this as a limitation and recommend that future studies incorporate these analyses to provide stronger evidence for factor retention. Additionally, for future studies conducting CFA, it would be valuable to briefly address discriminant validity using methods such as Fornell-Larcker or HTMT. A few subscales (AI-Supported Writing Process and AI Feedback and Writing Enhancement) showed Average Variance Extracted (AVE) values slightly below the recommended threshold of 0.50. While this suggests that the proportion of variance captured by the latent factor relative to measurement error is modest, these items were retained due to their theoretical relevance and essential role in representing the multidimensional nature of AI-assisted academic writing. This limitation highlights the need for future research to refine these subscales, potentially by reviewing item wording, adding additional items to better capture the construct, or conducting Confirmatory Factor Analysis (CFA) in larger and more diverse samples. Importantly, the Composite Reliability (CR) for all factors remained above the acceptable threshold (> 0.70), confirming strong internal consistency despite the slightly lower AVE. Finally, differences in internal consistency (Cronbach’s alpha) were observed between the Iranian and international subsamples for certain subscales. These variations may reflect the influence of cultural or contextual factors on participants’ responses. It is therefore recommended that future studies employ multi-group Confirmatory Factor Analysis (CFA) or measurement invariance testing to examine the stability and equivalence of the AI-AWQ across diverse populations. Conclusion In conclusion, the AI-AWQ provides preliminary evidence of internal consistency and may serve as an initial tool for assessing attitudes toward AI, particularly among Iranian medical and dental students. Our findings reveal the complex and varied public perceptions of AI, ranging from concerns about its impact on autonomy to acknowledgment of its benefits in areas such as education and writing. However, the subscales of the AI-AWQ require further refinement before they can be used independently, and future versions should consider adding targeted items to improve homogeneity. As AI technology continues to advance, so too will the attitudes and perceptions have captured by instruments like the AI-AWQ. Given that our sample consists exclusively of Iranian medical and dentistry students, the generalizability of these findings is limited, meaning that the AI-AWQ currently demonstrates initial validity only within this specific cultural and disciplinary context rather than across broader populations. Future research should focus on refining this tool and exploring additional dimensions of AI’s role in academic writing, as well as its broader societal implications. Supplementary Information Supplementary Material 1. Supplementary Material 1.