La lettre de motivation à l'ère de l'intelligence artificielle (ChatGPT comme exemple) Authors: Haifa Hachfi, Melek Kechida, Rania Kaddoussi, Hedhemi Rejeb, Wafa Alaya, Saida Hidouri, Helmi Ben Saad Published: 2024 ABSTRACT RESUME During the article submission process, a cover letter (CL) is often required and serves to promote the manuscript (1-3). The CL provides an opportunity to highlight the study's significance to the journal editors (1-3 ). In most cases, the CL is the first document read by editors and serves as the primary basis for deciding whether to reject the manuscript or send it for external peer review (1-3). Therefore, it is crucial to dedicate significant attention to writing the CL (1-3). The latter should include essential details such as the manuscript's title, the name of the target journal, a brief description of the study, its importance, and its potential interest to the readership (1-3). Additionally, it should address the manuscript's originality, disclose any conflicts of interest, and provide the author's contact information ( 1-3). Two previous papers published in Tunis Med (In French ( 1) and English (2) languages) have provided detailed guidelines on how to write effective CLs. Writing medical scientific papers, including CLs, is challenging, time-consuming, and prone to errors (4). Advanced artificial intelligence (AI) models, such as Chat Generative Pre-trained Transformer (ChatGPT), can simplify academic writing and publishing(4-8). ChatGPT has numerous applications in academic and scientific writing, including hypothesis generation, literature review, safety recommendations, troubleshooting, paraphrasing and summarizing, editing, proofreading, journal selection, and journal style formatting(4-9), as well as peer reviewing (10 ). AI tools like ChatGPT are already used to generate professional CLs for job applications (11), providing advantages such as timesaving and ensuring proper structure and format. However, as of September 22, 2024, to the best of the authors’ knowledge, only one study (3) has explored the usefulness of AI in writing CLs for scientific articles. This study specifically aimed to determine whether ChatGPT-4 (paid version) is as effective as humans in writing CLs for scientific papers (3). Given that a well-crafted, informative, and effective CL can help editors understand the research and its significance, thus giving the manuscript the best chance of publication, the aim of this editorial was to investigate the accuracy of ChatGPT 3.5 (free version) in writing CLs for scientific papers. Specifically, we examined the ability of ChatGPT 3.5 to include necessary information, adhere to professional standards, and clearly and concisely convey the researcher’s message. By understanding the strengths and limitations of ChatGPT 3.5 for this purpose, researchers can better decide how to integrate AI into their submission process. When requesting ChatGPT 3.5 to write a CL for a medical journal submission, it is essential to provide detailed and specific information to ensure the generated CL meets professional standards and accurately reflects the research. Box 1 outlines the necessary information for requesting a CL from ChatGPT for medical journal submissions. Once this essential information is gathered, authors can provide it to ChatGPT 3.5 with the following prompt: “" Draft a professional cover letter that incorporates the information included in the abstract and the table below, and aligns with the typical structure and tone expected by medical journals." Box 1. Essential information for requesting a cover letter (CL) from chatbots for medical journal submissions. In our prompt performed on September 16, 2024, we provided ChatGPT 3.5 with all the required information concerning a manuscript accepted by a high-ranking journal (Biology of Sport) (12). This pretested prompt was based on key points for CLs (Box 1) (1-3), along with the abstract of the original source (12). Since ChatGPT 3.5 was given the abstract of an accepted paper, the risk of plagiarism was low, as it used the source to generate its content ( 1). Appendix 1 (13) contains the real CL previously submitted to Biology of Sport, as part of the accepted manuscript (12). Seven authors of the manuscript (12), three of whom are co-authors of this editorial (RK, HR, and HBS in the authors’ list), wrote this CL. Appendix 2(14) provides a practical example of the prompt sent to ChatGPT 3.5 [Figure 1S inside the Appendix 2, represents the screenshot of the prompt sent to ChatGPT 3.5 on September 16, 2024]. The CL generated by ChatGPT 3.5 was transferred to a Word document with no content adjustments [Appendix 3 (15), and Figure 2S inside the Appendix 3 illustrates the screenshot of the response of ChatGPT 3.5]. Four experienced authors (i.e.; contributors to a practical guide on writing effective and succinct CLs (2), possessing an adequate understanding of scientific medical writing and not involved in the CL submitted to Biology of Sport (12)) were chosen as assessors. They were asked to evaluate both the CL provided by ChatGPT 3.5 (Figure 2 and Appendix 3 (15)) and the one written by humans (Appendix 1 (13)). The four-blinded assessors scored both CLs based on 16 essential elements of an effective CL (listed in Box 2); as described in a previous paper (2). A 1–10 Likert scale was used for each element, where one represented the lowest score and ten the highest (1). Each CL received a score ranging from 16 to 160 from each assessor. The mean ± standard deviation (SD) and range of each score were calculated, along with the total score for each CL. Scores was compared using the Student's t-test. Additionally, the assessors were asked which CL they preferred and to identify whether each CL was written by ChatGPT 3.5 or humans (3 ). Box 2 presents the scores assigned to both CLs. The human-written CL had significantly higher scores on seven elements and comparable scores on nine elements. Its total score was higher by approximately 60.5 points. All four assessors correctly identified whether the CLs were written by ChatGPT 3.5 or humans. When asked which CL they preferred, all four assessors chose the humanwritten CL. ChatGPT 3.5, when provided with all the necessary information for writing a CL, produced a CL that was less structured than the human-written version, and did not meet the "" standards" described in the literature (1 , 2). The ChatGPT 3.5-generated CL lacked precision, was not fully informative, and appeared “" inappropriately” tailored for submission to a medical journal. Our findings contradict those of Deveci et al. (3), who conducted a randomized non-inferiority study comparing 18 CLs written by humans to 18 generated by ChatGPT-4. Three blinded assessors evaluated all CLs based on impression, readability, criteria satisfaction, and level of detail (3). Deveci et al. (3) reported that ChatGPT-4 scored higher on objective readability tests and outperformed human writing in subjective readability assessments. The authors concluded that ChatGPT-4 was non-inferior to humans in writing CLs, suggesting it could streamline the submission process and offer equal opportunities for peer review (3). Given the limitations observed in our study, caution is advised when using free AI tools for writing CLs. Authors should be wary and take certain precautionary measures. First, they should carefully evaluate the accuracy of the data provided to AI chatbots. Second, authors must rely on their expertise and judgment to validate the AIgenerated content and consider alternative approaches. Third, authors should track the time spent using AI tools, as research outcomes could be time-sensitive. In conclusion, while AI continues to evolve and improve based on human feedback, scientists should remain cautious when using free chatbots to generate CLs. In the near future, as free chatbots like ChatGPT receive more comprehensive information on the elements of a CL, they may be able to generate CLs that are precise, informative, and appropriately tailored for submission to medical journals. Future studies should evaluate more advanced versions of these tools, such as ChatGPT-4o. Box 2. The blinded assessors’ (n=4) evaluations of the cover letters (CLs) writen by humans and Chat Generative Pre-trained Transformer (ChatGPT 3.5). References