The effectiveness of ChatGPT in teaching medicine through systematic mapping

Abstract

Introduction: With the increasing integration of artificial intelligence (AI) technologies into various educational domains, assessing their effectiveness becomes paramount. This study investigates the efficacy of ChatGPT, a state-of-the-art conversational AI model, in the field of medical education. Objective: To analyze the scientific production on medical education assisted by ChatGPT. Methodology: A systematic and descriptive mapping, with a qualitative nature, was employed. The research question was formulated following the PICO strategy (Population, Intervention, Comparison, Outcome): In medical students (P), how does the integration of ChatGPT-assisted teaching (I), as a complement to conventional teaching methods (C), positively influence academic performance improvement and learning satisfaction (O)? The screening of studies used the PRISMA tool (Preferred Reporting Items for Systematic and Meta-Analyses). For article selection, descriptors were defined: ChatGPT, Teaching, Medicine, Learning, Artificial Intelligence, Evaluation Studies as Topic, Educational Measurement, and Education, Medical, Undergraduate. Searches were conducted using long and short search strings: Long string: ChatGPT AND Medicine AND (Teaching OR Learning OR Assessment OR Evaluation Studies as Topic OR Educational Measurement OR Education Medical Undergraduate) end Short string: ChatGPT AND medicine AND (teaching OR learning), These searches were performed in virtual libraries (PubMed, CAPES Journals, Wiley Online Library, BVS, SciELO, and ScienceDirect) by combining descriptors in Portuguese and English, using the Boolean operator (AND). The search strings included: Portuguese: ChatGPT AND medicina AND (ensino OR aprendizagem OR avaliação) end English: ChatGPT AND medicine AND (teaching OR learning OR assessment). Categories were created using Bardin's content analysis method applied to ChatGPT. Results and Discussion: Fifty articles were selected. Using Bardin's content analysis, six thematic categories and one subcategory emerged: Applications of ChatGPT in Medical Education: ChatGPT is a complementary tool that enhances learning and critical skill development by offering personalized and interactive experiences. Its integration into medical education requires supervision to ensure efficacy; Ethical Considerations in ChatGPT Use in Medical Education: The use of ChatGPT demands human oversight and critical evaluation, without replacing medical expertise. Ethical training for future doctors is essential to ensure safe and effective application; Subcategory: Reliability of ChatGPT in Medical Education: ChatGPT's reliability requires constant supervision, as it may generate inaccurate information. Its application in education should be guided by qualified educators to ensure content accuracy; Challenges of ChatGPT in Medical Education: ChatGPT faces challenges such as inaccuracy, inconsistency, and lack of interpersonal skills, limiting its efficacy as a substitute for human knowledge. Close monitoring is essential; Contributions of ChatGPT Use in Medical Education: ChatGPT enhances medical education by personalizing learning, providing immediate feedback, and enriching the understanding of complex concepts, making it a valuable educational tool; ChatGPT’s Performance in Medical School Exams: ChatGPT performs well in medical exams, with approval rates comparable to human students, but its results require rigorous supervision and evaluation due to limitations; Future Directions and Recommendations for ChatGPT Integration in Medical Education: Effective and ethical integration of ChatGPT requires clear guidelines, AI literacy, and ongoing critical evaluation, ensuring it complements human clinical judgment. Conclusion: The analysis of scientific production on ChatGPT in medical education highlights its potential as a complementary tool capable of enhancing learning and critical skill development among students. While ChatGPT has shown promising results in medical exams and offers significant contributions, such as personalized learning and immediate feedback, its use demands constant supervision and critical evaluation. Challenges related to inaccuracy and the absence of interpersonal skills underscore the importance of careful and ethical integration. For effective implementation, the development of clear guidelines, AI literacy promotion, and continuous monitoring are recommended. When used ethically and under supervision, ChatGPT can enrich the educational experience, complement conventional methods, and contribute to the training of better-prepared and more critical medical professionals.