This paper is in the following e-collection/theme issue:

Theme Issue: ChatGPT and Generative Language Models in Medical Education (102) Chatbots and Conversational Agents (632) e-Learning and Digital Medical Education (793) Artificial Intelligence (1738) Generative Language Models Including ChatGPT (497)

Published on in Vol 9 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/48002, first published .
Performance of GPT-3.5 and GPT-4 on the Japanese Medical Licensing Examination: Comparison Study

Performance of GPT-3.5 and GPT-4 on the Japanese Medical Licensing Examination: Comparison Study

Performance of GPT-3.5 and GPT-4 on the Japanese Medical Licensing Examination: Comparison Study

Authors of this article:

Soshi Takagi1 Author Orcid Image ;   Takashi Watari1, 2, 3, 4 Author Orcid Image ;   Ayano Erabi1 Author Orcid Image ;   Kota Sakaguchi2 Author Orcid Image
Article Authors Cited by (158) Tweetations (28) Metrics

Journals

  1. Sallam M, Salim N, Barakat M, Al-Mahzoum K, Al-Tammemi A, Malaeb D, Hallit R, Hallit S. Assessing Health Students' Attitudes and Usage of ChatGPT in Jordan: Validation Study. JMIR Medical Education 2023;9:e48254 View
  2. Levkovich I, Elyoseph Z. Suicide Risk Assessments Through the Eyes of ChatGPT-3.5 Versus ChatGPT-4: Vignette Study. JMIR Mental Health 2023;10:e51232 View
  3. Gobira M, Nakayama L, Moreira R, Andrade E, Regatieri C, Belfort Jr. R. Performance of ChatGPT-4 in answering questions from the Brazilian National Examination for Medical Degree Revalidation. Revista da Associação Médica Brasileira 2023;69(10) View
  4. Kaneda Y, Takahashi R, Kaneda U, Akashima S, Okita H, Misaki S, Yamashiro A, Ozaki A, Tanimoto T. Assessing the Performance of GPT-3.5 and GPT-4 on the 2023 Japanese Nursing Examination. Cureus 2023 View
  5. Wang Y, Chen T. ChatGPT surges ahead: GPT-4 has arrived in the arena of medical research. Journal of the Chinese Medical Association 2023;86(9):784 View
  6. Toyama Y, Harigai A, Abe M, Nagano M, Kawabata M, Seki Y, Takase K. Performance evaluation of ChatGPT, GPT-4, and Bard on the official board examination of the Japan Radiology Society. Japanese Journal of Radiology 2024;42(2):201 View
  7. Taloni A, Borselli M, Scarsi V, Rossi C, Coco G, Scorcia V, Giannaccare G. Comparative performance of humans versus GPT-4.0 and GPT-3.5 in the self-assessment program of American Academy of Ophthalmology. Scientific Reports 2023;13(1) View
  8. Kaneda Y, Namba M, Kaneda U, Tanimoto T. Artificial Intelligence in Childcare: Assessing the Performance and Acceptance of ChatGPT Responses. Cureus 2023 View
  9. Huang R, Lu K, Meaney C, Kemppainen J, Punnett A, Leung F. Assessment of Resident and AI Chatbot Performance on the University of Toronto Family Medicine Residency Progress Test: Comparative Study. JMIR Medical Education 2023;9:e50514 View
  10. Miao J, Thongprayoon C, Garcia Valencia O, Krisanapan P, Sheikh M, Davis P, Mekraksakit P, Suarez M, Craici I, Cheungpasitporn W. Performance of ChatGPT on Nephrology Test Questions. Clinical Journal of the American Society of Nephrology 2024;19(1):35 View
  11. Gao Z, Li L, Ma S, Wang Q, Hemphill L, Xu R. Examining the Potential of ChatGPT on Biomedical Information Retrieval: Fact-Checking Drug-Disease Associations. Annals of Biomedical Engineering 2024;52(8):1919 View
  12. Shimizu I, Kasai H, Shikino K, Araki N, Takahashi Z, Onodera M, Kimura Y, Tsukamoto T, Yamauchi K, Asahina M, Ito S, Kawakami E. Developing Medical Education Curriculum Reform Strategies to Address the Impact of Generative AI: Qualitative Study. JMIR Medical Education 2023;9:e53466 View
  13. Guillen-Grima F, Guillen-Aguinaga S, Guillen-Aguinaga L, Alas-Brun R, Onambele L, Ortega W, Montejo R, Aguinaga-Ontoso E, Barach P, Aguinaga-Ontoso I. Evaluating the Efficacy of ChatGPT in Navigating the Spanish Medical Residency Entrance Examination (MIR): Promising Horizons for AI in Clinical Medicine. Clinics and Practice 2023;13(6):1460 View
  14. Torres-Zegarra B, Rios-Garcia W, Ñaña-Cordova A, Arteaga-Cisneros K, Chalco X, Ordoñez M, Rios C, Godoy C, Quezada K, Gutierrez-Arratia J, Flores-Cohaila J. Performance of ChatGPT, Bard, Claude, and Bing on the Peruvian National Licensing Medical Examination: a cross-sectional study. Journal of Educational Evaluation for Health Professions 2023;20:30 View
  15. Watari T, Takagi S, Sakaguchi K, Nishizaki Y, Shimizu T, Yamamoto Y, Tokuda Y. Performance Comparison of ChatGPT-4 and Japanese Medical Residents in the General Medicine In-Training Examination: Comparison Study. JMIR Medical Education 2023;9:e52202 View
  16. Abusoglu S, Serdar M, Unlu A, Abusoglu G. Comparison of three chatbots as an assistant for problem-solving in clinical laboratory. Clinical Chemistry and Laboratory Medicine (CCLM) 2024;62(7):1362 View
  17. Ohta K, Ohta S. The Performance of GPT-3.5, GPT-4, and Bard on the Japanese National Dentist Examination: A Comparison Study. Cureus 2023 View
  18. Sallam M, Al-Salahat K. Below average ChatGPT performance in medical microbiology exam compared to university students. Frontiers in Education 2023;8 View
  19. Morishita M, Fukuda H, Muraoka K, Nakamura T, Hayashi M, Yoshioka I, Ono K, Awano S. Evaluating GPT-4V’s performance in the Japanese national dental examination: A challenge explored. Journal of Dental Sciences 2024;19(3):1595 View
  20. Lanera C, Lorenzoni G, Barbieri E, Piras G, Magge A, Weissenbacher D, Donà D, Cantarutti L, Gonzalez-Hernandez G, Giaquinto C, Gregori D. Monitoring the Epidemiology of Otitis Using Free-Text Pediatric Medical Notes: A Deep Learning Approach. Journal of Personalized Medicine 2023;14(1):28 View
  21. Tangadulrat P, Sono S, Tangtrakulwanich B. Using ChatGPT for Clinical Practice and Medical Education: Cross-Sectional Survey of Medical Students’ and Physicians’ Perceptions. JMIR Medical Education 2023;9:e50658 View
  22. Miao J, Thongprayoon C, Suppadungsuk S, Garcia Valencia O, Qureshi F, Cheungpasitporn W. Ethical Dilemmas in Using AI for Academic Writing and an Example Framework for Peer Review in Nephrology Academia: A Narrative Review. Clinics and Practice 2023;14(1):89 View
  23. Roberts R, Ali S, Hutchings H, Dobbs T, Whitaker I. Comparative study of ChatGPT and human evaluators on the assessment of medical literature according to recognised reporting standards. BMJ Health & Care Informatics 2023;30(1):e100830 View
  24. Kollitsch L, Eredics K, Marszalek M, Rauchenwald M, Brookman-May S, Burger M, Körner-Riffard K, May M. How does artificial intelligence master urological board examinations? A comparative analysis of different Large Language Models’ accuracy and reliability in the 2022 In-Service Assessment of the European Board of Urology. World Journal of Urology 2024;42(1) View
  25. Odabashian R, Bastin D, Jones G, Manzoor M, Tangestaniapour S, Assad M, Lakhani S, Odabashian M, McGee S. Assessment of ChatGPT-3.5's Knowledge in Oncology: Comparative Study with ASCO-SEP Benchmarks. JMIR AI 2024;3:e50442 View
  26. Morjaria L, Burns L, Bracken K, Levinson A, Ngo Q, Lee M, Sibbald M. Examining the Efficacy of ChatGPT in Marking Short-Answer Assessments in an Undergraduate Medical Program. International Medical Education 2024;3(1):32 View
  27. Herrmann-Werner A, Festl-Wietek T, Holderried F, Herschbach L, Griewatz J, Masters K, Zipfel S, Mahling M. Assessing ChatGPT’s Mastery of Bloom’s Taxonomy Using Psychosomatic Medicine Exam Questions: Mixed-Methods Study. Journal of Medical Internet Research 2024;26:e52113 View
  28. Meyer A, Riese J, Streichert T. Comparison of the Performance of GPT-3.5 and GPT-4 With That of Medical Students on the Written German Medical Licensing Examination: Observational Study. JMIR Medical Education 2024;10:e50965 View
  29. Li D, Kao Y, Tsai S, Bai Y, Yeh T, Chu C, Hsu C, Cheng S, Hsu T, Liang C, Su K. Comparing the performance of ChatGPT GPT‐4, Bard, and Llama‐2 in the Taiwan Psychiatric Licensing Examination and in differential diagnosis with multi‐center psychiatrists. Psychiatry and Clinical Neurosciences 2024;78(6):347 View
  30. Yamaguchi S, Morishita M, Fukuda H, Muraoka K, Nakamura T, Yoshioka I, Soh I, Ono K, Awano S. Evaluating the efficacy of leading large language models in the Japanese national dental hygienist examination: A comparative analysis of ChatGPT, Bard, and Bing Chat. Journal of Dental Sciences 2024;19(4):2262 View
  31. Gordon M, Daniel M, Ajiboye A, Uraiby H, Xu N, Bartlett R, Hanson J, Haas M, Spadafore M, Grafton-Clarke C, Gasiea R, Michie C, Corral J, Kwan B, Dolmans D, Thammasitboon S. A scoping review of artificial intelligence in medical education: BEME Guide No. 84. Medical Teacher 2024;46(4):446 View
  32. Lee G, Hong D, Kim S, Kim J, Lee Y, Park S, Lee K. Comparison of the problem-solving performance of ChatGPT-3.5, ChatGPT-4, Bing Chat, and Bard for the Korean emergency medicine board examination question bank. Medicine 2024;103(9):e37325 View
  33. Nakao T, Miki S, Nakamura Y, Kikuchi T, Nomura Y, Hanaoka S, Yoshikawa T, Abe O. Capability of GPT-4V(ision) in the Japanese National Medical Licensing Examination: Evaluation Study. JMIR Medical Education 2024;10:e54393 View
  34. Lee Y, Kim S. Potential applications of ChatGPT in obstetrics and gynecology in Korea: a review article. Obstetrics & Gynecology Science 2024;67(2):153 View
  35. Funk P, Hoch C, Knoedler S, Knoedler L, Cotofana S, Sofo G, Bashiri Dezfouli A, Wollenberg B, Guntinas-Lichius O, Alfertshofer M. ChatGPT’s Response Consistency: A Study on Repeated Queries of Medical Examination Questions. European Journal of Investigation in Health, Psychology and Education 2024;14(3):657 View
  36. Noda M, Ueno T, Koshu R, Takaso Y, Shimada M, Saito C, Sugimoto H, Fushiki H, Ito M, Nomura A, Yoshizaki T. Performance of GPT-4V in Answering the Japanese Otolaryngology Board Certification Examination Questions: Evaluation Study. JMIR Medical Education 2024;10:e57054 View
  37. Nakajima N, Fujimori T, Furuya M, Kanie Y, Imai H, Kita K, Uemura K, Okada S. A Comparison Between GPT-3.5, GPT-4, and GPT-4V: Can the Large Language Model (ChatGPT) Pass the Japanese Board of Orthopaedic Surgery Examination?. Cureus 2024 View
  38. Şenoymak M, Erbatur N, Şenoymak İ, Fırat S. The Role of Artificial Intelligence in Endocrine Management: Assessing ChatGPT’s Responses to Prolactinoma Queries. Journal of Personalized Medicine 2024;14(4):330 View
  39. Aoki N, Miyagami T, Saita M, Naito T. AI Analysis of General Medicine in Japan: Present and Future Considerations. JMIR Formative Research 2024;8:e52566 View
  40. Sato H, Ogasawara K. ChatGPT (GPT-4) passed the Japanese National License Examination for Pharmacists in 2022, answering all items including those with diagrams: a descriptive study. Journal of Educational Evaluation for Health Professions 2024;21:4 View
  41. Zhu L, Mou W, Lai Y, Lin J, Luo P. Language and cultural bias in AI: comparing the performance of large language models developed in different countries on Traditional Chinese Medicine highlights the need for localized models. Journal of Translational Medicine 2024;22(1) View
  42. Żmudka K, Spychał A, Ochman B, Popowicz Ł, Piłat P, Jaroszewicz J. ChatGPT – a tool for assisted studying or a source of misleading medical information? AI performance on Polish Medical Final Examination. Annales Academiae Medicae Silesiensis 2024;78:94 View
  43. Takagi S, Koda M, Watari T. The Performance of ChatGPT-4V in Interpreting Images and Tables in the Japanese Medical Licensing Exam. JMIR Medical Education 2024;10:e54283 View
  44. Mayo-Yáñez M, Lechien J, Maria-Saibene A, Vaira L, Maniaci A, Chiesa-Estomba C. Examining the Performance of ChatGPT 3.5 and Microsoft Copilot in Otolaryngology: A Comparative Study with Otolaryngologists’ Evaluation. Indian Journal of Otolaryngology and Head & Neck Surgery 2024;76(4):3465 View
  45. Wu J, Wu X, Qiu Z, Li M, Lin S, Zhang Y, Zheng Y, Yuan C, Yang J. Large language models leverage external knowledge to extend clinical insight beyond language boundaries. Journal of the American Medical Informatics Association 2024;31(9):2054 View
  46. Noda M, Yoshimura H, Okubo T, Koshu R, Uchiyama Y, Nomura A, Ito M, Takumi Y. Feasibility of Multimodal Artificial Intelligence Using GPT-4 Vision for the Classification of Middle Ear Disease: Qualitative Study and Validation. JMIR AI 2024;3:e58342 View
  47. Hultberg P, Santandreu Calonge D, Kamalov F, Smail L, Mizuno T. Comparing and assessing four AI chatbots’ competence in economics. PLOS ONE 2024;19(5):e0297804 View
  48. Bharatha A, Ojeh N, Fazle Rabbi A, Campbell M, Krishnamurthy K, Layne-Yarde R, Kumar A, Springer D, Connell K, Majumder M. Comparing the Performance of ChatGPT-4 and Medical Students on MCQs at Varied Levels of Bloom’s Taxonomy. Advances in Medical Education and Practice 2024;Volume 15:393 View
  49. Yang W, Chan Y, Huang C, Chen T. Comparative analysis of GPT-3.5 and GPT-4.0 in Taiwan’s medical technologist certification: A study in artificial intelligence advancements. Journal of the Chinese Medical Association 2024;87(5):525 View
  50. Hirano Y, Hanaoka S, Nakao T, Miki S, Kikuchi T, Nakamura Y, Nomura Y, Yoshikawa T, Abe O. GPT-4 Turbo with Vision fails to outperform text-only GPT-4 Turbo in the Japan Diagnostic Radiology Board Examination. Japanese Journal of Radiology 2024;42(8):918 View
  51. Yavuz Y, Kahraman F. Evaluation of the prediagnosis and management of ChatGPT-4.0 in clinical cases in cardiology. Future Cardiology 2024;20(4):197 View
  52. Shikino K, Shimizu T, Otsuka Y, Tago M, Takahashi H, Watari T, Sasaki Y, Iizuka G, Tamura H, Nakashima K, Kunitomo K, Suzuki M, Aoyama S, Kosaka S, Kawahigashi T, Matsumoto T, Orihara F, Morikawa T, Nishizawa T, Hoshina Y, Yamamoto Y, Matsuo Y, Unoki Y, Kimura H, Tokushima M, Watanuki S, Saito T, Otsuka F, Tokuda Y. Evaluation of ChatGPT-Generated Differential Diagnosis for Common Diseases With Atypical Presentation: Descriptive Research. JMIR Medical Education 2024;10:e58758 View
  53. Kaneda Y, Tayuinosho A, Tomoyose R, Takita M, Hamaki T, Tanimoto T, Ozaki A. Evaluating ChatGPT's effectiveness and tendencies in Japanese internal medicine. Journal of Evaluation in Clinical Practice 2024;30(6):1017 View
  54. Igarashi Y, Nakahara K, Norii T, Miyake N, Tagami T, Yokobori S. Performance of a Large Language Model on Japanese Emergency Medicine Board Certification Examinations. Journal of Nippon Medical School 2024;91(2):155 View
  55. Mousavi M, Shafiee S, Harley J, Cheung J, Abbasgholizadeh Rahimi S. Performance of generative pre-trained transformers (GPTs) in Certification Examination of the College of Family Physicians of Canada. Family Medicine and Community Health 2024;12(Suppl 1):e002626 View
  56. Yaïci R, Cieplucha M, Bock R, Moayed F, Bechrakis N, Berens P, Feltgen N, Friedburg D, Gräf M, Guthoff R, Hoffmann E, Hoerauf H, Hintschich C, Kohnen T, Messmer E, Nentwich M, Pleyer U, Schaudig U, Seitz B, Geerling G, Roth M. ChatGPT und die deutsche Facharztprüfung für Augenheilkunde: eine Evaluierung. Die Ophthalmologie 2024;121(7):554 View
  57. Meyer A, Soleman A, Riese J, Streichert T. Comparison of ChatGPT, Gemini, and Le Chat with physician interpretations of medical laboratory questions from an online health forum. Clinical Chemistry and Laboratory Medicine (CCLM) 2024;62(12):2425 View
  58. Bhattaru A, Yanamala N, Sengupta P. Revolutionizing Cardiology With Words: Unveiling the Impact of Large Language Models in Medical Science Writing. Canadian Journal of Cardiology 2024;40(10):1950 View
  59. Griewing S, Knitza J, Boekhoff J, Hillen C, Lechner F, Wagner U, Wallwiener M, Kuhn S. Evolution of publicly available large language models for complex decision-making in breast cancer care. Archives of Gynecology and Obstetrics 2024;310(1):537 View
  60. Harada Y, Suzuki T, Harada T, Sakamoto T, Ishizuka K, Miyagami T, Kawamura R, Kunitomo K, Nagano H, Shimizu T, Watari T. Performance evaluation of ChatGPT in detecting diagnostic errors and their contributing factors: an analysis of 545 case reports of diagnostic errors. BMJ Open Quality 2024;13(2):e002654 View
  61. Liu M, Okuhara T, Chang X, Shirabe R, Nishiie Y, Okada H, Kiuchi T. Performance of ChatGPT Across Different Versions in Medical Licensing Examinations Worldwide: Systematic Review and Meta-Analysis. Journal of Medical Internet Research 2024;26:e60807 View
  62. Suwała S, Szulc P, Guzowski C, Kamińska B, Dorobiała J, Wojciechowska K, Berska M, Kubicka O, Kosturkiewicz O, Kosztulska B, Rajewska A, Junik R. ChatGPT-3.5 passes Poland’s medical final examination—Is it possible for ChatGPT to become a doctor in Poland?. SAGE Open Medicine 2024;12 View
  63. Ando K, Sato M, Wakatsuki S, Nagai R, Chino K, Kai H, Sasaki T, Kato R, Nguyen T, Guo N, Sultan P. A comparative study of English and Japanese ChatGPT responses to anaesthesia-related medical questions. BJA Open 2024;10:100296 View
  64. Ming S, Guo Q, Cheng W, Lei B. Influence of Model Evolution and System Roles on ChatGPT’s Performance in Chinese Medical Licensing Exams: Comparative Study. JMIR Medical Education 2024;10:e52784 View
  65. Lahat A, Sharif K, Zoabi N, Shneor Patt Y, Sharif Y, Fisher L, Shani U, Arow M, Levin R, Klang E. Assessing Generative Pretrained Transformers (GPT) in Clinical Decision-Making: Comparative Analysis of GPT-3.5 and GPT-4. Journal of Medical Internet Research 2024;26:e54571 View
  66. Fukuda H, Morishita M, Muraoka K, Yamaguchi S, Nakamura T, Yoshioka I, Awano S, Ono K. Evaluating the image recognition capabilities of GPT-4V and Gemini Pro in the Japanese national dental examination. Journal of Dental Sciences 2025;20(1):368 View
  67. Lin C, Plooy K, Gray A, Brown D, Hobbs L, Patterson T, Tan V, Fridberg D, Hsu C. The Performance of ChatGPT on Short-answer Questions in a Psychiatry Examination: A Pilot Study. Taiwanese Journal of Psychiatry 2024;38(2):94 View
  68. Zhao N, Wu T, Wang W, Zhang L, Gong X. Review and Comparative Analysis of Methods and Advancements in Predicting Protein Complex Structure. Interdisciplinary Sciences: Computational Life Sciences 2024;16(2):261 View
  69. Bortoli M, Fiore M, Tedeschi S, Oliveira V, Sousa R, Bruschi A, Campanacci D, Viale P, De Paolis M, Sambri A. GPT-based chatbot tools are still unreliable in the management of prosthetic joint infections. MUSCULOSKELETAL SURGERY 2024;108(4):459 View
  70. Jo E, Song S, Kim J, Lim S, Kim J, Cha J, Kim Y, Joo H. Assessing GPT-4’s Performance in Delivering Medical Advice: Comparative Analysis With Human Experts. JMIR Medical Education 2024;10:e51282 View
  71. Shang L, Li R, Xue M, Guo Q, Hou Y. Evaluating the application of ChatGPT in China’s residency training education: An exploratory study. Medical Teacher 2025;47(5):858 View
  72. Terwilliger E, Bcharah G, Bcharah H, Bcharah E, Richardson C, Scheffler P. Advancing Medical Education: Performance of Generative Artificial Intelligence Models on Otolaryngology Board Preparation Questions With Image Analysis Insights. Cureus 2024 View
  73. Sabri H, Saleh M, Hazrati P, Merchant K, Misch J, Kumar P, Wang H, Barootchi S. Performance of three artificial intelligence (AI)‐based large language models in standardized testing; implications for AI‐assisted dental education. Journal of Periodontal Research 2025;60(2):121 View
  74. Cherif H, Moussa C, Missaoui A, Salouage I, Mokaddem S, Dhahri B. Appraisal of ChatGPT’s Aptitude for Medical Education: Comparative Analysis With Third-Year Medical Students in a Pulmonology Examination. JMIR Medical Education 2024;10:e52818 View
  75. Hirosawa T, Harada Y, Mizuta K, Sakamoto T, Tokumasu K, Shimizu T. Diagnostic performance of generative artificial intelligences for a series of complex case reports. DIGITAL HEALTH 2024;10 View
  76. Hale J, Alexander S, Wright S, Gilliland K. Generative AI in Undergraduate Medical Education: A Rapid Review. Journal of Medical Education and Curricular Development 2024;11 View
  77. Nicikowski J, Szczepański M, Miedziaszczyk M, Kudliński B. The potential of ChatGPT in medicine: an example analysis of nephrology specialty exams in Poland. Clinical Kidney Journal 2024;17(8) View
  78. Vij O, Calver H, Myall N, Dey M, Kouranloo K, Fernandes T. Evaluating the competency of ChatGPT in MRCP Part 1 and a systematic literature review of its capabilities in postgraduate medical assessments. PLOS ONE 2024;19(7):e0307372 View
  79. Chow J, Cheng T, Chien T, Chou W. Assessing ChatGPT’s Capability for Multiple Choice Questions Using RaschOnline: Observational Study. JMIR Formative Research 2024;8:e46800 View
  80. Ishida K, Hanada E. Potential of ChatGPT to Pass the Japanese Medical and Healthcare Professional National Licenses: A Literature Review. Cureus 2024 View
  81. Moglia A, Georgiou K, Cerveri P, Mainardi L, Satava R, Cuschieri A. Large language models in healthcare: from a systematic review on medical examinations to a comparative analysis on fundamentals of robotic surgery online test. Artificial Intelligence Review 2024;57(9) View
  82. Ishida K, Arisaka N, Fujii K. Analysis of Responses of GPT-4 V to the Japanese National Clinical Engineer Licensing Examination. Journal of Medical Systems 2024;48(1) View
  83. Sallam M, Al-Salahat K, Eid H, Egger J, Puladi B. Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5 and Humans in Clinical Chemistry Multiple-Choice Questions. Advances in Medical Education and Practice 2024;Volume 15:857 View
  84. Wang D, Zhang S. Large language models in medical and healthcare fields: applications, advances, and challenges. Artificial Intelligence Review 2024;57(11) View
  85. Jin H, Lee H, Kim E. Performance of ChatGPT-3.5 and GPT-4 in national licensing examinations for medicine, pharmacy, dentistry, and nursing: a systematic review and meta-analysis. BMC Medical Education 2024;24(1) View
  86. Yoon S, Oh S, Lim B, Lee H. Performance of ChatGPT in the In-Training Examination for Anesthesiology and Pain Medicine Residents in South Korea: Observational Study. JMIR Medical Education 2024;10:e56859 View
  87. Sanders A, Lim R, Jones D, Vosburg R. Artificial intelligence large language model scores highly on focused practice designation in metabolic and bariatric surgery board practice questions. Surgical Endoscopy 2024;38(11):6678 View
  88. Fujimoto M, Kuroda H, Katayama T, Yamaguchi A, Katagiri N, Kagawa K, Tsukimoto S, Nakano A, Imaizumi U, Sato-Boku A, Kishimoto N, Itamiya T, Kido K, Sanuki T. Evaluating Large Language Models in Dental Anesthesiology: A Comparative Analysis of ChatGPT-4, Claude 3 Opus, and Gemini 1.0 on the Japanese Dental Society of Anesthesiology Board Certification Exam. Cureus 2024 View
  89. Brin D, Sorin V, Konen E, Nadkarni G, Glicksberg B, Klang E. How GPT models perform on the United States medical licensing examination: a systematic review. Discover Applied Sciences 2024;6(10) View
  90. Shiraishi M, Tsuruda S, Tomioka Y, Chang J, Hori A, Ishii S, Fujinaka R, Ando T, Ohba J, Okazaki M. Advancement of Generative Pre-trained Transformer Chatbots in Answering Clinical Questions in the Practical Rhinoplasty Guideline. Aesthetic Plastic Surgery 2025;49(7):1874 View
  91. Mankowski M, Jaffe I, Xu J, Bae S, Oermann E, Aphinyanaphongs Y, McAdams‐DeMarco M, Lonze B, Orandi B, Stewart D, Levan M, Massie A, Gentry S, Segev D. ChatGPT Solving Complex Kidney Transplant Cases: A Comparative Study With Human Respondents. Clinical Transplantation 2024;38(10) View
  92. Hirata K, Matsui Y, Yamada A, Fujioka T, Yanagawa M, Nakaura T, Ito R, Ueda D, Fujita S, Tatsugami F, Fushimi Y, Tsuboyama T, Kamagata K, Nozaki T, Fujima N, Kawamura M, Naganawa S. Generative AI and large language models in nuclear medicine: current status and future prospects. Annals of Nuclear Medicine 2024;38(11):853 View
  93. Seo J, Choi D, Kim T, Cha W, Kim M, Yoo H, Oh N, Yi Y, Lee K, Choi E. Evaluation Framework of Large Language Models in Medical Documentation: Development and Usability Study. Journal of Medical Internet Research 2024;26:e58329 View
  94. Goto H, Shiraishi Y, Okada S. Performance of Generative Pre-trained Transformer (GPT)-4 and Gemini Advanced on the First-Class Radiation Protection Supervisor Examination in Japan. Cureus 2024 View
  95. Huwiler J, Oechslin L, Biaggi P, Tanner F, Wyss C. Experimental assessment of the performance of artificial intelligence in solving multiple-choice board exams in cardiology. Swiss Medical Weekly 2024;154(10):3547 View
  96. Ros-Arlanzón P, Perez-Sempere A. Evaluating AI Competence in Specialized Medicine: Comparative Analysis of ChatGPT and Neurologists in a Neurology Specialist Examination in Spain. JMIR Medical Education 2024;10:e56762 View
  97. Wu J, Nishida T, Liu T. Accuracy of large language models in answering ophthalmology board-style questions: A meta-analysis. Asia-Pacific Journal of Ophthalmology 2024;13(5):100106 View
  98. Li C, Zhang J, Abdul‐Masih J, Zhang S, Yang J. Performance of ChatGPT and Dental Students on Concepts of Periodontal Surgery. European Journal of Dental Education 2025;29(1):36 View
  99. Liu M, Okuhara T, Chang X, Okada H, Kiuchi T, Khlaif Z. Performance of ChatGPT in medical licensing examinations in countries worldwide: A systematic review and meta-analysis protocol. PLOS ONE 2024;19(10):e0312771 View
  100. Harigai A, Toyama Y, Nagano M, Abe M, Kawabata M, Li L, Yamamura J, Takase K. Response accuracy of GPT-4 across languages: insights from an expert-level diagnostic radiology examination in Japan. Japanese Journal of Radiology 2025;43(2):319 View
  101. Zeng J, Zou X, Li S, Tang Y, Teng S, Li H, Wang C, Wu Y, Zhang L, Zhong Y, Liu J, Liu S. Assessing the Role of the Generative Pretrained Transformer (GPT) in Alzheimer’s Disease Management: Comparative Study of Neurologist- and Artificial Intelligence–Generated Responses. Journal of Medical Internet Research 2024;26:e51095 View
  102. Tokgöz Kaplan T, Cankar M. Evidence‐Based Potential of Generative Artificial Intelligence Large Language Models on Dental Avulsion: ChatGPT Versus Gemini. Dental Traumatology 2025;41(2):178 View
  103. Taniguchi M, Lindsey J. Performance of chatbots in queries concerning fundamental concepts in photochemistry. Photochemistry and Photobiology 2024 View
  104. Uehara O, Morikawa T, Harada F, Sugiyama N, Matsuki Y, Hiraki D, Sakurai H, Kado T, Yoshida K, Murata Y, Matsuoka H, Nagasawa T, Furuichi Y, Abiko Y, Miura H. Performance of ChatGPT‐3.5 and ChatGPT‐4o in the Japanese National Dental Examination. Journal of Dental Education 2025;89(4):459 View
  105. Aster A, Laupichler M, Rockwell-Kollmann T, Masala G, Bala E, Raupach T. ChatGPT and Other Large Language Models in Medical Education — Scoping Literature Review. Medical Science Educator 2024;35(1):555 View
  106. Ho C, Tian T, Ayers A, Aaron R, Phillips V, Wolf R, Mathioudakis N, Dai T, Klonoff D. Qualitative metrics from the biomedical literature for evaluating large language models in clinical decision-making: a narrative review. BMC Medical Informatics and Decision Making 2024;24(1) View
  107. Chen Y, Huang X, Yang F, Lin H, Lin H, Zheng Z, Liang Q, Zhang J, Li X. Performance of ChatGPT and Bard on the medical licensing examinations varies across different cultures: a comparison study. BMC Medical Education 2024;24(1) View
  108. Mavrych V, Ganguly P, Bolgova O. Using large language models (ChatGPT, Copilot, PaLM, Bard, and Gemini) in Gross Anatomy course: Comparative analysis. Clinical Anatomy 2025;38(2):200 View
  109. Miyazaki Y, Hata M, Omori H, Hirashima A, Nakagawa Y, Eto M, Takahashi S, Ikeda M. Performance of ChatGPT-4o on the Japanese Medical Licensing Examination: Evalution of Accuracy in Text-Only and Image-Based Questions. JMIR Medical Education 2024;10:e63129 View
  110. Lee J, Park S, Shin J, Cho B. Analyzing evaluation methods for large language models in the medical field: a scoping review. BMC Medical Informatics and Decision Making 2024;24(1) View
  111. 石田 開, 有阪 直, 藤井 清. Analysis of Japanese Clinical Engineer License Examinations using ChatGPT (GPT－4V). Iryou kikigaku (The Japanese journal of medical instrumentation) 2024;94(5):514 View
  112. Li Y, Huang C, Hu Y, Zhou X, He C, Zhong J. Exploring the performance of large language models on hepatitis B infection-related questions: A comparative study. World Journal of Gastroenterology 2025;31(3) View
  113. Kusaka S, Akitomo T, Hamada M, Asao Y, Iwamoto Y, Tachikake M, Mitsuhata C, Nomura R. Usefulness of Generative Artificial Intelligence (AI) Tools in Pediatric Dentistry. Diagnostics 2024;14(24):2818 View
  114. Ramnani S, Bhalla M, Bassi R. A Comparative Study of ChatGPT and BingAI in Answering the National Eligibility Entrance Test for Postgraduates (NEET-PG)-Style Practice Questions: A Cross-Sectional Analysis. Cureus 2024 View
  115. Fukushima T, Manabe M, Yada S, Wakamiya S, Yoshida A, Urakawa Y, Maeda A, Kan S, Takahashi M, Aramaki E. Evaluating and Enhancing Japanese Large Language Models for Genetic Counseling Support: Comparative Study of Domain Adaptation and the Development of an Expert-Evaluated Dataset. JMIR Medical Informatics 2025;13:e65047 View
  116. Pavone M, Palmieri L, Bizzarri N, Rosati A, Campolo F, Innocenzi C, Taliento C, Restaino S, Catena U, Vizzielli G, Akladios C, Ianieri M, Marescaux J, Campo R, Fanfani F, Scambia G. Artificial Intelligence, the ChatGPT Large Language Model: Assessing the Accuracy of Responses to the Gynaecological Endoscopic Surgical Education and Assessment (GESEA) Level 1-2 knowledge tests. Facts, Views and Vision in ObGyn 2024;16(4):449 View
  117. Roos J, Martin R, Kaczmarczyk R. Evaluating Bard Gemini Pro and GPT-4 Vision Against Student Performance in Medical Visual Question Answering: Comparative Case Study. JMIR Formative Research 2024;8:e57592 View
  118. Abdelgadir Y, Thongprayoon C, Craici I, Cheungpasitporn W, Miao J. Enhancing Patient Comprehension of Glomerular Disease Treatments Using ChatGPT. Healthcare 2024;13(1):57 View
  119. Ishida K, Hanada E. ChatGPT (GPT-4V) Performance on the Healthcare Information Technologist Examination in Japan. Cureus 2025 View
  120. Johri S, Jeong J, Tran B, Schlessinger D, Wongvibulsin S, Barnes L, Zhou H, Cai Z, Van Allen E, Kim D, Daneshjou R, Rajpurkar P. An evaluation framework for clinical use of large language models in patient interaction tasks. Nature Medicine 2025;31(1):77 View
  121. Sun D, Xu P, Zhang J, Liu R, Zhang J. How Self-Regulated Learning Is Affected by Feedback Based on Large Language Models: Data-Driven Sustainable Development in Computer Programming Learning. Electronics 2025;14(1):194 View
  122. Abdulrab S, Abada H, Mashyakhy M, Mostafa N, Alhadainy H, Halboub E. Performance of 4 Artificial Intelligence Chatbots in Answering Endodontic Questions. Journal of Endodontics 2025;51(5):602 View
  123. Kondo T, Okamoto M, Kondo Y. Pilot Study on Using Large Language Models for Educational Resource Development in Japanese Radiological Technologist Exams. Medical Science Educator 2025;35(2):919 View
  124. Kim J, Vajravelu B. Assessing the Current Limitations of Large Language Models in Advancing Health Care Education. JMIR Formative Research 2025;9:e51319 View
  125. Öztürk Z, Bal C, Çelikkaya B. Evaluation of Information Provided by ChatGPT Versions on Traumatic Dental Injuries for Dental Students and Professionals. Dental Traumatology 2025 View
  126. Wang Y, Shen H, Chen T, Chiang S, Lin T. Performance of ChatGPT-3.5 and ChatGPT-4 in the Taiwan National Pharmacist Licensing Examination: Comparative Evaluation Study. JMIR Medical Education 2025;11:e56850 View
  127. Yoşumaz İ. Generative Artificial Intelligence and Usage in Academia. Fırat Üniversitesi Sosyal Bilimler Dergisi 2025;35(1):1 View
  128. Nomura A, Takeji Y, Shimojima M, Takamura M. Digitalomics: Towards Artificial Intelligence / Machine Learning-Based Precision Cardiovascular Medicine. Circulation Journal 2025 View
  129. Franke S, Pott C, Rutinowski J, Pauly M, Reining C, Kirchheim A. Can ChatGPT Solve Undergraduate Exams from Warehousing Studies? An Investigation. Computers 2025;14(2):52 View
  130. Salman I, Ameer O, Khanfar M, Hsieh Y. Artificial intelligence in healthcare education: evaluating the accuracy of ChatGPT, Copilot, and Google Gemini in cardiovascular pharmacology. Frontiers in Medicine 2025;12 View
  131. Borsetto D, Sia E, Axon P, Donnelly N, Tysome J, Anschuetz L, Bernardeschi D, Capriotti V, Caye-Thomasen P, West N, Erbele I, Franchella S, Gatto A, Hess-Erga J, Kunst H, Marinelli J, Mannion R, Panizza B, Trabalzini F, Obholzer R, Vaira L, Polesel J, Giudici F, Carlson M, Tirelli G, Boscolo-Rizzo P. Quality of Information Provided by Artificial Intelligence Chatbots Surrounding the Management of Vestibular Schwannomas: A Comparative Analysis Between ChatGPT-4 and Claude 2. Otology & Neurotology 2025;46(4):432 View
  132. Liu Q, Hu A, Gladman T, Gallagher S. Eight Months into Reality: A Scoping Review of the Application of ChatGPT in Higher Education Teaching and Learning. Innovative Higher Education 2025 View
  133. Işik G, Kafadar-Gürbüz İ, Elgün F, Kara R, Berber B, Özgül S, Günbay T. Is Artificial Intelligence a Useful Tool for Clinical Practice of Oral and Maxillofacial Surgery?. Journal of Craniofacial Surgery 2025;36(2):558 View
  134. Topcu T, Husain M, Ofsa M, Wach P. Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert‐Like Systems Engineering Artifacts and a Characterization of Failure Modes. Systems Engineering 2025 View
  135. Hallquist E, Gupta I, Montalbano M, Loukas M. Applications of Artificial Intelligence in Medical Education: A Systematic Review. Cureus 2025 View
  136. Chaddad A, Jiang Y. Integrating Technologies in the Metaverse for Enhanced Healthcare and Medical Education. IEEE Transactions on Learning Technologies 2025;18:216 View
  137. Peng L, Wu Y, Sun J, Xing Y, Li M, Li M. Will Artificial Intelligence Nurse Practitioners Become True? Performance Evaluation of ChatGPT in the American Association of Nurse Practitioners Exams. AI, Computer Science and Robotics Technology 2025;4 View
  138. Oprea S, Bâra A. Interpreting text corpora from androids-related stories using large language models: “Machines like me” by Ian McEwan in generative AI. Humanities and Social Sciences Communications 2025;12(1) View
  139. Sav N. Assessing the performance of large language models (GPT-3.5 and GPT-4) and accurate clinical information for pediatric nephrology. Pediatric Nephrology 2025 View
  140. Gan W, Ouyang J, She G, Xue Z, Zhu L, Lin A, Mou W, Jiang A, Qi C, Cheng Q, Luo P, Li H, Zheng X. ChatGPT’s role in alleviating anxiety in total knee arthroplasty consent process: a randomized controlled trial pilot study. International Journal of Surgery 2025;111(3):2546 View
  141. Ishida K. Evaluating Chat Generative Pretrained Transformer (GPT-4o) Problem-Solving Performance in the Japan Certificate Examination for Biomedical Engineering Class 1. Cureus 2025 View
  142. Mavrych V, Yaqinuddin A, Bolgova O. Claude, ChatGPT, Copilot, and Gemini performance versus students in different topics of neuroscience. Advances in Physiology Education 2025;49(2):430 View
  143. Rodrigues Alessi M, Gomes H, Oliveira G, Lopes de Castro M, Grenteski F, Miyashiro L, do Valle C, Tozzini Tavares da Silva L, Okamoto C. Comparative Performance of Medical Students, ChatGPT-3.5 and ChatGPT-4.0 in Answering Questions From a Brazilian National Medical Exam: Cross-Sectional Questionnaire Study. JMIR AI 2025;4:e66552 View
  144. Yang B, Zhou F, Bai N, Zhou S, Luo C, Wang Q, Wong A, Lin F. Digital and Intelligence Education in Medicine: A Bibliometric and Visualization Analysis Using CiteSpace and VOSviewer. Frontiers of Digital Education 2025;2(1) View
  145. Matsutomo N, Fukami M, Yamamoto T. Can interactive artificial intelligence be used for patient explanations of nuclear medicine examinations in Japanese?. Annals of Nuclear Medicine 2025 View
  146. Yu Y, Kim S, Lee W, Koo B. Evaluating ChatGPT on Korea's BIM Expertise Exam and improving its performance through RAG. Journal of Computational Design and Engineering 2025;12(4):94 View
  147. Luo D, Liu M, Yu R, Liu Y, Jiang W, Fan Q, Kuang N, Gao Q, Yin T, Zheng Z. Evaluating the performance of GPT-3.5, GPT-4, and GPT-4o in the Chinese National Medical Licensing Examination. Scientific Reports 2025;15(1) View
  148. Koçak M, Oğuz A, Akçalı Z. The role of artificial intelligence in medical education: an evaluation of Large Language Models (LLMs) on the Turkish Medical Specialty Training Entrance Exam. BMC Medical Education 2025;25(1) View
  149. Buhl L. The answer may vary: large language model response patterns challenge their use in test item analysis. Medical Teacher 2025:1 View
  150. Yitzhaki S, Peled N, Kaplan E, Kadmon G, Nahum E, Gendler Y, Weissbach A. Comparing ChatGPT‐4 and a Paediatric Intensive Care Specialist in Responding to Medical Education Questions: A Multicenter Evaluation. Journal of Paediatrics and Child Health 2025 View
  151. Chen R, Zhang S, Zheng Y, Yu Q, Wang C. Enhancing treatment decision-making for low back pain: a novel framework integrating large language models with retrieval-augmented generation technology. Frontiers in Medicine 2025;12 View
  152. Wang C, Wang F, Li S, Ren Q, Tan X, Fu Y, Liu D, Qian G, Cao Y, Yin R, Li K. Patient Triage and Guidance in Emergency Departments Using Large Language Models: Multimetric Study. Journal of Medical Internet Research 2025;27:e71613 View
  153. Weuthen F, Otte N, Krabbe H, Kraus T, Krabbe J. Comparison of ChatGPT and Internet Research for Clinical Research and Decision-Making in Occupational Medicine: Randomized Controlled Trial. JMIR Formative Research 2025;9:e63857 View
  154. Wang L, Li J, Zhuang B, Huang S, Fang M, Wang C, Li W, Zhang M, Gong S. Accuracy of Large Language Models When Answering Clinical Research Questions: Systematic Review and Network Meta-Analysis. Journal of Medical Internet Research 2025;27:e64486 View

Books/Policy Documents

  1. Huang D, Wang Z. Trends and Applications in Knowledge Discovery and Data Mining. View
  2. Semujanga B, Mikalef P. Disruptive Innovation in a Digitally Connected Healthy World. View

Conference Proceedings

  1. Xu B, Wang R, Ping L, Zhu C, Liu X, Lin H, Tian L, Xia F. 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). MAT: Medical AI-generated Text Detection Dataset from Multi-models and Multi-Methods View
  2. Hidayaturrahman , Prawira I. 2024 International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS). Leveraging Zero-Shot Learning in Large Language Models for Sentiment Analysis: A Comparative Study on the Indonesian Language View