This paper is in the following e-collection/theme issue:

Artificial Intelligence (AI) in Medical Education (112) Generative Language Models Including ChatGPT (432) e-Learning and Digital Medical Education (760) New Methods and Approaches in Medical Education (287) Research Instruments, Questionnaires, and Tools (638) Chatbots and Conversational Agents (595)

Published on in Vol 9 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/52202, first published .
Performance Comparison of ChatGPT-4 and Japanese Medical Residents in the General Medicine In-Training Examination: Comparison Study

Performance Comparison of ChatGPT-4 and Japanese Medical Residents in the General Medicine In-Training Examination: Comparison Study

Performance Comparison of ChatGPT-4 and Japanese Medical Residents in the General Medicine In-Training Examination: Comparison Study

Authors of this article:

Takashi Watari1, 2, 3 Author Orcid Image ;   Soshi Takagi4 Author Orcid Image ;   Kota Sakaguchi1 Author Orcid Image ;   Yuji Nishizaki5 Author Orcid Image ;   Taro Shimizu6 Author Orcid Image ;   Yu Yamamoto7 Author Orcid Image ;   Yasuharu Tokuda8 Author Orcid Image
Article Authors Cited by (17) Tweetations (3) Metrics

Journals

  1. Noda M, Ueno T, Koshu R, Takaso Y, Shimada M, Saito C, Sugimoto H, Fushiki H, Ito M, Nomura A, Yoshizaki T. Performance of GPT-4V in Answering the Japanese Otolaryngology Board Certification Examination Questions: Evaluation Study. JMIR Medical Education 2024;10:e57054 View
  2. Gravina A, Pellegrino R, Palladino G, Imperio G, Ventura A, Federico A. Charting new AI education in gastroenterology: Cross-sectional evaluation of ChatGPT and perplexity AI in medical residency exam. Digestive and Liver Disease 2024;56(8):1304 View
  3. Wang S, Mo C, Chen Y, Dai X, Wang H, Shen X. Exploring the Performance of ChatGPT-4 in the Taiwan Audiologist Qualification Examination: Preliminary Observational Study Highlighting the Potential of AI Chatbots in Hearing Care. JMIR Medical Education 2024;10:e55595 View
  4. GURBUZ D, VARIS E. Is ChatGPT knowledgeable of acute coronary syndromes and pertinent European Society of Cardiology Guidelines?. Minerva Cardiology and Angiology 2024;72(3) View
  5. Liu M, Okuhara T, Chang X, Shirabe R, Nishiie Y, Okada H, Kiuchi T. Performance of ChatGPT Across Different Versions in Medical Licensing Examinations Worldwide: Systematic Review and Meta-Analysis. Journal of Medical Internet Research 2024;26:e60807 View
  6. Takahashi H, Shikino K, Kondo T, Komori A, Yamada Y, Saita M, Naito T. Educational Utility of Clinical Vignettes Generated in Japanese by ChatGPT-4: Mixed Methods Study. JMIR Medical Education 2024;10:e59133 View
  7. Sallam M, Al-Mahzoum K, Alshuaib O, Alhajri H, Alotaibi F, Alkhurainej D, Al-Balwah M, Barakat M, Egger J. Language discrepancies in the performance of generative artificial intelligence models: an examination of infectious disease queries in English and Arabic. BMC Infectious Diseases 2024;24(1) View
  8. Liu M, Okuhara T, Dai Z, Huang W, Gu L, Okada H, Furukawa E, Kiuchi T. Evaluating the Effectiveness of advanced large language models in medical Knowledge: A Comparative study using Japanese national medical examination. International Journal of Medical Informatics 2025;193:105673 View
  9. Eoh K, Kwon G, Lee E, Lee J, Lee I, Kim Y, Nam E. Efficacy of large language models and their potential in Obstetrics and Gynecology education. Obstetrics & Gynecology Science 2024;67(6):550 View
  10. Ho C, Tian T, Ayers A, Aaron R, Phillips V, Wolf R, Mathioudakis N, Dai T, Klonoff D. Qualitative metrics from the biomedical literature for evaluating large language models in clinical decision-making: a narrative review. BMC Medical Informatics and Decision Making 2024;24(1) View
  11. Huang T, Hsieh P, Chang Y. Performance Comparison of Junior Residents and ChatGPT in the Objective Structured Clinical Examination (OSCE) for Medical History Taking and Documentation of Medical Records: Development and Usability Study. JMIR Medical Education 2024;10:e59902 View
  12. Burisch C, Bellary A, Breuckmann F, Ehlers J, Thal S, Sellmann T, Gödde D. ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial. JMIR Research Protocols 2025;14:e63887 View
  13. Fukushima T, Manabe M, Yada S, Wakamiya S, Yoshida A, Urakawa Y, Maeda A, Kan S, Takahashi M, Aramaki E. Evaluating and Enhancing Japanese Large Language Models for Genetic Counseling Support: Comparative Study of Domain Adaptation and the Development of an Expert-Evaluated Dataset. JMIR Medical Informatics 2025;13:e65047 View
  14. Xiao J, Li M, Cai R, Huang H, Yu H, Huang L, Li J, Yu T, Zhang J, Cheng S. Smart Pharmaceutical Monitoring System With Personalized Medication Schedules and Self-Management Programs for Patients With Diabetes: Development and Evaluation Study. Journal of Medical Internet Research 2025;27:e56737 View
  15. Gungor N, Esen F, Tasci T, Gungor K, Cil K. Navigating Gynecological Oncology with Different Versions of ChatGPT: A Transformative Breakthrough or the Next Black Box Challenge?. Oncology Research and Treatment 2024;48(3):102 View
  16. Ye H, Xu J, Huang D, Xie M, Guo J, Yang J, Bao H, Zhang M, Zheng C. Assessment of large language models’ performances and hallucinations for Chinese postgraduate medical entrance examination. Discover Education 2025;4(1) View
  17. Tseng L, Lu Y, Tseng L, Chen Y, Chen H. Performance of ChatGPT-4 on Taiwanese Traditional Chinese Medicine Licensing Examinations: Cross-Sectional Study. JMIR Medical Education 2025;11:e58897 View