This paper is in the following e-collection/theme issue:

Artificial Intelligence (AI) in Medical Education (182) Generative Language Models Including ChatGPT (708) Research Instruments, Questionnaires, and Tools (738) Artificial Intelligence (2129) Chatbots and Conversational Agents (740) Theme Issue: ChatGPT and Generative Language Models in Medical Education (115)

Published on in Vol 10 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/54393, first published .
Capability of GPT-4V(ision) in the Japanese National Medical Licensing Examination: Evaluation Study

Capability of GPT-4V(ision) in the Japanese National Medical Licensing Examination: Evaluation Study

Capability of GPT-4V(ision) in the Japanese National Medical Licensing Examination: Evaluation Study

Authors of this article:

Takahiro Nakao1 Author Orcid Image ;   Soichiro Miki1 Author Orcid Image ;   Yuta Nakamura1 Author Orcid Image ;   Tomohiro Kikuchi1, 2 Author Orcid Image ;   Yukihiro Nomura1, 3 Author Orcid Image ;   Shouhei Hanaoka4 Author Orcid Image ;   Takeharu Yoshikawa1 Author Orcid Image ;   Osamu Abe4 Author Orcid Image
Article Authors Cited by (28) Tweetations (4) Metrics

Journals

  1. Koga S, Du W, Ono D. Response to “Can ChatGPT Vision diagnose melanoma? An exploratory diagnostic accuracy study.”. Journal of the American Academy of Dermatology 2024;91(3):e61 View
  2. Liu M, Okuhara T, Chang X, Shirabe R, Nishiie Y, Okada H, Kiuchi T. Performance of ChatGPT Across Different Versions in Medical Licensing Examinations Worldwide: Systematic Review and Meta-Analysis. Journal of Medical Internet Research 2024;26:e60807 View
  3. Ishida K, Hanada E. Potential of ChatGPT to Pass the Japanese Medical and Healthcare Professional National Licenses: A Literature Review. Cureus 2024 View
  4. Tong W, Zhang X, Zeng H, Pan J, Gong C, Zhang H. Reforming China’s Secondary Vocational Medical Education: Adapting to the Challenges and Opportunities of the AI Era. JMIR Medical Education 2024;10:e48594 View
  5. Liu C, Ho C, Wu T. Custom GPTs Enhancing Performance and Evidence Compared with GPT-3.5, GPT-4, and GPT-4o? A Study on the Emergency Medicine Specialist Examination. Healthcare 2024;12(17):1726 View
  6. Kipp M. From GPT-3.5 to GPT-4.o: A Leap in AI’s Medical Exam Performance. Information 2024;15(9):543 View
  7. Brin D, Sorin V, Konen E, Nadkarni G, Glicksberg B, Klang E. How GPT models perform on the United States medical licensing examination: a systematic review. Discover Applied Sciences 2024;6(10) View
  8. Okada A. Editorial Comment on Can artificial intelligence pass the Japanese urology board examinations?. International Journal of Urology 2024;31(12):1442 View
  9. Morishita M, Fukuda H, Yamaguchi S, Muraoka K, Nakamura T, Hayashi M, Yoshioka I, Ono K, Awano S. An exploratory assessment of GPT-4o and GPT-4 performance on the Japanese National Dental Examination. The Saudi Dental Journal 2024;36(12):1577 View
  10. Zong H, Wu R, Cha J, Wang J, Wu E, Li J, Zhou Y, Zhang C, Feng W, Shen B. Large Language Models in Worldwide Medical Exams: Platform Development and Comprehensive Analysis. Journal of Medical Internet Research 2024;26:e66114 View
  11. Güneş Y, Ülkir M. Comparative Performance Evaluation of Multimodal Large Language Models, Radiologist, and Anatomist in Visual Neuroanatomy Questions. Uludağ Üniversitesi Tıp Fakültesi Dergisi 2025;50(3):551 View
  12. Schramm S, Preis S, Metz M, Jung K, Schmitz-Koep B, Zimmer C, Wiestler B, Hedderich D, Kim S. Impact of Multimodal Prompt Elements on Diagnostic Performance of GPT-4V in Challenging Brain MRI Cases. Radiology 2025;314(1) View
  13. Nguyen H, Dang H, Nguyen T, Hoang V, Nguyen V, Wu J. Accuracy of latest large language models in answering multiple choice questions in dentistry: A comparative study. PLOS ONE 2025;20(1):e0317423 View
  14. Scherr R, Spina A, Dao A, Andalib S, Halaseh F, Blair S, Wiechmann W, Rivera R. Novel Evaluation Metric and Quantified Performance of ChatGPT-4 Patient Management Simulations for Early Clinical Education: Experimental Study. JMIR Formative Research 2025;9:e66478 View
  15. Yang Z, Yao Z, Tasmin M, Vashisht P, Jang W, Ouyang F, Wang B, McManus D, Berlowitz D, Yu H. Unveiling GPT-4V's hidden challenges behind high accuracy on USMLE questions: Observational Study. Journal of Medical Internet Research 2025;27:e65146 View
  16. Mine Y, Taji T, Okazaki S, Takeda S, Peng T, Shimoe S, Kaku M, Nikawa H, Kakimoto N, Murayama T. Analyzing the performance of multimodal large language models on visually-based questions in the Japanese National Examination for Dental Technicians. Journal of Dental Sciences 2025;20(4):2460 View
  17. Luo D, Liu M, Yu R, Liu Y, Jiang W, Fan Q, Kuang N, Gao Q, Yin T, Zheng Z. Evaluating the performance of GPT-3.5, GPT-4, and GPT-4o in the Chinese National Medical Licensing Examination. Scientific Reports 2025;15(1) View
  18. Hamada M, Kikuchi S, Akitomo T, Kusaka S, Iwamoto Y, Nomura R. Applications and potential of ChatGPT in dentistry: Scoping review of research perspectives. Journal of Dental Sciences 2025 View
  19. Liu Z, Zhao C, Lin H. DeepSeek-R1 vs Open-Weight AI in Ophthalmology. JAMA Ophthalmology 2025;143(10):842 View
  20. Wang T, Jheng J, Tseng Y, Chen L, Chen Y. Evaluating GPT-4’s visual interpretation and clinical reasoning on emergency settings: A 5-year analysis. Journal of the Chinese Medical Association 2025;88(9):672 View
  21. Jaleel A, Aziz U, Farid G, Zahid Bashir M, Mirza T, Khizar Abbas S, Aslam S, Sikander R. Evaluating the Potential and Accuracy of ChatGPT-3.5 and 4.0 in Medical Licensing and In-Training Examinations: Systematic Review and Meta-Analysis. JMIR Medical Education 2025;11:e68070 View
  22. Saowaprut P, Wabina R, Yang J, Siriwat L. Performance of large language models on Thailand’s national medical licensing examination: a cross-sectional study. Journal of Educational Evaluation for Health Professions 2025;22:16 View
  23. Kim H, Jung K, Shin S, Lee W, Lee J, Park H, Choi Q. Performance evaluation of large language models on Korean medical licensing examination: a three-year comparative analysis. Scientific Reports 2025;15(1) View
  24. Kasagga A, Sapkota A, Changaramkumarath G, Abucha J, Wollel M, Somannagari N, Husami M, Hailu K, Kasagga E. Performance of ChatGPT and Large Language Models on Medical Licensing Exams Worldwide: A Systematic Review and Network Meta-Analysis With Meta-Regression. Cureus 2025 View
  25. Nguyen V, Vuong T, Nguyen V, Ma, H. Benchmarking large-language-model vision capabilities in oral and maxillofacial anatomy: A cross-sectional study. PLOS One 2025;20(10):e0335775 View
  26. Engelstein H, Ramon-Gonen R, Barbash I, Beinart R, Cohen-Shelly M, Sabbag A. Estimating LVEF from ECG with GPT-4o Fine-Tuned Vision: A Novel Approach in AI-Driven Cardiac Diagnostics. Journal of Medical Systems 2025;49(1) View

Conference Proceedings

  1. Xu M, Ye C, Zeng Z, Chang C, Qi S, Wu Y, Yang H, Chen Y, Huang H, Liu L, Cao Z, Deng X. 2024 IEEE International Conference on Digital Health (ICDH). Adopting Generative AI with Precaution in Dentistry: A Review and Reflection View
  2. Wu G, Cheng C, Pang T. 2024 IEEE International Conference on Future Machine Learning and Data Science (FMLDS). Defect Classification and Localization in Material Extrusion with Multi-Modal Large Language Models View