%0 Journal Article
%@ 2369-3762
%I JMIR Publications
%V 9
%N 
%P e48002
%T Performance of GPT-3.5 and GPT-4 on the Japanese Medical Licensing Examination: Comparison Study
%A Takagi,Soshi
%A Watari,Takashi
%A Erabi,Ayano
%A Sakaguchi,Kota
%+ General Medicine Center, Shimane University Hospital, 89-1, Enya, Izumo, 693-8501, Japan, 81 0853 20 2217, wataritari@gmail.com
%K ChatGPT
%K Chat Generative Pre-trained Transformer
%K GPT-4
%K Generative Pre-trained Transformer 4
%K artificial intelligence
%K AI
%K medical education
%K Japanese Medical Licensing Examination
%K medical licensing
%K clinical support
%K learning model
%D 2023
%7 29.6.2023
%9 Original Paper
%J JMIR Med Educ
%G English
%X Background: The competence of ChatGPT (Chat Generative Pre-Trained Transformer) in non-English languages is not well studied. Objective: This study compared the performances of GPT-3.5 (Generative Pre-trained Transformer) and GPT-4 on the Japanese Medical Licensing Examination (JMLE) to evaluate the reliability of these models for clinical reasoning and medical knowledge in non-English languages. Methods: This study used the default mode of ChatGPT, which is based on GPT-3.5; the GPT-4 model of ChatGPT Plus; and the 117th JMLE in 2023. A total of 254 questions were included in the final analysis, which were categorized into 3 types, namely general, clinical, and clinical sentence questions. Results: The results indicated that GPT-4 outperformed GPT-3.5 in terms of accuracy, particularly for general, clinical, and clinical sentence questions. GPT-4 also performed better on difficult questions and specific disease questions. Furthermore, GPT-4 achieved the passing criteria for the JMLE, indicating its reliability for clinical reasoning and medical knowledge in non-English languages. Conclusions: GPT-4 could become a valuable tool for medical education and clinical support in non–English-speaking regions, such as Japan. 
%M 37384388
%R 10.2196/48002
%U https://mededu.jmir.org/2023/1/e48002
%U https://doi.org/10.2196/48002
%U http://www.ncbi.nlm.nih.gov/pubmed/37384388