This paper is in the following e-collection/theme issue:

Theme Issue: ChatGPT and Generative Language Models in Medical Education (92) Chatbots and Conversational Agents (595) Generative Language Models Including ChatGPT (432) Clinical Information and Decision Making (1567) Artificial Intelligence (1623) Natural Language Processing (767)

Published on in Vol 10 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/49970, first published .
A Novel Evaluation Model for Assessing ChatGPT on Otolaryngology–Head and Neck Surgery Certification Examinations: Performance Study

A Novel Evaluation Model for Assessing ChatGPT on Otolaryngology–Head and Neck Surgery Certification Examinations: Performance Study

A Novel Evaluation Model for Assessing ChatGPT on Otolaryngology–Head and Neck Surgery Certification Examinations: Performance Study

Authors of this article:

Cai Long1 Author Orcid Image ;   Kayle Lowe2 Author Orcid Image ;   Jessica Zhang2 Author Orcid Image ;   André dos Santos3 Author Orcid Image ;   Alaa Alanazi1 Author Orcid Image ;   Daniel O'Brien4 Author Orcid Image ;   Erin D Wright1 Author Orcid Image ;   David Cote1 Author Orcid Image
Article Authors Cited by (18) Tweetations (3) Metrics

Journals

  1. Zhu L, Mou W, Hong C, Yang T, Lai Y, Qi C, Lin A, Zhang J, Luo P. The Evaluation of Generative AI Should Include Repetition to Assess Stability. JMIR mHealth and uHealth 2024;12:e57978 View
  2. Lechien J, Rameau A. Applications of ChatGPT in Otolaryngology–Head Neck Surgery: A State of the Art Review. Otolaryngology–Head and Neck Surgery 2024;171(3):667 View
  3. Buhr C, Smith H, Huppertz T, Bahr-Hamm K, Matthias C, Cuny C, Snijders J, Ernst B, Blaikie A, Kelsey T, Kuhn S, Eckrich J. Assessing unknown potential—quality and limitations of different large language models in the field of otorhinolaryngology. Acta Oto-Laryngologica 2024;144(3):237 View
  4. Lechien J. Generative AI and Otolaryngology—Head & Neck Surgery. Otolaryngologic Clinics of North America 2024;57(5):753 View
  5. Patel E, Fleischer L, Filip P, Eggerstedt M, Hutz M, Michaelides E, Batra P, Tajudeen B. Comparative Performance of ChatGPT 3.5 and GPT4 on Rhinology Standardized Board Examination Questions. OTO Open 2024;8(2) View
  6. Kipp M. From GPT-3.5 to GPT-4.o: A Leap in AI’s Medical Exam Performance. Information 2024;15(9):543 View
  7. Merlino D, Brufau S, Saieed G, Van Abel K, Price D, Archibald D, Ator G, Carlson M. Comparative Assessment of Otolaryngology Knowledge Among Large Language Models. The Laryngoscope 2025;135(2):629 View
  8. Patel J, Robinson P, Illing E, Anthony B, Grewal H. Is ChatGPT 3.5 smarter than Otolaryngology trainees? A comparison study of board style exam questions. PLOS ONE 2024;19(9):e0306233 View
  9. Sanders A, Lim R, Jones D, Vosburg R. Artificial intelligence large language model scores highly on focused practice designation in metabolic and bariatric surgery board practice questions. Surgical Endoscopy 2024;38(11):6678 View
  10. Shaari A, Herzog I, Kokush E, Zabielski S, Langer S, Filimonov A. Exploring the Ability of ChatGPT to Act as a Research Aid in Otolaryngology. Indian Journal of Otolaryngology and Head & Neck Surgery 2024;76(6):6100 View
  11. Seifen C, Huppertz T, Gouveris H, Bahr-Hamm K, Pordzik J, Eckrich J, Smith H, Kelsey T, Blaikie A, Matthias C, Kuhn S, Buhr C. Chasing sleep physicians: ChatGPT-4o on the interpretation of polysomnographic results. European Archives of Oto-Rhino-Laryngology 2025;282(3):1631 View
  12. Mete U. Evaluating the Performance of ChatGPT, Gemini, and Bing Compared with Resident Surgeons in the Otorhinolaryngology In-service Training Examination. Turkish Archives of Otorhinolaryngology 2024 View
  13. Cheng H. ChatGPT’s Attitude, Knowledge, and Clinical Application in Geriatrics Practice and Education: Exploratory Observational Study. JMIR Formative Research 2025;9:e63494 View
  14. Ma T. Systematically visualizing ChatGPT used in higher education: Publication trend, disciplinary domains, research themes, adoption and acceptance. Computers and Education: Artificial Intelligence 2025;8:100336 View
  15. Sabaner M, Anguita R, Antaki F, Balas M, Boberg-Ans L, Ferro Desideri L, Grauslund J, Hansen M, Klefter O, Potapenko I, Rasmussen M, Subhi Y. Opportunities and Challenges of Chatbots in Ophthalmology: A Narrative Review. Journal of Personalized Medicine 2024;14(12):1165 View
  16. Helvacioglu-Yigit D, Demirturk H, Ali K, Tamimi D, Koenig L, Almashraqi A. Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology 2025 View
  17. Qiu Y, Liu C. Capable exam-taker and question-generator: the dual role of generative AI in medical education assessment. Global Medical Education 2025 View
  18. Bany Abdelnabi A, Soykan B, Bhatti D, Rabadi G. Usefulness of Large Language Models (LLMs) for Student Feedback on H&P During Clerkship: Artificial Intelligence for Personalized Learning. ACM Transactions on Computing for Healthcare 2025 View