This paper is in the following e-collection/theme issue:

Theme Issue: ChatGPT and Generative Language Models in Medical Education (84) e-Learning and Medical Education (691) Artificial Intelligence (1371) Adoption and Change Management of eHealth Systems (712) Chatbots and Conversational Agents (531) Generative Language Models Including ChatGPT (311) Artificial Intelligence (AI) in Medical Education (78) Artificial Intelligence (86)

Published on in Vol 10 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/51148, first published .
Pure Wisdom or Potemkin Villages? A Comparison of ChatGPT 3.5 and ChatGPT 4 on USMLE Step 3 Style Questions: Quantitative Analysis

Pure Wisdom or Potemkin Villages? A Comparison of ChatGPT 3.5 and ChatGPT 4 on USMLE Step 3 Style Questions: Quantitative Analysis

Pure Wisdom or Potemkin Villages? A Comparison of ChatGPT 3.5 and ChatGPT 4 on USMLE Step 3 Style Questions: Quantitative Analysis

Authors of this article:

Leonard Knoedler1 Author Orcid Image ;   Michael Alfertshofer2 Author Orcid Image ;   Samuel Knoedler1, 3 Author Orcid Image ;   Cosima C Hoch4 Author Orcid Image ;   Paul F Funk5 Author Orcid Image ;   Sebastian Cotofana6, 7 Author Orcid Image ;   Bhagvat Maheta8 Author Orcid Image ;   Konstantin Frank9 Author Orcid Image ;   Vanessa Brébant1 Author Orcid Image ;   Lukas Prantl1 Author Orcid Image ;   Philipp Lamby1 Author Orcid Image
Article Authors Cited by (10) Tweetations (4) Metrics

Journals

  1. Funk P, Hoch C, Knoedler S, Knoedler L, Cotofana S, Sofo G, Bashiri Dezfouli A, Wollenberg B, Guntinas-Lichius O, Alfertshofer M. ChatGPT’s Response Consistency: A Study on Repeated Queries of Medical Examination Questions. European Journal of Investigation in Health, Psychology and Education 2024;14(3):657 View
  2. Knoedler L, Knoedler S, Hoch C, Prantl L, Frank K, Soiderer L, Cotofana S, Dorafshar A, Schenck T, Vollbach F, Sofo G, Alfertshofer M. In-depth analysis of ChatGPT’s performance based on specific signaling words and phrases in the question stem of 2377 USMLE step 1 style questions. Scientific Reports 2024;14(1) View
  3. Liu M, Okuhara T, Chang X, Shirabe R, Nishiie Y, Okada H, Kiuchi T. Performance of ChatGPT Across Different Versions in Medical Licensing Examinations Worldwide: Systematic Review and Meta-Analysis. Journal of Medical Internet Research 2024;26:e60807 View
  4. Hibino M, Gillinov M. “Pseudo” Intelligence or Misguided or Mis-sourced Intelligence?. The Annals of Thoracic Surgery 2024;118(1):281 View
  5. Hirosawa T, Harada Y, Mizuta K, Sakamoto T, Tokumasu K, Shimizu T. Diagnostic performance of generative artificial intelligences for a series of complex case reports. DIGITAL HEALTH 2024;10 View
  6. Davis N, El-Said E, Fortune P, Shen A, Succi M. Transforming Health Care Landscapes: The Lever of Radiology Research and Innovation on Emerging Markets Poised for Aggressive Growth. Journal of the American College of Radiology 2024;21(10):1552 View
  7. Jin H, Lee H, Kim E. Performance of ChatGPT-3.5 and GPT-4 in national licensing examinations for medicine, pharmacy, dentistry, and nursing: a systematic review and meta-analysis. BMC Medical Education 2024;24(1) View
  8. Alfertshofer M, Knoedler S, Hoch C, Cotofana S, Panayi A, Kauke-Navarro M, Tullius S, Orgill D, Austen W, Pomahac B, Knoedler L. Analyzing Question Characteristics Influencing ChatGPT’s Performance in 3000 USMLE®-Style Questions. Medical Science Educator 2024 View
  9. Künzle P, Paris S. Performance of large language artificial intelligence models on solving restorative dentistry and endodontics student assessments. Clinical Oral Investigations 2024;28(11) View
  10. Ramgopal S, Varma S, Gorski J, Kester K, Shieh A, Suresh S. Evaluation of a Large Language Model on the American Academy of Pediatrics' PREP Emergency Medicine Question Bank. Pediatric Emergency Care 2024 View