Pure Wisdom or Potemkin Villages? A Comparison of ChatGPT 3.5 and ChatGPT 4 on USMLE Step 3 Style Questions: Quantitative Analysis

Leonard Knoedler   ¹ , MD ;   Michael Alfertshofer   ² , MD ;   Samuel Knoedler   ^{1, 3} , BSc ;   Cosima C Hoch   ⁴ , BSc ;   Paul F Funk   ⁵ , BSc ;   Sebastian Cotofana   ^{6, 7} , MD, PhD ;   Bhagvat Maheta   ⁸ , BS ;   Konstantin Frank   ⁹ , MD ;   Vanessa Brébant   ¹ , MD ;   Lukas Prantl   ¹ , MD, PhD ;   Philipp Lamby   ¹ , MD, PhD

¹ Department of Plastic, Hand and Reconstructive Surgery, University Hospital Regensburg, Regensburg, Germany

² Division of Hand, Plastic and Aesthetic Surgery, Ludwig-Maximilians University Munich, Munich, Germany

³ Division of Plastic Surgery, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, United States

⁴ Department of Otolaryngology, Head and Neck Surgery, School of Medicine, Technical University of Munich, Munich, Germany

⁵ Department of Otolaryngology, Head and Neck Surgery, University Hospital Jena, Friedrich Schiller University Jena, Jena, Germany

⁶ Department of Dermatology, Erasmus Hospital, Rotterdam, Netherlands

⁷ Centre for Cutaneous Research, Blizard Institute, Queen Mary University of London, London, United Kingdom

⁸ College of Medicine, California Northstate University, Elk Grove, CA, United States

⁹ Ocean Clinic, Marbella, Spain