Accessibility settings

This paper is in the following e-collection/theme issue:

Graduate and Postgraduate Education for Health Professionals (261) Testing and Assessment in Medical Education (240) Artificial Intelligence (5021) Artificial Intelligence (AI) in Medical Education (859) Generative Language Models Including ChatGPT (1635) Applications of AI (1092)

Published on in Vol 11 (2025)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/58375, first published .
Hand filling in bubbles on a standardized test answer sheet with a white pencil.

Performance of Plug-In Augmented ChatGPT and Its Ability to Quantify Uncertainty: Simulation Study on the German Medical Board Examination

Performance of Plug-In Augmented ChatGPT and Its Ability to Quantify Uncertainty: Simulation Study on the German Medical Board Examination

Authors of this article:

Julian Madrid1 Author Orcid Image ;   Philipp Diehl1 Author Orcid Image ;   Mischa Selig2, 3 Author Orcid Image ;   Bernd Rolauffs2, 3 Author Orcid Image ;   Felix Patricius Hans2, 4 Author Orcid Image ;   Hans-Jörg Busch2, 4 Author Orcid Image ;   Tobias Scheef2, 5 Author Orcid Image ;   Leo Benning2, 4 Author Orcid Image
Article Authors Cited by (13) Tweetations Metrics

Journals

  1. Wang R, Ding Y, Shen Y, Liu H, Wang P, Gao Z. Comparative Evaluation of Teaching Plans on Prostate Cancer Generated by Various Large Language Models and a Human Expert. Engineering Reports 2025;7(8) View
  2. Kasagga A, Sapkota A, Changaramkumarath G, Abucha J, Wollel M, Somannagari N, Husami M, Hailu K, Kasagga E. Performance of ChatGPT and Large Language Models on Medical Licensing Exams Worldwide: A Systematic Review and Network Meta-Analysis With Meta-Regression. Cureus 2025 View
  3. Shao M, Zhang H. Two-stage prompting framework with predefined verification steps for evaluating diagnostic reasoning tasks on two datasets. npj Digital Medicine 2025;8(1) View
  4. Li J, Ai F, Wang J, Cheng B, Li Y, Chen Z. Application of AI-Generated Content in Medical Education: Systematic Review of the Impact on Critical Thinking Abilities of Medical Students. JMIR Medical Education 2026;12:e79939 View
  5. Koç A, Ataş A, Yosunkaya Ş, Vatansev H. Performance of large language models on sleep medicine certification examination: a comprehensive multi-model analysis. Frontiers in Medicine 2026;13 View
  6. Zhou Y, Lin S, Wang X, Hu K. Knowledge localization is associated with higher performance of domestic large language models in a Chinese radiation oncology examination. Frontiers in Oncology 2026;16 View
  7. Cheng L, Bai J, Liu A. Early identification of risk factors for obstructive sleep apnea hypopnea syndrome based on large language models. Frontiers in Medicine 2026;13 View
  8. Hier D, Obafemi-Ajayi T. Reframing ontology fact acquisition during large language model fine-tuning as a time-to-event process. Frontiers in Artificial Intelligence 2026;9 View
  9. Ugrinowitsch C, Lamas L, Libardi C. Reasoning under uncertainty in graduate health education: a scaffolded framework using large language models. Frontiers in Education 2026;11 View
  10. Baskaran R, Sirikonda S, Singh A, Srinivasan S, Leveridge B, Casals-Farre O, Kaur H, Manivannan S, Elangovan K, Quek C, Fukutsu K, Cave J, Somani B, Ting D, Hassoulas A. The temporal changes in GPT-4 performance on UKMLA practice questions: educational and clinical implications. Frontiers in Medicine 2026;13 View
  11. Li Q, Li Y, Ye C, Zhang K, Bai J, Liu X, Li Z, Meng X, Min X, Guo J. Decoding high-order clinical correlations: a knowledge-driven large language model framework for specialized medical decision-making. Frontiers in Artificial Intelligence 2026;9 View
  12. Cirkel L, Knitza J, Schillings V, Oksche A, Becker J, Kuhn S. Evaluating foundation models on official German medical licensing examinations: Implications for high-stakes assessment and AI-assisted medical education. npj Digital Medicine 2026;9(1) View
  13. Benning L, Haar M, Haftenberger A, Antweiler D, Steidle O, Möller R, Schiff R, Wolfrum S, Hartwig M, Seitz M, Widhalm G, Wingen S, Busch H, Michels G. Digitalisierung und Künstliche Intelligenz in der Intensiv- und Notfallmedizin. Medizinische Klinik - Intensivmedizin und Notfallmedizin 2026;121(7):574 View