Test-Retest Reliability of Large Language Models and Their Concordance With Human Experts in Rubric-Based Scoring of Clinical Summary Statements: Experimental Comparative Study

Patrick Stadler   ¹ , MSc ;   Piotr Bucichowski   ¹ , MD ;   Ylva Holzhausen   ¹ , Dr phil ;   Inga Hege   ² , MD, MCompSc ;   Harm Peters   ¹ , MD, MHPE ;   Julius Josef Kaminski   ¹ , MD, MHPE

¹ Charité – University Medical Center Berlin, Dieter Scheffner Center for Medical Education and Educational Research, Dean's Office for Study Affairs, Berlin, Germany

² Faculty of Health Sciences Brandenburg, Institute of Research in Health Sciences Education, Brandenburg Medical School, Neuruppin, Germany