Performance Evaluation of Large Language Models in Multilingual Medical Multiple-Choice Questions: Mixed Methods Study

Livia Maria Strasser   ^{1 *} , BMed, MMed ;   Wilma Anschuetz   ^{2 *} , MD ;   Fabio Dennstädt   ^{1, 3} , MD ;   Janna Hastings   ^{1, 4, 5} , BSc, MSc, MA, PhD

¹ Medical Knowledge and Decision Support, School of Medicine, University of St.Gallen, St.Gallen, Switzerland

² Department for Assessment and Evaluation, Institute for Medical Education, University of Bern, Bern, Switzerland

³ Department of Radiation Oncology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland

⁴ Institute for Implementation Science in Health Care, University of Zurich, Zurich, Switzerland

⁵ Idiap Research Institute, Martigny, Switzerland

*these authors contributed equally