This paper is in the following e-collection/theme issue:

Theme Issue: ChatGPT and Generative Language Models in Medical Education (86) Natural Language Processing (706) New Methods and Approaches in Medical Education (245)

Published on in Vol 9 (2023)

This is a member publication of Imperial College London (Jisc)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/47737, first published .
Performance of ChatGPT on UK Standardized Admission Tests: Insights From the BMAT, TMUA, LNAT, and TSA Examinations

Performance of ChatGPT on UK Standardized Admission Tests: Insights From the BMAT, TMUA, LNAT, and TSA Examinations

Performance of ChatGPT on UK Standardized Admission Tests: Insights From the BMAT, TMUA, LNAT, and TSA Examinations

Authors of this article:

Panagiotis Giannos1, 2 Author Orcid Image ;   Orestis Delardas2 Author Orcid Image
Article Authors Cited by (52) Tweetations (3) Metrics

Journals

  1. Oztermeli A, Oztermeli A. ChatGPT performance in the medical specialty exam: An observational study. Medicine 2023;102(32):e34673 View
  2. Sallam M, Salim N, Barakat M, Al-Mahzoum K, Al-Tammemi A, Malaeb D, Hallit R, Hallit S. Assessing Health Students' Attitudes and Usage of ChatGPT in Jordan: Validation Study. JMIR Medical Education 2023;9:e48254 View
  3. Giannos P. Evaluating the limits of AI in medical specialisation: ChatGPT’s performance on the UK Neurology Specialty Certificate Examination. BMJ Neurology Open 2023;5(1):e000451 View
  4. Miao J, Thongprayoon C, Garcia Valencia O, Krisanapan P, Sheikh M, Davis P, Mekraksakit P, Suarez M, Craici I, Cheungpasitporn W. Performance of ChatGPT on Nephrology Test Questions. Clinical Journal of the American Society of Nephrology 2024;19(1):35 View
  5. Gencer A, Aydin S. Can ChatGPT pass the thoracic surgery exam?. The American Journal of the Medical Sciences 2023;366(4):291 View
  6. Levin G, Horesh N, Brezinov Y, Meyer R. Performance of ChatGPT in medical examinations: A systematic review and a meta‐analysis. BJOG: An International Journal of Obstetrics & Gynaecology 2024;131(3):378 View
  7. Cohen A, Alter R, Lessans N, Meyer R, Brezinov Y, Levin G. Performance of ChatGPT in Israeli Hebrew OBGYN national residency examinations. Archives of Gynecology and Obstetrics 2023;308(6):1797 View
  8. Borchert R, Hickman C, Pepys J, Sadler T. Performance of ChatGPT on the Situational Judgement Test—A Professional Dilemmas–Based Examination for Doctors in the United Kingdom. JMIR Medical Education 2023;9:e48978 View
  9. Abi-Rafeh J, Xu H, Kazan R, Tevlin R, Furnas H. Large Language Models and Artificial Intelligence: A Primer for Plastic Surgeons on the Demonstrated and Potential Applications, Promises, and Limitations of ChatGPT. Aesthetic Surgery Journal 2024;44(3):329 View
  10. Shang L, Xue M, Hou Y, Tang B. Can ChatGPT pass China's national medical licensing examination?. Asian Journal of Surgery 2023;46(12):6112 View
  11. Pushpanathan K, Lim Z, Er Yew S, Chen D, Hui'En Lin H, Lin Goh J, Wong W, Wang X, Jin Tan M, Chang Koh V, Tham Y. Popular large language model chatbots’ accuracy, comprehensiveness, and self-awareness in answering ocular symptom queries. iScience 2023;26(11):108163 View
  12. Lai U, Wu K, Hsu T, Kan J. Evaluating the performance of ChatGPT-4 on the United Kingdom Medical Licensing Assessment. Frontiers in Medicine 2023;10 View
  13. Büttner M, Leser U, Schneider L, Schwendicke F. Natural Language Processing: Chances and Challenges in Dentistry. Journal of Dentistry 2024;141:104796 View
  14. Ahimaz P, Bergner A, Florido M, Harkavy N, Bhattacharyya S. Genetic counselors' utilization of ChatGPT in professional practice: A cross‐sectional study. American Journal of Medical Genetics Part A 2024;194(4) View
  15. Madrid-García A, Rosales-Rosado Z, Freites-Nuñez D, Pérez-Sancristóbal I, Pato-Cour E, Plasencia-Rodríguez C, Cabeza-Osorio L, Abasolo-Alcázar L, León-Mateos L, Fernández-Gutiérrez B, Rodríguez-Rodríguez L. Harnessing ChatGPT and GPT-4 for evaluating the rheumatology questions of the Spanish access exam to specialized medical training. Scientific Reports 2023;13(1) View
  16. Sallam M, Al-Salahat K. Below average ChatGPT performance in medical microbiology exam compared to university students. Frontiers in Education 2023;8 View
  17. Miao J, Thongprayoon C, Suppadungsuk S, Garcia Valencia O, Qureshi F, Cheungpasitporn W. Ethical Dilemmas in Using AI for Academic Writing and an Example Framework for Peer Review in Nephrology Academia: A Narrative Review. Clinics and Practice 2023;14(1):89 View
  18. Zheltukhina M, Sergeeva O, Masalimova A, Budkevich R, Kosarenko N, Nesterov G. A bibliometric analysis of publications on ChatGPT in education: Research patterns and topics. Online Journal of Communication and Media Technologies 2024;14(1):e202405 View
  19. Davies N, Wilson R, Winder M, Tunster S, McVicar K, Thakrar S, Williams J, Reid A. ChatGPT sits the DFPH exam: large language model performance and potential to support public health learning. BMC Medical Education 2024;24(1) View
  20. Sallam M, Barakat M, Sallam M. A Preliminary Checklist (METRICS) to Standardize the Design and Reporting of Studies on Generative Artificial Intelligence–Based Models in Health Care Education and Practice: Development Study Involving a Literature Review. Interactive Journal of Medical Research 2024;13:e54704 View
  21. Roemer G, Li A, Mahmood U, Dauer L, Bellamy M. Artificial intelligence model GPT4 narrowly fails simulated radiological protection exam. Journal of Radiological Protection 2024;44(1):013502 View
  22. Arslan B, Eyupoglu G, Korkut S, Turkdogan K, Altinbilek E. The accuracy of AI-assisted chatbots on the annual assessment test for emergency medicine residents. Journal of Medicine, Surgery, and Public Health 2024;3:100070 View
  23. Mai D, Da C, Hanh N. The use of ChatGPT in teaching and learning: a systematic review through SWOT analysis approach. Frontiers in Education 2024;9 View
  24. Su M, Lin L, Lin L, Chen Y. Assessing question characteristic influences on ChatGPT's performance and response-explanation consistency: Insights from Taiwan's Nursing Licensing Exam. International Journal of Nursing Studies 2024;153:104717 View
  25. Bhattacharya M, Pal S, Chatterjee S, Alshammari A, Albekairi T, Jagga S, Ige Ohimain E, Zayed H, Byrareddy S, Lee S, Wen Z, Agoramoorthy G, Bhattacharya P, Chakraborty C. ChatGPT’s scorecard after the performance in a series of tests conducted at the multi-country level: A pattern of responses of generative artificial intelligence or large language models. Current Research in Biotechnology 2024;7:100194 View
  26. Wei Q, Yao Z, Cui Y, Wei B, Jin Z, Xu X. Evaluation of ChatGPT-generated medical responses: A systematic review and meta-analysis. Journal of Biomedical Informatics 2024;151:104620 View
  27. Raitskaya L, Lambovska M. Prospects for ChatGPT Application in Higher Education: A Scoping Review of International Research. Integration of Education 2023;28(1):10 View
  28. Wang S, Mo C, Chen Y, Dai X, Wang H, Shen X. Exploring the Performance of ChatGPT-4 in the Taiwan Audiologist Qualification Examination: Preliminary Observational Study Highlighting the Potential of AI Chatbots in Hearing Care. JMIR Medical Education 2024;10:e55595 View
  29. Rosenberg G, Magnéli M, Barle N, Kontakis M, Müller A, Wittauer M, Gordon M, Brodén C. ChatGPT-4 generates orthopedic discharge documents faster than humans maintaining comparable quality: a pilot study of 6 cases. Acta Orthopaedica 2024;95:152 View
  30. Zhu L, Mou W, Hong C, Yang T, Lai Y, Qi C, Lin A, Zhang J, Luo P. The Evaluation of Generative AI Should Include Repetition to Assess Stability. JMIR mHealth and uHealth 2024;12:e57978 View
  31. Sawamura S, Bito T, Ando T, Masuda K, Kameyama S, Ishida H. Evaluation of the accuracy of ChatGPT’s responses to and references for clinical questions in physical therapy. Journal of Physical Therapy Science 2024;36(5):234 View
  32. Cong-Lem N, Soyoof A, Tsering D. A Systematic Review of the Limitations and Associated Opportunities of ChatGPT. International Journal of Human–Computer Interaction 2024:1 View
  33. Mcintosh T, Liu T, Susnjak T, Watters P, Halgamuge M. A Reasoning and Value Alignment Test to Assess Advanced GPT Reasoning. ACM Transactions on Interactive Intelligent Systems 2024;14(3):1 View
  34. Pregowska A, Perkins M. Artificial intelligence in medical education: Typologies and ethical approaches. Ethics & Bioethics 2024;14(1-2):96 View
  35. Ülkü A. The performance of artificial intelligence in the exams of tourist guidance. Journal of Multidisciplinary Academic Tourism 2024 View
  36. Rossettini G, Rodeghiero L, Corradi F, Cook C, Pillastrini P, Turolla A, Castellini G, Chiappinotto S, Gianola S, Palese A. Comparative accuracy of ChatGPT-4, Microsoft Copilot and Google Gemini in the Italian entrance test for healthcare sciences degrees: a cross-sectional study. BMC Medical Education 2024;24(1) View
  37. Kim H, Yang J, Chang D, Lenke L, Pizones J, Castelein R, Watanabe K, Trobisch P, Mundis Jr G, Suh S, Suk S. Assessing the Reproducibility of the Structured Abstracts Generated by ChatGPT and Bard Compared to Human-Written Abstracts in the Field of Spine Surgery: Comparative Analysis. Journal of Medical Internet Research 2024;26:e52001 View
  38. Miao Y, Luo Y, Zhao Y, Li J, Liu M, Wang H, Chen Y, Wu Y. Performance of GPT-4 on Chinese Nursing Examination. Nurse Educator 2024;49(6):E338 View
  39. Frenkel M, Emara H. ChatGPT‐3.5 and ‐4.0 and mechanical engineering: Examining performance on the FE mechanical engineering and undergraduate exams. Computer Applications in Engineering Education 2024;32(6) View
  40. Moglia A, Georgiou K, Cerveri P, Mainardi L, Satava R, Cuschieri A. Large language models in healthcare: from a systematic review on medical examinations to a comparative analysis on fundamentals of robotic surgery online test. Artificial Intelligence Review 2024;57(9) View
  41. Sadeq M, Ghorab R, Ashry M, Abozaid A, Banihani H, Salem M, Aisheh M, Abuzahra S, Mourid M, Assker M, Ayyad M, Moawad M. AI chatbots show promise but limitations on UK medical exam questions: a comparative performance study. Scientific Reports 2024;14(1) View
  42. Rehana H, Çam N, Basmaci M, Zheng J, Jemiyo C, He Y, Özgür A, Hur J, Lengauer T. Evaluating GPT and BERT models for protein–protein interaction identification in biomedical text. Bioinformatics Advances 2024;4(1) View
  43. Waldock W, Zhang J, Guni A, Nabeel A, Darzi A, Ashrafian H. The Accuracy and Capability of Artificial Intelligence Solutions in Health Care Examinations and Certificates: Systematic Review and Meta-Analysis. Journal of Medical Internet Research 2024;26:e56532 View
  44. Zhou H, Wang H, Duan Y, Yan Z, Luo R, Lv X, Xie Y, Zhang J, Yang J, Xue M, Fang Y, Lu L, Liu P, Ye Z. Enhancing Orthopedic Knowledge Assessments: The Performance of Specialized Generative Language Model Optimization. Current Medical Science 2024;44(5):1001 View
  45. Ros-Arlanzón P, Perez-Sempere A. Evaluating AI Competence in Specialized Medicine: Comparative Analysis of ChatGPT and Neurologists in a Neurology Specialist Examination in Spain. JMIR Medical Education 2024;10:e56762 View
  46. Giunti M, Garavaglia F, Giuntini R, Sergioli G, Pinna S, Khodi A. ChatGPT as a prospective undergraduate and medical school student. PLOS ONE 2024;19(10):e0308157 View
  47. Nedungadi P, Tang K, Raman R. The Transformative Power of Generative Artificial Intelligence for Achieving the Sustainable Development Goal of Quality Education. Sustainability 2024;16(22):9779 View
  48. Chen Y, Huang X, Yang F, Lin H, Lin H, Zheng Z, Liang Q, Zhang J, Li X. Performance of ChatGPT and Bard on the medical licensing examinations varies across different cultures: a comparison study. BMC Medical Education 2024;24(1) View
  49. Alahmadi M, Alharbi M, Tayeb A, Alshangiti M. Evaluating Large Language Models' Proficiency in Answering Arabic GAT Exam Questions. Engineering, Technology & Applied Science Research 2024;14(6):17774 View

Books/Policy Documents

  1. Htet A, Liana S, Aung T, Bhaumik A. Advanced Applications of Generative AI and Natural Language Processing Models. View