Published on in Vol 9 (2023)

Preprints (earlier versions) of this paper are available at, first published .
Putting ChatGPT’s Medical Advice to the (Turing) Test: Survey Study

Putting ChatGPT’s Medical Advice to the (Turing) Test: Survey Study

Putting ChatGPT’s Medical Advice to the (Turing) Test: Survey Study

Authors of this article:

Oded Nov1 Author Orcid Image ;   Nina Singh2 Author Orcid Image ;   Devin Mann2, 3 Author Orcid Image


  1. Sallam M, Salim N, Barakat M, Al-Mahzoum K, Al-Tammemi A, Malaeb D, Hallit R, Hallit S. Assessing Health Students' Attitudes and Usage of ChatGPT in Jordan: Validation Study. JMIR Medical Education 2023;9:e48254 View
  2. Osipov M. On the question of the specifics of the formulation and use of the Turing test for the ChatGPT. Программные системы и вычислительные методы 2023;(4):1 View
  3. Turner J. Triangle of Trust in Cancer Care? The Physician, the Patient, and Artificial Intelligence Chatbot. Cancer Biotherapy and Radiopharmaceuticals 2023;38(9):581 View
  4. Erren T. Patients, Doctors, and Chatbots. JMIR Medical Education 2024;10:e50869 View
  5. Howe P, Fay N, Saletta M, Hovy E. ChatGPT’s advice is perceived as better than that of professional advice columnists. Frontiers in Psychology 2023;14 View
  6. Blease C. Open AI meets open notes: surveillance capitalism, patient privacy and online record access. Journal of Medical Ethics 2024;50(2):84 View
  7. Kerr W, McFarlane K. Machine Learning and Artificial Intelligence Applications to Epilepsy: a Review for the Practicing Epileptologist. Current Neurology and Neuroscience Reports 2023;23(12):869 View
  8. Scquizzato T, Semeraro F, Swindell P, Simpson R, Angelini M, Gazzato A, Sajjad U, Bignami E, Landoni G, Keeble T, Mion M. Testing ChatGPT ability to answer laypeople questions about cardiac arrest and cardiopulmonary resuscitation. Resuscitation 2024;194:110077 View
  9. Alsadhan A, Al-Anezi F, Almohanna A, Alnaim N, Alzahrani H, Shinawi R, AboAlsamh H, Bakhshwain A, Alenazy M, Arif W, Alyousef S, Alhamidi S, Alghamdi A, AlShrayfi N, Rubaian N, Alanzi T, AlSahli A, Alturki R, Herzallah N. The opportunities and challenges of adopting ChatGPT in medical research. Frontiers in Medicine 2023;10 View
  10. Younis H, Eisa T, Nasser M, Sahib T, Noor A, Alyasiri O, Salisu S, Hayder I, Younis H. A Systematic Review and Meta-Analysis of Artificial Intelligence Tools in Medicine and Healthcare: Applications, Considerations, Limitations, Motivation and Challenges. Diagnostics 2024;14(1):109 View
  11. Hillmann H, Angelini E, Karfoul N, Feickert S, Mueller-Leisse J, Duncker D. Accuracy and comprehensibility of chat-based artificial intelligence for patient information on atrial fibrillation and cardiac implantable electronic devices. Europace 2023;26(1) View
  12. Tailor P, Dalvin L, Chen J, Iezzi R, Olsen T, Scruggs B, Barkmeier A, Bakri S, Ryan E, Tang P, Parke D, Belin P, Sridhar J, Xu D, Kuriyan A, Yonekawa Y, Starr M. A Comparative Study of Responses to Retina Questions from Either Experts, Expert-Edited Large Language Models, or Expert-Edited Large Language Models Alone. Ophthalmology Science 2024;4(4):100485 View
  13. Meyer A, Riese J, Streichert T. Comparison of the Performance of GPT-3.5 and GPT-4 With That of Medical Students on the Written German Medical Licensing Examination: Observational Study. JMIR Medical Education 2024;10:e50965 View
  14. Ata A, Aras B, Yılmaz Taşdelen Ö, Çelik C, Çulha C. Evaluation of Informative Content on Cerebral Palsy in the Era of Artificial Intelligence: The Value of ChatGPT. Physical & Occupational Therapy In Pediatrics 2024:1 View
  15. Mandal S, Wiesenfeld B, Mann D, Szerencsy A, Iturrate E, Nov O. Quantifying the impact of telemedicine and patient medical advice request messages on physicians' work-outside-work. npj Digital Medicine 2024;7(1) View
  16. Wong L, Park H, Looi C. From hype to insight: Exploring ChatGPT's early footprint in education via altmetrics and bibliometrics. Journal of Computer Assisted Learning 2024 View
  17. Figari Jordan R, Sandrone S, Southerland A. Opportunities and Challenges for Incorporating Artificial Intelligence and Natural Language Processing in Neurology Education. Neurology Education 2024;3(1) View
  18. Mubin O, Alnajjar F, Trabelsi Z, Ali L, Parambil M, Zou Z. Tracking ChatGPT Research: Insights from the literature and the web. IEEE Access 2024:1 View
  19. Le M, Davis M. ChatGPT Yields a Passing Score on a Pediatric Board Preparatory Exam but Raises Red Flags. Global Pediatric Health 2024;11 View
  20. Zampatti S, Peconi C, Megalizzi D, Calvino G, Trastulli G, Cascella R, Strafella C, Caltagirone C, Giardina E. Innovations in Medicine: Exploring ChatGPT’s Impact on Rare Disorder Management. Genes 2024;15(4):421 View
  21. Tailor P, Dalvin L, Starr M, Tajfirouz D, Chodnicki K, Brodsky M, Mansukhani S, Moss H, Lai K, Ko M, Mackay D, Di Nome M, Dumitrascu O, Pless M, Eggenberger E, Chen J. A Comparative Study of Large Language Models, Human Experts, and Expert-Edited Large Language Models to Neuro-Ophthalmology Questions. Journal of Neuro-Ophthalmology 2024 View
  22. Moise A, Centomo-Bozzo A, Orishchak O, Alnoury M, Daniel S. Can ChatGPT Replace an Otolaryngologist in Guiding Parents on Tonsillectomy?. Ear, Nose & Throat Journal 2024 View
  23. Howard E, Chong N, Carnino J, Levi J. Comparison of ChatGPT knowledge against 2020 consensus statement on ankyloglossia in children. International Journal of Pediatric Otorhinolaryngology 2024;180:111957 View
  24. Aharoni E, Fernandes S, Brady D, Alexander C, Criner M, Queen K, Rando J, Nahmias E, Crespo V. Attributions toward artificial agents in a modified Moral Turing Test. Scientific Reports 2024;14(1) View
  25. Connors C, Gupta K, Khusid J, Khargi R, Yaghoubian A, Levy M, Gallante B, Atallah W, Gupta M. Evaluation of the Current Status of Artificial Intelligence for Endourology Patient Education: A Blind Comparison of ChatGPT and Google Bard Against Traditional Information Resources. Journal of Endourology 2024 View
  26. Zhang L, Shu J, Hu J, Li F, He J, Wang P, Shen Y. Exploring the Potential of Large Language Models in Radiological Imaging Systems: Improving User Interface Design and Functional Capabilities. Electronics 2024;13(11):2002 View
  27. Martynov A, Bechrakis N, Lever M. Das Aderhautmelanom im Zeitalter der generativen künstlichen Intelligenz – im Gespräch mit ChatGPT. Klinische Monatsblätter für Augenheilkunde 2024 View
  28. Meyer A, Soleman A, Riese J, Streichert T. Comparison of ChatGPT, Gemini, and Le Chat with physician interpretations of medical laboratory questions from an online health forum. Clinical Chemistry and Laboratory Medicine (CCLM) 2024;0(0) View