Letter to the Editor
See also: https://mededu.jmir.org/2023/1/e46885
ChatGPT has recently been shown to pass the United States Medical Licensing Examination (USMLE). We tested ChatGPT (Feb 13, 2023 release) using a typical clinical toxicology case of acute organophosphate poisoning. ChatGPT fared well in answering all of our queries regarding it.JMIR Med Educ 2023;9:e46876
Since its public launch on November 30, 2022, ChatGPT, which ironically has not been specifically trained in medicine, has been taking the medical world by storm [- ]. Developed by the San Francisco–based OpenAI Inc/LP, ChatGPT is a very large language model that uses deep learning artificial intelligence (AI) techniques to generate human-like responses to natural language queries. It is based on the Generative Pre-trained Transformer 3 (GPT-3 x) architecture, which has been trained on gigantic amounts of data. ChatGPT is currently being integrated into the Microsoft Bing search engine, which will soon make it readily accessible to hundreds of millions of online users worldwide, including patients, medical and nursing students, and clinicians [ ].
We tested ChatGPT (Feb 13, 2023, release; standalone, available via OpenAI ) using a typical clinical toxicology vignette (a case of acute organophosphate poisoning) retrieved from an online presentation [ ]. The case, as we modified it for and typed it in ChatGPT, as well as ChatGPT’s answer (which followed only a couple of seconds after inputting the case), are shown in . The figure also shows a regenerated ChatGPT response after we pressed the “Regenerate response” button at the bottom of the first ChatGPT answer.
The clinical case example we used is a very straightforward one, unlikely to be missed by any practitioner in the field, and ChatGPT fared well in answering all of our queries regarding it. Both the first ChatGPT response and the regenerated one were fine and offered good explanations of the underlying reasoning. However, the pressing problem in real life is not one of finding the correct diagnosis but of taking appropriate history and being able to elicit and ascertain the correct signs. In real life, junior clinicians may arrive at the wrong diagnosis because they missed or confused the signs. As ChatGPT becomes further developed and specifically adapted for medicine, it could one day be useful in less common clinical cases (ie, cases that experts sometimes miss). Rather than AI replacing humans (clinicians), we see it as “clinicians using AI” replacing “clinicians who do not use AI” in the coming years.
Conflicts of Interest
- Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, et al. How does ChatGPT perform on the United States Medical Licensing Examination? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ 2023 Feb 08;9:e45312 [FREE Full text] [CrossRef] [Medline]
- Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health 2023 Feb 9;2(2):e0000198 [FREE Full text] [CrossRef] [Medline]
- Hailu R, Beam A, Mehrotra A. ChatGPT-assisted diagnosis: is the future suddenly here? STAT. 2023 Feb 13. URL: https://www.statnews.com/2023/02/13/chatgpt-assisted-diagnosis/ [accessed 2023-03-06]
- Mehdi Y. Reinventing search with a new AI-powered Microsoft Bing and Edge, your copilot for the web. Official Microsoft Blog. 2023 Feb 07. URL: https://blogs.microsoft.com/blog/2023/02/07/reinventing-search-with-a-new-ai-powered-microsoft-bing-and-edge-your-copilot-for-the-web/ [accessed 2023-03-06]
- ChatGPT (Feb 13, 2023, release). OpenAI. URL: https://chat.openai.com/ [accessed 2023-03-06]
- Jiranantakan T. Toxicology case studies: medical preparedness & responses for chemical and radiological emergencies (slide 11). Mahidol University. 2018 Jan 21. URL: http://envocc.ddc.moph.go.th/uploads/OEHA2/ELM/Module%204/3.2%206P_Handout_MOPH-ExecutiveLevel_ToxicologyandOEM_2018_Thanjira_Jiranantakan.pdf [accessed 2023-03-06]
|AI: artificial intelligence|
|GPT-3 x: Generative Pre-trained Transformer 3|
Edited by G Eysenbach; This is a non–peer-reviewed article. submitted 28.02.23; accepted 03.03.23; published 08.03.23Copyright
©Mary Sabry Abdel-Messih, Maged N Kamel Boulos. Originally published in JMIR Medical Education (https://mededu.jmir.org), 08.03.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on https://mededu.jmir.org/, as well as this copyright and license information must be included.