The protocol for this review was developed in accordance with the Joanna Briggs Institute (JBI) Reviewers Manual for Evidence Synthesis [34] and guided by the methodological framework developed by Arksey and O’Malley [35], supplemented by Levac et al [36]. The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) [37] was used when reporting results, and is reported in Multimedia Appendix 1. The protocol was registered on Open Science Framework Registries and published on JBI Evidence Synthesis [38].

All types of studies were included, such as theoretical work, program descriptions, and empirical studies. Commentaries, reviews, perspectives, opinions, as well as position papers and any companion papers associated were also included. All study designs for empirical studies using qualitative, quantitative, or mixed methods studies were eligible for inclusion. These include experimental and quasi-experimental studies (such as randomized controlled trials, quasi-randomized controlled trials, nonrandomized clinical trials, interrupted time series, and controlled before-and-after studies), observational studies (such as cohort, case control, cross-sectional, and case series studies), qualitative studies (such as ethnography, narrative, phenomenological, grounded theory, and case studies), and mixed methods studies. Conference abstracts and protocols were excluded. Conference abstracts often contain preliminary findings that may not be as comprehensive or validated as full-text articles. As they are brief summaries of studies, they may lack the detailed methodology and results needed for a thorough understanding and synthesis in our scoping review. Furthermore, as protocols are plans of how to conduct the research, they do not provide findings or results that are necessary for a scoping review’s goal to map the extent, range, and nature of research activity in a given field. Therefore, considering the provided justifications, we decided to exclude conference abstracts and protocols.

The following search strategy has been developed by a specialized librarian. The text words contained in the titles and abstracts of relevant papers and the index terms used to describe the papers were used to develop a full search strategy. The search strategy took an iterative approach, initially using general terms such as “artificial intelligence,” with the later addition of variations and synonyms such as “deep learning” and “machine learning.” In addition, terms for the concepts of medical education and curriculum were added. An initial limited search of MEDLINE (PubMed) was conducted to identify relevant papers on this topic. An information specialist (GG) performed a comprehensive search in the following bibliographic databases: Ovid MEDLINE, Ovid Embase, CENTRAL (Cochrane Library), CINAHL, and Scopus. To identify any unpublished frameworks, web searches of Google, New York Academy of Medicine Grey Literature Report, and medical learning institutional websites were searched. Reference lists of all included research papers and all relevant reviews were back searched, and Google Scholar was used for forward citation tracking to identify further studies.

Papers were restricted to English and French due to the constraints of the research team. Papers were also restricted by date beginning in the year of 2000, as during the 1950s to the late 1990s AI was in its early phase with reduced funding and interest of AI in medicine [40]. The initial search was conducted in November 2021 and later updated in May 2023.

Following the search, all identified records were collated and uploaded into a reference management system, EndNote (version 20.3; Clarivate Analytics), where duplicates were removed. Following a pilot test with 2 reviewers (RT and AB) using 10% (510/5104) of the studies, titles and abstracts were then screened using Rayyan, a web-based research platform, by 2 independent reviewers (RT and AB) for assessment against the inclusion criteria for the review. The full text of selected citations was assessed in detail against the inclusion criteria by 2 independent reviewers (RT and AB). Any disagreements that arose between the 2 reviewers were resolved by a third reviewer (SAR).

Data were extracted by 2 reviewers (RT and AB) using a data extraction tool on an Excel (Microsoft Corp) sheet developed and validated by the team. The data extraction tool was created and validated using previously validated data extraction tools [32-34] and input from experts in the field. It focuses on key characteristics related to curriculum framework elements and educational program details. Any disagreements that arose between the 2 reviewers were resolved by a third reviewer (SAR). Data on paper characteristics (eg, authors, title, country of origin, type of study, and year of publication), curriculum framework elements, and educational program details were extracted.

The results of the review are presented as a table of the data extracted from the included literature to highlight the key findings with respect to the aims of this scoping review. Descriptive statistics (eg, frequency) was used when reporting paper characteristics and education program details. For curriculum frameworks described, reviewers presented main elements, including (1) the need and purpose of curriculum, (2) the learning objectives and outcomes, (3) course content that will facilitate the accomplishment of the objectives or learning outcomes, (4) the organization of the content, and (5) implementation of curriculum. For current educational programs described, reviewers independently recorded and presented data on the framework, theory, or pedagogy that may have been used; the delivery of the educational program; and curricular content; and if the educational program was evaluated, it was described according to the model of training evaluation developed by Kirkpatrick et al [39].

The model of training evaluation developed by Kirkpatrick et al [39] was used to categorize educational outcomes evaluations (Figure 1 [39]). Level 1 describes the degree to which learners find the training favorable, engaging, and relevant; level 2 describes the degree to which learners acquire the intended knowledge, skills, confidence, and commitment based on their participation in the training; level 3 describes the degree to which learners apply what they learned during training when they are back to work; and level 4 describes the degree in whether the targeted outcomes resulted from the training program at an organizational level [39]. A narrative summary accompanied [41] the charted results and described what and how AI curriculum content is being delivered to trainees of various medical education stages.

Due to the nature of this review, the methodological quality or risk of bias of the included papers was not appraised, which is consistent with scoping reviews guidelines [34,37].

From the systematic search, 5076 total papers were identified. These papers were extracted from web-based databases, and the computer software EndNote was used to manage these references. Following removal of duplicates on EndNote, 2458 papers were uploaded to Rayyan software and screened by title and abstract. After abstract and title screening, 60 papers remained for full-text screening. A gray literature search identified 60 papers from Google Scholar and reference lists, from which 28 (47%) papers were retrieved for full-text screening, and 32 (53%) papers were not retrieved or were irrelevant. Following full-text screening of databases and gray literature, 21 papers were included for further analysis [12,25,31-33,42-57]. Refer to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram (Figure 2) [58].

Data was collected from 21 included studies and summarized in Multimedia Appendix 2 [12,25,31-33,42-57]. A total of 12 studies were published in the United States [12,31,32,42-45,48,51,52,54,57]; 6 in Canada [33,46,47,49,50,55]; and 1 each in Germany [25], Korea [53], and Oman [56] (Multimedia Appendix 3). The earliest publication retrieved was from 2016, with 77% (15/21) of the papers [25,31-33,42,43,45,47,49-51,54-57] published in the last 3 years since the pandemic began (Multimedia Appendix 3). From the 21 studies, 6 (29%) were reviews [31-33,45,53,54], 4 (19%) were commentaries [44,47,50,51], 4 (19%) were opinions [12,48,52,56], 3 (14%) were perspectives [43,55,57], 3 (14%) were empirical studies using a cross-sectional survey design [25,42,49], and 1 (5%) was a position paper [46].

In terms of setting, 43% (9/21) of the papers mentioned multiple levels of education ranging from UME, PGME, to CME [12,31-33,46,50,51,54,56], while 24% (5/21) of the papers specified on UME [42,44,47,53,55], 19% (4/21) of the papers specified on PGME [48,49,52,57], and 14% (3/21) of the papers were focused on CME [25,43,45]. Across the 21 included studies, 19 (90%) altogether described 30 current or previously offered educational programs [12,25,31-33,42-55] and 2 (10%) described elements of a curriculum framework [56,57].

Two papers described the main elements of a curriculum framework (Table 1) [56,57]. The first paper was an opinion paper by Masters [56], which mentions 3 of the 5 elements of a curriculum framework. The paper describes the need and purpose of a curriculum, course content, and brief descriptions in terms of organization of content. The second paper to describe elements of a curriculum framework was the study by Valikodath et al [57], which provides information for all 5 elements. This includes the main purpose of an ophthalmology AI curriculum, the learning objectives, course content topics, a 4-year resident organization plan, and implementation of the curriculum, as outlined in Table 1. We noticed similarities in relation to what medical trainees should learn, as emphasized in Figure 3 [56,57].

From our comparisons, we found that the main curricular topics presented by Masters [56] appropriately corresponded to the curricular topics presented by Valikodath et al [57], for example, a main curricular topic of “AI in Medical Systems,” which describes the way in which students should learn the structures and processes of AI systems that they will be using in the future. This corresponds to “Clinical Applications” and “Surgical Applications” in which the content is targeted into learning how to use AI applications for ophthalmology. It appears that Masters’ [56] framework on course content can work as the foundation on what curricular concepts a program should include. This is because previous reviews have detailed similar curricular topics currently being taught.

From the 19 papers that describe an educational program, 30 current or previously offered educational programs were identified (Table 2) [12,25,31-33,42-55]. A total of 13 papers described, mentioned, or presented 24 educational programs [12,31-33,43,45-47,50-54], while 6 papers described and assessed 6 educational programs using evaluation methods (eg, survey and test scores) [25,42,44,48,49,55]. No papers described a theory, pedagogy, or framework that guided the educational program.

Of the 30 educational programs described collectively in the 19 remaining papers, 15 (50%) programs were courses, 4 (13%) were project-related initiatives, 4 (13%) were didactic lectures dedicated to AI, 3 (10%) were webinars, 2 (7%) were an educational summit or conference, and 2 (7%) were 1-day workshops. “AI courses were defined as elective courses focused on AI-based education. Didactic lectures dedicated to AI are 1 or 2 lectures that mention AI education but not a full course. There were 77% (23/30) educational programs delivered from a medical school, while 23% (7/30) were delivered from recognized national or international medical associations. Furthermore, it is important to clarify that some papers used multiple educational program delivery approaches. For example, an included paper explained their educational intervention was a course, but this course included didactic lectures, mentorship, and a final project. However, the reporting of this educational program’s delivery is classified as only a course and not counted as another delivery approach to minimize confusion.

Of the 30 educational programs described collectively in the 19 remaining papers, 17 (57%) UME educational programs were targeted toward medical students. Of these 17 programs, 16 (94%) were UME educational programs focused on general topics of AI in medicine and 1 (6%) was an UME educational program focused on radiology concepts. In total, 17% (5/30) of the postgraduate educational programs were for residents who were in the radiology specialty. Of the 30 educational programs, 8 (26%) were specified for practicing physicians (n=4, 50% were CME educational programs focused on general topics of AI in medicine, n=3, 37% were radiology for CME education, and n=1, 13% was in cardiology for CME). The educational program characteristics are provided in Table 2.

The following curricular concepts were adapted and framed from previous similar reviews [32,33]. The curricular content and concepts were divided into 2 types: theoretical curricular concepts and application-based curricular concepts. The subcategories and their descriptions are outlined in Table 3. The following describe the theoretical curricular concepts: fundamental of AI for using AI systems (15/19, 79%) [12,25,31-33,42-47,49,51-53]; fundamentals of health care data science for using AI systems (10/19, 53%) [12,25,31-33,45,47,49-51]; strengths and limitations of AI (9/19, 47%) [31-33,45-49]; and ethical, legal, and economic considerations of AI systems (11/19, 58%) [12,25,31-33,42,45-48,52]. The following describe the application-based curricular concepts: applications of AI systems (19/19, 100%) [12,25,31-33,42-55], operating AI systems in health care settings (10/19, 53%) [12,25,31-33,43,46,47,52,55], impact of AI on clinical reasoning and medical decision-making (7/19, 37%) [12,25,31-33,43,55], communication of AI results to patients (4/19, 21%) [12,31-33], and critical appraisal of AI systems (7/19, 37%) [12,31-33,50,53,54].

Of the 19 papers, 6 (32%) presented the results of their evaluation of an educational program (Table 4) [25,42,44,48,49,55]. Two papers described only level 1 evaluation outcomes (eg, learner reaction and satisfaction with the educational program) in which participants were overall very satisfied with the AI content learned [42,48]. Four papers described level 2 evaluation outcomes (eg, change in attitude, knowledge, or skill) in which learners demonstrated acquisition of a variety of competencies (linear algebra pertaining to AI and basics of AI) and skills (eg, incorporate medical decisions given by an algorithm and implementing AI in clinical practice) [25,44,49,55] where two of these papers also evaluated level 1 outcomes [25,49]. There were no outcomes that could be categorized as level 3 or level 4; thus, the program evaluations did not comment on the change in behavior or affect at the organizational level or on patient outcomes.

The development and implementation of AI in medical education has greatly increased within the last decade, specifically since the COVID-19 pandemic where there was a global shift into the digital world accelerating the development of AI technology [59]. This can be seen as the majority (15/21, 77%) of included papers within this review were published since COVID-19 pandemic. Although there is a growing field within research and practice, AI medical education, specifically within curricula development, is still limited. We found that the current curriculum frameworks for AI medical education are limited, indicating a need for further research. We also found that the current state of AI educational programs lack the use of a theory, framework, or pedagogy. In addition, we uncovered alternative methods and different levels of in-depth curriculum planning for AI in medical education.

This is the first review to identify curriculum frameworks for AI medical education, and our findings demonstrate that they are very limited. Although the literature is abundant in terms of recommendations and potential plans of actions for integrating AI education within medical education, there is an inadequate amount of formal curricula or frameworks [20,60,61]. For example, curricular recommendations lack specific learning outcomes and are not based on a particular education theory, as they usually focus solely on the content or competencies that should be taught [32,56]. Although understanding what concepts should be taught in AI is important, curriculum frameworks must be as comprehensive as possible.

From the identified frameworks, Masters [56] outlines a broad framework for any level of education, while Valikodath et al [57] outlines a complete framework for ophthalmology residency education. Their frameworks remain dissimilar in all aspects, except in how their course content was described. As seen with these 2 papers, the lack of curriculum frameworks in the literature is staggering. Further studies should focus on the development of these frameworks and start thinking on how to plan for the impending changes in medical education. As Valikodath et al [57] demonstrated their AI curriculum framework for ophthalmology, other specialties should follow suit, as AI affects each specialty differently [9]. Overall, the current state of curriculum framework in medical education appears to be far from sufficient in the existing literature, and further research is needed.

Curriculum planning of AI educational initiatives within medical education is insufficient. Although limited studies of curriculum frameworks were published, other forms of curriculum planning can be seen in the literature. Some medical institutions have conducted AI perception surveys [69,70], curriculum needs assessment surveys [71-73], and an interview [74] to understand what should be integrated into the AI medical curriculum. These studies are promising and contribute to the overall efforts to understanding how current educators, medical students, residents, and physicians consider AI within their educational system.

The absence of curriculum frameworks is staggering, especially given that AI competence is likely to become a required skill for medical graduates [75]. The development of AI curricula and frameworks have already been gaining traction across other fields of education and levels. This can be seen as early as childhood education; for example, Su and Zhong [76] present their own curriculum framework, which outlines their concepts, teaching methods, teaching activities, projects, and assessment suggestions for AI education.

From a global perspective, the United Nations Educational, Scientific, and Cultural Organization, a specialized agency of the United Nations, released a document outlining the current practices of developing and implementing AI curricula in primary and secondary school education (K-12) [77]. From their report, several types of frameworks for AI literacy have been suggested, such as the AI Literacy Competency Framework, the AI4K12: 5 Big Ideas Framework, and the Machine Learning Education Framework. These recent reports and papers suggest increased efforts to integrate AI education before postsecondary school, which further stresses the importance of developing AI curricula and frameworks in medical education. Although there are current educational frameworks for AI education, each target audience must have their own specialized curricula to tailor the educational needs of the learners.

Medical educators can develop their curriculum through several different methodologies, such as the 10 key questions to be addressed while developing a curriculum [78] and the 6-step approach for curricular design [79]. However, curriculum frameworks allow a visual and detailed road map to implement a curriculum. Through this detailed format, educators are able to easily navigate the curriculum and its implementation, especially for new concepts in medicine, such as AI. To develop curriculum frameworks for AI in medicine, there must be an interdisciplinary team consisting of medical educators, AI experts and users, researchers, and curriculum designers due to the multiple fields incorporated.

The introduction of AI in medicine must be properly structured and organized within UME, PGME, and CME. Therefore, curriculum frameworks should be properly established through different levels of education and specialties. This has been emphasized by other reviews that call for integration of AI education in all levels and, thus, all specialties of medicine [17,33]. For example, a curriculum framework for UME will be different than a curriculum framework for PGME in dermatology. Curriculum frameworks can be adapted and they most likely will be, especially since AI education in medical education is still in its infancy. This is where leaders in UME, PGME, and CME organizations (eg, policy makers, medical educators, and researchers) must communicate effectively to eliminate any crossover education and repeated information. New technology and innovations in relation to AI and medicine will inevitably occur; however, it is important to be cognizant of the fundamentals of AI and how it will affect a physician’s practice at the time. Sufficient planning of an AI curricula will deliver effective education for physicians who will increasingly be using AI technology in the near future; therefore, medical educators and institutions must begin to consider curriculum planning.

The literature emphasizes the need to introduce AI in the medical education curriculum [12,13,15-20]; however, there are several challenges that have been discussed in terms of implementing this type of education. This includes insufficient time, insufficient resources (eg, lack of teaching staff or knowledge), and variable aptitude and interest in AI [80-82]. However, this review details several approaches to implementation as well as 6 studies that have evaluated their educational program. These successful educational programs can provide medical schools and national and international medical organizations with examples of current AI content topics and implementation methods that have worked for others. These medical education institutions can view how AI-based medical education is currently being offered around the world and understand any challenges, opportunities, and strengths about these programs. Although the content and provision of AI education is heterogenous, this heterogeneity can allow educators and students to view the many types of programs that were offered. As AI education for medicine is still in its infancy, educators should explore these programs where they can then potentially modify an educational program that best suits their needs. As seen in this review, there are several ways to incorporate AI material into the current curriculum seamlessly, such as an AI fundamentals lecture or module, an AI elective, or a research project.

Medical students, residents, and practicing physicians also have the opportunity to advocate for the inclusion of AI education at their respective institutions [46]. For example, there are several North American university chapters of the Artificial Intelligence in Medicine Student Society, such as the University of Toronto and University of Alberta, which organizes workshops, conferences, and multiple speaker sessions throughout the year [46]. These student interest groups demonstrate the increased interest for AI and can potentially build momentum and advocate for AI education at their respective institutions. As some of the offerings at these student interest groups include brief educational material for AI, medical institutions can work with these students as a starting point.

The strengths of this review include the comprehensive search strategies, the inclusion of a variety of information sources, and rigorous methodological approaches that are replicable. For example, study selection was completed by 2 reviewers, and disagreements were resolved by discussion or consensus involving a third investigator. Furthermore, a scoping review protocol was registered and published to improve transparency of the methodological process.

Although this study was conducted in a structured and systematic manner, there are some limitations that are important to consider. A limited number of papers were retrieved during the search and selection process. Only 2 papers reported having a curriculum framework, with 1 reporting a full curricula plan related to AI in medicine. This can be because AI technology is emerging and continuing to change within medicine and it has been limiting in terms of educational advances. Because of the nature of the scoping review, the quality of each identified study was not assessed.

Medicine is rapidly evolving from the information age to the age of AI, where machines will become an integral part of medical practice. Thus, medical education needs to keep pace with changes in medical practice. This review synthesized knowledge from the literature on curriculum frameworks and current educational programs that focus on the teaching and learning of AI for medical students, residents, and practicing physicians. To better integrate AI curricula into the continuum of medical education, discussions surrounding curriculum planning of AI should begin where institutions are recommended to work collaboratively with teams of curriculum designers, data scientists, and medical educators to develop AI curricula and educational programs. There is a need to (1) develop a general AI education curriculum framework for UME; (2) develop a specific AI education curriculum framework for each specialty within PGME and CME; and (3) design, implement, and evaluate current educational programs. Overall, institutions must begin equipping current and future physicians with the knowledge, skills, and confidence to effectively use AI applications as it will continue to grow within the field of health care.