Original Paper
Abstract
Background: As health care moves to a more digital environment, there is a growing need to train future family doctors on the clinical uses of artificial intelligence (AI). However, family medicine training in AI has often been inconsistent or lacking.
Objective: The aim of the study is to develop a curriculum framework for family medicine postgraduate education on AI called “Artificial Intelligence Training in Postgraduate Family Medicine Education” (AIFM-ed).
Methods: First, we conducted a comprehensive scoping review on existing AI education frameworks guided by the methodological framework developed by Arksey and O’Malley and Joanna Briggs Institute methodological framework for scoping reviews. We adhered to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist for reporting the results. Next, 2 national expert panels were conducted. Panelists included family medicine educators and residents knowledgeable in AI from family medicine residency programs across Canada. Participants were purposively sampled, and panels were held via Zoom, recorded, and transcribed. Data were analyzed using content analysis. We followed the Standards for Reporting Qualitative Research for panels.
Results: An integration of the scoping review results and 2 panel discussions of 14 participants led to the development of the AIFM-ed curriculum framework for AI training in postgraduate family medicine education with five key elements: (1) need and purpose of the curriculum, (2) learning objectives, (3) curriculum content, (4) organization of curriculum content, and (5) implementation aspects of the curriculum.
Conclusions: Using the results of this study, we developed the AIFM-ed curriculum framework for AI training in postgraduate family medicine education. This framework serves as a structured guide for integrating AI competencies into medical education, ensuring that future family physicians are equipped with the necessary skills to use AI effectively in their clinical practice. Future research should focus on the validation and implementation of the AIFM-ed framework within family medicine education. Institutions also are encouraged to consider adapting the AIFM-ed framework within their own programs, tailoring it to meet the specific needs of their trainees and health care environments.
doi:10.2196/66828
Keywords
Introduction
The College of Family Physicians of Canada (CFPC) establishes standards for postgraduate family medicine training and its accreditation []. It promotes a competency-based curriculum model known as Triple C (comprehensive, continuous, and centered in family medicine) [] based on the Canadian Medical Education Directives for Specialists (CanMEDS)—Family Medicine framework [] and on assessment objectives for certification in family medicine []. To ensure that medical curricula respond to new developments in health care, education, and societal trends, they must undergo periodic review, modification, and renewal [-]. Accordingly, a number of new content areas have been introduced in the recent past into the family medicine curricula. They include leadership [], social determinants of health [], ethics [], global health [-], and physician wellness and burnout [-]. The increasing complexity of the medical needs of an aging population, the exponential growth in medical knowledge, and an increasingly digitalized environment suggest the need for digital-mediated solutions to support medical practitioners.
Artificial intelligence (AI) and its applications have made a rapid impact on many segments of society, including medicine [] and notably, in primary health care []. While there is no universal consensus on the definition of AI, the World Health Organization [] describes it as “the performance by computer programs of tasks that are commonly associated with intelligent beings.” The introduction, integration, and implementation of AI-based tools and systems into family medicine education and practice assume an adequately trained cohort of users, but to date, training of family physicians on relevant aspects of AI to ensure effective and safe implementation has been absent or inconsistent [,]. As such, the CFPC’s Outcomes of Training project has identified digital care and health informatics as a training gap and an area for educational enhancement requiring priority attention across the 17 family medicine postgraduate programs in Canada [,]. There have been efforts to include AI education globally within each level of medical training. These efforts are led by national medical associations such as the UK National Health Service, the US American Medical Association, and Canada’s Royal College of Physicians and Surgeons. They have released documents recommending policies for integrating AI within their respective medical educational institutions [-].
Initiatives of AI teaching directed at physicians already in practice include the development of a continuing professional development 3-module CFPC Learn e-course titled, “Artificial Intelligence for Family Medicine” []. The first module of this course reviews the basic functionality of AI with applications in family medicine, while the second module focuses on core terminology and related concepts as well as potential harms or risks associated with AI. The last module reviews the concepts of the first 2 and focuses on learning how to tell if an AI-based tool is working well [].
Competency about a particular subject has been described as the ability to carry out a certain task or action at a basic or acceptable level []. Liaw et al [] have recently proposed six competency domains for family medicine training in AI: (1) foundational knowledge (What is this tool?), (2) critical appraisal (Should I use this tool?), (3) medical decision-making (When should I use this tool?), (4) technical use (How do I use this tool?), (5) patient communication (How should I communicate with patients regarding the use of this tool?), and (6) awareness of unintended consequences (What are the “side effects” of this tool?).” These authors suggest that such competencies can be integrated within current residency training during existing sessions on health informatics or evidence-based medicine but emphasize that these competencies are a “point of departure” and must be further worked on [].
A curriculum framework can be described as “a core policy document that describes a range of requirements, regulations and advice which should be respected by all stakeholders in the education system, and which should guide the work of schools, teachers and the developers of other curriculum documents” []. Curriculum frameworks allow for a visual and detailed roadmap to develop and implement a curriculum []. Input from an interdisciplinary team of medical educators, AI experts, end users, researchers, and curriculum designers [] can effectively support the development of a curriculum framework for teaching AI in family medicine postgraduate training programs. Our comprehensive review of the available curriculum frameworks [,] highlighted that there is no framework designed specifically for family medicine residency and no paper that described a systematic approach to design one. From the 2 frameworks uncovered, one framework was incomplete, while the other framework was brief and focused on ophthalmology []. The ophthalmology curriculum framework lacks adaptability, as it may prove inadequate for family medicine residency due to the diverse, community-based nature of family medicine, which differs significantly from the highly technological and hospital-based focus of ophthalmology.
Considering the gaps mentioned previously and the foundational importance of curriculum frameworks in the creation of new educational structures, our objective was to design and develop a curriculum framework for AI family medicine education, that is, Artificial Intelligence Training in Postgraduate Family Medicine Education (AIFM-ed), ensuring alignment with current competencies and educational goals. To achieve this, a combination of validated methods including 2 national expert panel discussions were conducted, supplemented by a previous comprehensive systematic scoping review []. Developing a framework based on expert insights would help address gaps in AI education and provide an adaptable guide for family medicine educators, curriculum designers, postgraduate residency program directors, medical education researchers, and policy makers in health care education. Due to the systematic approach in designing this framework, audiences can adopt this framework to other fields and specialties, considering that our review did not find any systematically developed frameworks.
Methods
Study Design
For the construction of an AIFM-ed framework, we followed the analysis, design, development, implementation, and evaluation model for instructional design, using the first 3 activities to guide our work. We followed a two-step approach suggested by Redwood-Campbell et al [] for framework development, wherein (1) a review of the literature was made focusing on curriculum frameworks and core competencies for AI education in medicine [,] and (2) a working group used qualitative or consensus methods for final development of the framework.
Our scoping review aimed to synthesize knowledge from the literature on curriculum frameworks and current educational programs that focus on the teaching and learning of AI for medical students, residents, and practicing physicians, and adhered to PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Details of this comprehensive study have been published elsewhere [,]. Our review specifically identified several AI educational curricula programs (eg, courses, workshops, webinars, and projects) and 2 curriculum frameworks for AI education, one outlining a broad framework for any level of education [], while the other described a complete framework for ophthalmology residency education [].
The outcome of our review was the identification of early concepts that could be applied to elements of the curriculum framework for family medicine and AI [,]. This initial curriculum framework was later used during the panel discussion as part of the co-development and redesigning of the framework. This discussion applied the curriculum framework structure described by Obadeji [], which examines six common elements: (1) the need and the purpose of a curriculum or a program, (2) learning objectives and outcomes, (3) course content that will facilitate the accomplishment of the objectives or learning outcomes, (4) organization of the content, (5) implementation of curriculum, and (6) curriculum evaluation and refinement. This study examined all elements except the final element (curriculum evaluation and refinement). The initial framework was deemed successful by the expert team based on the following indicators: relevance to medical educators and curriculum designers, alignment to current family medicine competencies and educational goals, clarity of AI-specific content, and its potential for further validation. However, we acknowledge that further studies are needed.
Qualitative Methodology
The expert panel methodology follows the SRQR (Standards for Reporting Qualitative Research) checklist []. Expert panels help to attempt to reach consensus on controversial subjects [,] such as the risk of AI tools leading to reduced proficiency in independent critical thinking and clinical judgment among physicians. The use of qualitative consensus methods for curriculum development facilitates input from a wide range of stakeholders (eg, physicians and curriculum developers) in order to assess and validate expert knowledge []. The use of expert panel discussions to assist in creating curricula has become established in pedagogical research and development []. Examples within the field of medicine include discussions around social determinants of health for undergraduate medical education [], telemedicine opportunities for postgraduate medical education [], and geriatric oncology in continuing medical education [].
Participant Recruitment and Sampling Strategy
Our panel size fell within the recommended average of 8 members or a median of 6 []. The definition of an expert in our case is flexible due to the limited knowledge and experience on this emerging topic; this is emphasized by Duncan et al [], who state that, “[t]oo narrow a definition, however, can restrict the number of potential participants.” In our case, we chose experts according to the definitions of Fink et al [], which state that they must be, “representative of their professional group, with either sufficient expertise not to be disputed or the power required to instigate the findings.” This was reinforced by Mead and Mosely [], which state that, “experts can be defined in a number of ways, such as their position in a hierarchy [...] or as recommended by other participants in a study.” Therefore, from these definitions, we selected panelists based on their academic qualifications, their number and relevance of AI-related publications, professional experience within the development, implementation or research of AI, and finally, any participation in AI-specific projects or conferences.
For panel 1, we reached out to family medicine (clinical) educators from affiliated universities and professional organizations across Canada via email. Snowballing by this initial group generated the names of others known as family medicine educators. For panel 2, family medicine residents were invited from an initial group of residents who were knowledgeable and aware of AI, and that initial group helped to recruit relevant residents for this study.
Each participant voluntarily participated in the study by providing their explicit consent and agreement, which was confirmed through email correspondence. To uphold confidentiality, data were safeguarded through limited, secure data access, the disposal of audiotapes after transcription, and the anonymous analysis of transcripts.
Ethical Considerations
This study involved a panel discussion with experts, which does not require formal ethics board approval under the Economic and Social Research Council Framework for Research Ethics guidelines []. According to these guidelines as well as Canada’s Tri-Council Policy Statement on Ethical Conduct for Research Involving Humans, research that presents minimal risk and does not involve sensitive information may be exempt from formal ethics review []. This study adhered to these recognized guidelines, ensuring that all participants were treated in accordance with principles of research integrity, voluntary participation, and informed consent.
Participant Eligibility Criteria
Input from 2 different types of panelists was desired, and they were included as participants within 2 distinct expert panels. The first included family medicine educators practicing in Canada who were somewhat knowledgeable or have expertise in AI education. No limitations were placed on years of practice experience, years of knowledge or experience in AI, language proficiency, work setting, or the types of patients for whom they provided care. The second expert panel included participants who were at the time of the study family medicine residents at McGill University, and who were somewhat knowledgeable in AI. No limitations were placed on language proficiency, years of knowledge or experience in AI, work settings, or the types of patients they provided care for.
Data Collection
We conducted a recorded session of each expert panel via Zoom (version 5.16.10; Zoom Video Communications). The use of a web-based expert panel minimizes costs associated with travel; it also mitigates potential biases linked to panelists []. Each web-based expert panel discussion was approximately 2 hours long, followed the same format, used congruent discussion guides, and was facilitated by 2 members of the research team (RT and SAR). The discussions began with a brief presentation given by RT on the results of the first step of the project, that is, the comprehensive scoping review in the field [,]. Following the presentation, each of the five elements of the curriculum framework: (1) the need and the purpose of a curriculum or a program, (2) learning objectives and outcomes, (3) course content that will facilitate the accomplishment of the objectives or learning outcomes, (4) organization of the content, and (5) implementation of the curriculum were discussed sequentially and at length. When presenting each element, participants were invited to respond and discuss their opinions and thoughts related to each element, allowing for the co-development and redesigning of the framework together.
Data Analysis
Expert panel discussion data were analyzed using content analysis strategies [,] as previously used in a study developed for a training model for nurses using a literature review and expert panel discussions, in which data were analyzed using a descriptive qualitative approach that includes content analysis []. In our work, the preparation phase included transcribing the data, immersing in the data, and obtaining a sense of whole through reading the transcript multiple times. In our study, once the recordings from the expert panel discussions were received, one of the authors (RT) listened to the entire recording and subsequently transcribed it verbatim. The next stage of data analysis was the organizing phase, in which open coding and the creating of categories were conducted along with the grouping of codes under higher-order headings. These were carried out by one of the authors (RT) and verified by the senior author (SAR).
As the analysis of data used an inductive approach, no prior coding systems were used, such that coded categories were derived directly from the data []. Sentences and phrases from the panelists were captured. In vivo coding was used to prioritize participants’ language and perspectives, while descriptive coding aided in categorizing key themes. Two independent coders reviewed the data (RT and SAR), with discrepancies resolved through discussion between coders and the research team. Saturation was achieved when no new themes emerged during the coding of the final transcript. The final step included the presentation of the final curriculum framework, which resulted from the incidence of codes and categories and its relation to the literature. Codes and categories derived were prioritized and highlighted with how frequently they appeared during the panel discussion as well as the overlap between both groups. These highlighted findings were then compared with existing literature to either support or challenge them. If these codes and categories were supported by the literature, they were subsequently integrated into the framework.
Results
Panelists Characteristics
A total of 37 educator and resident experts were invited, 14 for the educator group and 23 for the resident group. Ultimately, 8 from the former and 6 from the latter group participated, for a total of 14 participants. Scheduling problems were the most common reasons for nonparticipation. The characteristics of those included in the expert panel discussion are displayed in 
Table 1.
| Educator experts (n=8), n (%) | Resident experts (n=6), n (%) | ||
| Sex | |||
| Male | 3 (38) | 4 (66) | |
| Female | 5 (62) | 2 (33) | |
| Educational background | |||
| Doctoral (PhD) | 7 (88) | 0 (0) | |
| Master | 1 (22) | 2 (33) | |
| Bachelor or MD only | 0 (0) | 4 (66) | |
| Affiliation | |||
| McGill University | 5 (62) | 6 (100) | |
| Other academic institution | 3 (38) | 0 (0) | |
Curriculum Framework for AIFM-ed
Overview
Our project has identified five elements of the curriculum framework for AI training in postgraduate family medicine education: (1) need and purpose of the curriculum, (2) learning objectives, (3) curriculum content, (4) organization of curriculum content, and (5) implementation of the curriculum. A condensed visual representation of the AIFM-ed curriculum framework is displayed in 
Figure 1, while each element is discussed in detail below.
Element 1: Need and Purpose of the AIFM-ed Curriculum
When modifying a curriculum in family medicine postgraduate training, it is important to understand why it must be changed and for what purpose. Both panels discussed the current low priority of AI curricula. Residents emphasized a lack of exposure in training and practice. Both panels agreed that the integration of an AI curriculum will inevitably become imperative, recognizing its potential as an essential toolset in practice. One educator summarized this thought by saying, “AI will continue to evolve quickly, so a curriculum must be built for the future.”
To describe the need and purpose for AI education in family medicine, we co-developed the following: “The purpose of an AI curriculum for family medicine residents is for future family physicians to: (a) gain foundational and adaptive knowledge and clinical skills, (b) to critically appraise evolving AI tools, and (c) to align AI integration in Family Medicine with the quintuple aim for health care improvement (i.e., improving population health, improving the provider and patient experience, reducing costs, and advancing health equity [,]).”
Using various definitions of AI, the educator panel debated what constitutes AI specifically in family medicine. The term “AI-based tools” is used throughout the results of this paper as a way of describing technologies empowered or enabled with AI algorithms to support clinical practice. This term has been used in previous literature on AI in the context of family medicine training [,].
Element 2: AIFM-ed Learning Objectives
Learning objectives are statements that describe significant and essential learning that learners need to be familiar with and reliably demonstrate at the end of a course or educational program []. The following outlines the learning objectives for AI training, aligning with CanMEDS and family medicine roles. Table 2 presents each CanMEDS role on the left column [], with their affiliated learning objectives for AI family medicine education as structured by participants during the panel on the right column. Although the learning objectives are comprehensive and their practical application for most family doctors may be limited, they are ideal for advancing the knowledge and skills of AI-empowered family physicians.
| CanMEDS roles | The learner engaged in AI education will be able to | |
| Family medicine expert with AI knowledge | ||
| Family physicians are skilled generalists who should be able to understand and use technology including AI tools to provide high-quality, responsive, community-adaptive care across the lifecycle, from prevention to palliation, in multiple settings, and for diverse populations. |
| |
| Communicator | ||
| Family physicians foster life-long therapeutic relationships with patients and their families. This incorporates the dynamic exchanges that occur before, during, and after the medical encounter that facilitates gathering and sharing essential information for effective patient-centered health care []. |
| |
| Collaborator | ||
| Family physicians work with patients, families, communities, and other health care providers to provide safe, high-quality, patient-centered care []. |
| |
| Leader | ||
| Family physicians must actively contribute to implementing and maintaining a high-quality health care system and take responsibility for delivering excellent patient care. This includes prioritizing and using health care resources efficiently, executing tasks collaboratively with colleagues, and contributing to ongoing quality improvement initiatives within their own practice and its management []. |
| |
| Advocate | ||
| Family physicians leverage AI-driven insights to advocate for patients and communities, using their expertise to identify health needs, drive meaningful change, and mobilize resources for improved care outcomes []. |
| |
| Scholar | ||
| Family physicians demonstrate a lifelong commitment to excellence in practice through continuous learning and teaching others; gather, combine, and evaluate evidence; and contribute to the creation and dissemination of knowledge []. |
| |
| Professional | ||
| Family physicians are committed to the health and well-being of their patients and society through competent medical practice; accountability to their patients, the profession, their colleagues, and society; profession-led regulation; ethical behavior; and maintenance of personal well-being []. |
| |
Element 3: AIFM-ed Curriculum Content
When developing a curriculum, a crucial task is to identify relevant subject knowledge, skills, attitudes, and behaviors that will help form the learning objectives []. Currently, there is no required AI education in Canadian undergraduate medical education. However, both educators and residents in our study agreed that for AI to be efficiently introduced in family medicine residency, it must be preceded by education in undergraduate medical education. This earlier introduction of principles and concepts of AI will facilitate learning the more difficult material that is to come. The panels envisaged a basic stream of education in residency for those who had no exposure in undergraduate years. This would address fundamentals and basic knowledge of AI (eg, history, AI model development process, and core algorithms). A more advanced stream of AI education for residents would summarize the fundamentals and focus on how to use AI-based tools (applications) along with how to decide when to use and evaluate them (critical appraisal).
Residents noted that understanding how AI-based tools are used in clinical practice was the preferred content area for study, with less attention devoted to ethical, legal, and social considerations of AI. A resident put this in context, noting that they “do not need or want to learn the history of ChatGPT, but rather how to write effective prompts within this natural language processing chatbot.” Table 3 summarizes the key concepts and areas of interest that family physicians should learn and content to include in the curriculum, as viewed by the participants.
| Main curricular topic | Subtopics | |
| Overview of AIa and data science in family medicine | ||
| Providing an overview of AI definitions and concepts including machine learning as well as topics related to data science (eg, mathematics and statistics) and clinical epidemiology for family medicine. |
| |
| Ethics, legal, and social considerations | ||
| Understanding the ethical, legal, and social concerns of AI as it impacts family medicine clinical practice. |
| |
| Application of AI in family medicine | ||
| Understanding how to choose and engage with AI-based tools in clinical settings and workflows with the ability to understand, interpret, and apply results of AI systems in clinical practice. |
| |
| Appraisal of family medicine AI-based tools | ||
| Assessing and reviewing AI-based tools to ensure safe and effective integration and use in clinical practice. |
| |
aAI: artificial intelligence.
Element 4: Organization of AIFM-ed Curriculum Content
Family medicine postgraduate training is 24 months long in Canada. Given that the current curriculum is considered very heavy, educators and residents emphasized that the addition of another competency could be a burden to both educators and resident learners. They nonetheless agreed that AI curricula will eventually need to be added to that and an organized teaching structure would need to be established. Residents favored incorporating the teaching within the existing, already tight, 24-month core teaching, so that the benefits of longitudinal learning could be taken advantage of. The educators saw AI knowledge–based training during the first postgraduate year, followed by the development of AI-based clinical skills in the second postgraduate year. Educators proposed that if deeper AI education is needed, an additional third-year training program could be introduced for a select group of interested trainees to develop advanced AI skills in family medicine.
Element 5: Implementation of AIFM-ed Curriculum
Curriculum implementation will require the identification of appropriate resources (eg, educators and materials) along with educational strategies that will facilitate teaching activities and learner evaluation.
Curriculum Delivery
Residents highlight that AI education must be longitudinal, as it must be built upon throughout the medical education continuum. Furthermore, educators emphasized that residency is student-centered with learners coming from diverse backgrounds where they must replicate the actual tasks performed during in practice. Therefore, the learning theory of constructivism appears to be a sound and advantageous choice. This learning theory posits that learners actively construct their own learning by drawing upon their prior experiences [].
There are several methods to implement an AI education curriculum to family medicine residents; however, there are certain methods that are recommended by both educators and residents. In terms of learning about the knowledge and background of AI (eg, review of AI concepts or the ethical, legal, and social considerations of AI), hybrid (web-based and in-person) courses with asynchronous web-based modules, and in-person workshops, problem-solving sessions could be applied. Residents emphasized that didactic large group lectures especially in regard to a novel topic such as AI would be less engaging. The learning of such content should be considered a refresher with emphasis on the context of AI in family medicine. Both educators and residents then suggest that the in-person sessions would serve as a space for questions and answers and problem-solving activities.
To execute these educational methods, human resources (eg, AI medical educators) and material resources (eg, existing AI-based tools) are pertinent. Educators and residents highlighted that experts in the field of AI and family medicine would be ideal; however, educators emphasized that the faculty challenges such as the current number of experts are limited to provide this education. To overcome this, residents suggested that once an AI curriculum is established, further educators could be sourced from recently graduated residents who completed the AI in family medicine curriculum. With respect to material resources such as family physician–focused AI-based tools, both groups emphasized that they must be validated before use in educational settings.
Assessment and Evaluation Methods
Residents emphasize that the assessment and evaluation methods for the curriculum should be simple in context and focus on learners’ participation and exposure. More specifically, learners should be able to have the capacity to demonstrate how to use AI-enabled tools and techniques in a health care setting. This can be seen through the completion of projects and problem-based and simulation-based assessments. Educators on the other hand emphasized taking into account Kirkpatrick’s 4 levels of training evaluation model [], where assessments should be directly related to the activity’s learning objectives.
Discussion
The First Curriculum Framework for AI in Family Medicine (AIFM-ed)
In this study, we introduced a novel and evidence-based initial curriculum framework, that is, AIFM-ed developed for AI literacy education in family medicine postgraduate training. This systematically co-developed framework used a combination of validated methods including a comprehensive scoping review, resident and educator panel discussions, and the involvement of interdisciplinary experts in the field. During the development and cocreation of this framework, several key findings emerged. These include the crucial role of multiple resource partners and innovative practices when integrating AI educational content in family medicine education. For example, AI technology vendors specializing in health care, upcoming startups, and AI-focused organizations.
Furthermore, educators and residents stressed the importance of learning about the application of AI-based tools and simulating their use as a method of learning. Several innovative practices have already been implemented including case-based learning and flipped classroom models. Moreover, the adoption of AI-based tools can be diverse depending on its context (eg, teaching and learning and clinical practice) with several barriers and enablers. Additionally, the study identified several challenges in effectively integrating an AI curriculum framework into existing educational structures. These include the lack of AI definition standardization, the reduced urgency in practice due to the lack of time and resources, as well as the capacity to balance theoretical and practical curricular content.
Interprofessional Collaboration and Resources
During the development of the AIFM-ed curriculum framework, several resource partners were identified when discussing the implementation of AI education in family medicine. Interprofessional collaboration within multidisciplinary teams is essential in order for an AI curriculum to be effective []. Other researchers emphasize this sentiment when listing their recommendations of ensuring a responsible integration of AI technologies in medical education []. This multidisciplinary team and resource partners may include several stakeholders such as nurses, social workers, epidemiologists, AI experts, data engineers, software developers, and patients []. Other resource partners identified included AI technology vendors specializing in health care, upcoming startups, and AI-focused organizations. Residents brought up the concept that AI-based tools and AI in general will substantially change in the future (eg, improved tools, systems, and integrations) and thus stressed the importance of continuous partnerships with other professionals in order for relevant information and AI tools.
Educators emphasized that they were unaware of many AI-based tools for patient support and were thus apprehensive in advocating for AI-based tools. Therefore, family physicians and other primary care team members (eg, administrative staff and nurses) should share AI-based tools and resources, when available and known, with other family physician and family medicine communities. Additionally, residents have suggested that before advocating or suggesting AI-based tools, a list of recommended AI-based tools must be developed and released from a medical organization such as the CFPC. Currently, there is a scoping review and inventory that has identified and evaluated published studies that have tested or implemented AI in primary care settings [,]. This can be a starting point for such a list of recommended AI-based tools.
Application and Simulation of AI-Based Tools
Both educators and residents emphasize that a curriculum should focus on how to use AI-based tools (application) along with how to decide when to use and evaluate them (critical appraisal). Residents are already doing this comparatively as seen through their discussions of using ChatGPT, an AI-based chatbot launched by OpenAI that can be used as a digital consultant (eg, simple inquiries about diagnoses and treatment plans). One resident stressed that although they use ChatGPT at times for inquiries related to patient care, they are cautious of the information, as they are aware that ChatGPT can make mistakes and always consult other resources. As ChatGPT rises in prominence, its impact on medical education has been evident through the resident panel discussion and the literature [,].
The incorporation of AI content in medical education has already begun with innovative practices, which include case-based learning and flipped classroom models. Case-based learning incorporates real-world AI use cases, where AI is used in clinical practice as examples for physicians []. Through this learning approach, students have a better understanding of the technical aspects of AI, as it allows physicians to compare their thought processes with other students and critically reflect or challenge their assumptions and biases of AI and clinical practice []. One study assessed the capabilities of ChatGPT within the framework of a preclerkship case-based active learning curriculum. Although the AI chatbot is not comprehensive enough to serve as a textbook, it was shown to answer questions, generate test questions, and appropriately respond to prompts in case-based learning scenarios []. According to a scoping review of teaching AI ethics in medical education, 5 publications reported in using case-based learning when understanding ethical challenges []. Resident panelists believe that simulation of these tools is beneficial, as it allows residents to enjoy the learning process and realize how these AI-based tools would operate in actual clinical settings. During these simulation sessions or case-based learning approaches, educator panelists highlighted reviewing the capabilities and basic functions of AI-based tools.
Another practical example of incorporating AI content through innovative practices is the flipped classroom model approach. Flipped classroom models can consist of web-based content supplemented by in-person classroom sessions [], a key observation reinforced by residents of the panel discussion. One study designed and evaluated a novel AI course for medical students using a flipped classroom model, and they found that attending the course can increase self-perceived AI readiness in medical students []. In addition, educators have also commented on facilitating AI learning by integrating family medicine AI-based tools in quality improvement projects, which has been emphasized and recommended by other researchers [].
Adopting AI in Education and Clinical Practice
Family physicians use AI, when implemented, primarily for diagnosis, detection, or surveillance purposes []. Although educators have flexibility in choosing from a wide range of AI tools, certain tools have proven to be particularly essential for effective integration. These include AI-enabled chatbots, clinical documentation support, and diagnostic decision support, which have shown to improve physicians’ efficiency and accuracy in their work [-]. However, there have been several barriers identified in previous reviews, which have made the adoption of AI-based tools difficult [-]. These issues include a lack of trust among educators, students, and clinicians; insufficient training and digital literacy; and resistance to change [].
Additional challenges include data privacy and patient safety concerns, ethical and legal issues, interoperability issues, lack of funding, and inequities in access to AI tools—particularly between rural and urban settings []. In contrast, several strategies and enablers have been identified in order to better facilitate the adoption of AI and its continued use. These strategies include strategies fostering interdisciplinary collaboration between educators, clinicians, and AI developers; providing targeted training programs to build AI literacy; developing high-quality datasets for diverse use cases; and creating supportive regulatory frameworks []. Establishing national or local community networks to share resources and best practices, while leveraging trusted relationships within these networks, can also significantly enhance confidence in and adoption of AI-based tools. To identify relevant enablers and barriers to AI adoption of a certain audience, a comprehensive, stakeholder-centered approach is essential. For example, researchers in Canada conducted in-depth interviews with primary health care and digital health stakeholders and were able to ascertain their current barriers and potential facilitators of AI [].
It is important to note that AI systems exist in diverse contexts and content with distinct implications, risks, and ethical and legal challenges depending on their application and domain. For example, in education, AI-enabled tools using large language models may offer personalized education, but biases may be propagated, inaccurate information may be generated, or students may overrely on AI, undermining their critical thinking skills [,]. In addition, there is potential for the exacerbation of inequities in accessing AI tools as well as the misuse of AI-generated content. In comparison, AI-enabled tools in clinical practice, such as decision-support systems, could carry risks of incorrect or biased recommendations that may directly impact patient outcomes [,], thus, raising ethical concerns about patient autonomy and safety as well as legal liability in cases of harm. Therefore, the differences of AI in each domain are important to understand in order to identify appropriate safeguards. Future research should conduct comparative analyses of AI’s risks, implications, and ethical and legal dimensions in educational versus clinical settings, examining factors such as accuracy, equity, accountability, and trust. These studies can inform best practices and policies to optimize AI’s potential while mitigating domain-specific risks.
Curriculum Framework Challenges
During the development of this curriculum framework, there were several challenges in effectively integrating an AI curriculum framework into a family medicine residency training program. During the expert panel discussions, many experts emphasized the issue regarding the lack of standardization with the definition of AI. Although a definition of AI was chosen for the purpose of the panel, a specific and committed definition of AI within medical education has not been established [-]. Panelists argued that an AI definition must be properly explained to avoid confusion or misrepresentation. In relation to family medicine, a recent primer for AI in primary care was published, which provided the definition, “The field of AI is broad and rapidly expanding. The field is centred on how computers might be able to perform humanlike ‘intelligent tasks,’ such as summarizing large amounts of information or making inferences about a situation” []. The discussions regarding this framework highlight the necessity of a standardized AI definition for better development of teaching and learning content. This is especially true when specializing in different fields of medical education, including family medicine and primary care.
There is a need to introduce AI education within family medicine; however, the low urgency and priority to integrate this type of education at the moment were noted throughout the discussions. This can be due to the lack of AI-enabled tools for family physicians currently being developed, tested, and implemented in practice [,]. Furthermore, some residency programs lack the appropriate AI tools or are in lower-resource settings. As a result of the minimal exposure family physicians have with AI, their motivation to learn about the topic can also be reduced. This reduced priority of AI education competes with the CFPC’s 105 topics of family medicine curricula []. This is exacerbated by the fact that Canada is in a unique position, in which the length of residency training is only 2 years. In addition, the rapid advancement of AI introduces an extra layer of complexity. As new AI-based tools emerge and existing ones advance, educators and family physicians must frequently reassess and update their knowledge and skills. For example, the recent introduction of generative AI and generative AI tools such as ChatGPT has gained widespread popularity in medical and academic settings []. Thus, it is difficult to maintain a robust framework due to the inevitable rapid changes of AI in health care. Therefore, the eagerness to integrate this type of education within the curriculum should be met with caution to manage the expectations of both educators and learners.
A key observation made throughout the panel discussion was about the AI content and how much should a family physician know about AI. During the discussions, many of the participants voiced support on the application and appraisal of AI-enabled tools. This is especially challenging when residency is only 24 months, and there are no required AI educational programs presented in the Canadian undergraduate medical education system. Therefore, within the learning objectives, in regard to how much a family physician should know about AI remains undetermined. Further research must be conducted to investigate the level of AI education a family physician should be aware of. Overall, the aforementioned challenges must be addressed in order for this curriculum framework to be effectively implemented.
Future Studies
Following the analysis, design, development, implementation, and evaluation model process, researchers may move forward to the implementation and evaluation of the AIFM-ed framework. During the implementation step, an educational program such as a course or workshop can be developed with the main concepts originating from the curriculum framework. The training for family medicine is already packed; thus, the implementation of this framework will depend on several factors including the current use of AI-enabled tools in family medicine training, previous training in AI (eg, the undergraduate foundation of AI), and the capacity of experienced teachers. However, once implemented, certain success indicators will need to be evaluated to understand its impact as well as any areas for improvement. Future studies could explore indicators such as the perceived impact of the framework, degree of implementation, as well as knowledge and skill apprehensions. These indicators can be evaluated through the framework-derived educational training program according to the Kirkpatrick model [].
Strengths and Limitations of This Study
This study had several strengths, including the formation of a national, multidisciplinary panel of family medicine educators. This diverse panel facilitated enriching discussions with varied expertise and insights, allowing for a comprehensive understanding of practical implications and current perspectives on AI education in family medicine postgraduate training. Additionally, by involving both educators and residents, the AIFM-ed curriculum framework ensures the representation from key stakeholders involved in the teaching and learning process of AI education. This co-design approach enhanced the relevance and applicability of the AIFM-ed curriculum framework. Regarding the overall development of this framework, a multi-method systematic approach was used, which includes a comprehensive systematic scoping review and multiple expert panel discussions. This approach allowed us to identify and build on existing AI curriculum topics and resources while also creating new ones. Furthermore, this structured and reproducible methodology ensures a robust foundation that can be used by other educators and researchers to develop training programs (eg, courses) following the established framework.
Despite the strengths, this study also had few limitations. First, the study was developed for programs in Canada, which limits its applicability to other countries due to the different medical education structures globally and their current relationships with AI. However, this could be a starting guide for other researchers to adapt it to their own context. Additionally, expert panel diversity was limiting, where the resident panel came from a single institution, which may further limit the generalizability of the framework. Furthermore, as the participants for the panel discussion were not randomized and were purposively recruited, the results may be subject to selection bias.
Conclusions
We co-developed an AIFM-ed framework for family medicine residency training that outlines its curricular purpose, learning objectives, AI curricular topics, delivery methods, and evaluation strategies to be used by medical institutions. The AIFM-ed curriculum framework ultimately aims to enhance the education of future family physicians, equipping them to effectively integrate AI-enabled tools into their practice and patient care. It is hoped that this framework will provide further advocacy, productivity, and gradual change within the area of curriculum development and AI medical education. Overall, medical institutions are encouraged to begin equipping future physicians with the knowledge, skills, and confidence to effectively use AI-enabled tools, as these technologies will continue to grow within the field of health care and family medicine.
Acknowledgments
The authors would like to thank the panelists for their support, time, and contributions to this research. The authors would also like to acknowledge Dr Pierre Pluye who provided key guidance to the development of this project prior to his passing in August 2023. SAR is Canada Research Chair (Tier II) in artificial intelligence and Advanced Digital Primary Health Care, received salary support from a Research Scholar Junior 1 Career Development Award from the Fonds de Recherche du Québec-Santé during part of this project, and her research program is supported by the Natural Sciences Research Council Discovery (grant 2020-05246).
Authors' Contributions
Conceptualization was led by SAR and RT, who established the study’s goals, design, and research questions and obtained the funding for the project. The methodology was developed by SAR and RT. The data collection was done by RT and SAR. Data curation was managed by RT. A formal analysis was conducted by RT. The original draft was written by RT. SAR, FH-E, and MY provided critical revisions. Reviewing and editing were a collaborative effort with all authors. Supervision and overall project leadership were provided by SAR.
Conflicts of Interest
None declared.
References
- Hennen BK. Academic family medicine in Canada. CMAJ. 1993;148(9):1559-1563. [FREE Full text] [Medline]
- Oandasan I, Working Group on Postgraduate Curriculum Review. Advancing Canada's family medicine curriculum: Triple C. Can Fam Physician. 2011;57(6):739-40, e237. [FREE Full text] [Medline]
- Shaw E, Oandasan I, Fowler N. CanMEDS-FM 2017: a competency framework for family physicians across the continuum. The College of Family Physicians of Canada. 2017. URL: https://www.cfpc.ca/CFPC/media/Resources/Medical-Education/CanMEDS-Family-Medicine-2017-ENG.pdf [accessed 2025-03-15]
- Crichton T, Schultz K, Lawrence K, Donoff M, Laughlin T, Brailovsky C, et al. Assessment objectives for certification in family medicine. College of Family Physicians of Canada. 2020. URL: https://www.cfpc.ca/CFPC/media/Resources/Examinations/Assessment-Objectives-for-Certification-in-FM-full-document.pdf [accessed 2025-03-15]
- Wojda T, Hoffman C, Jackson J, Conti T, Maier J. AI in healthcare: implications for family medicine and primary care. In: Artificial Intelligence in Medicine and Surgery—An Exploration of Current Trends, Potential Opportunities, and Evolving Threats—Volume 1. London. IntechOpen; 2023.
- Buja LM. Medical education today: all that glitters is not gold. BMC Med Educ. 2019;19(1):110. [FREE Full text] [CrossRef] [Medline]
- Jamieson S. State of the science: quality improvement of medical curricula—how should we approach it? Med Educ. 2023;57(1):49-56. [FREE Full text] [CrossRef] [Medline]
- Mcleod P, Steinert Y. Twelve tips for curriculum renewal. Med Teach. 2015;37(3):232-238. [CrossRef] [Medline]
- Jones R, Higgs R, de Angelis C, Prideaux D. Changing face of medical curricula. Lancet. 2001;357(9257):699-703. [CrossRef] [Medline]
- Sultan N, Torti J, Haddara W, Inayat A, Inayat H, Lingard L. Leadership development in postgraduate medical education: a systematic review of the literature. Acad Med. 2019;94(3):440-449. [CrossRef] [Medline]
- Hunter KA, Thomson B. A scoping review of social determinants of health curricula in post-graduate medical education. Can Med Ed J. 2019;10(3):e61-e71. [CrossRef]
- Hong DZ, Goh JL, Ong ZY, Ting JJQ, Wong MK, Wu J, et al. Postgraduate ethics training programs: a systematic scoping review. BMC Med Educ. 2021;21(1):338. [FREE Full text] [CrossRef] [Medline]
- Pritchard J, Alavian S, Soogoor A, Bartels S, Hall A. Global health competencies in postgraduate medical education: a scoping review and mapping to the CanMEDS physician competency framework. Can Med Educ J. 2023;14(1):70-79. [FREE Full text] [CrossRef] [Medline]
- Gupta A, Talavlikar R, Ng V, Chorny Y, Chawla A, Farrugia M, et al. Global health curriculum in family medicine: resident perspective. Can Fam Physician. 2012;58(2):143-146. [FREE Full text] [Medline]
- Drain PK, Primack A, Hunt DD, Fawzi WW, Holmes KK, Gardner P. Global health in medical education: a call for more training and opportunities. Acad Med. 2007;82(3):226-230. [CrossRef] [Medline]
- Runyan C, Savageau JA, Potts S, Weinreb L. Impact of a family medicine resident wellness curriculum: a feasibility study. Med Educ Online. 2016;21:30648. [FREE Full text] [CrossRef] [Medline]
- Penwell-Waines L, Runyan C, Kolobova I, Grace A, Brennan J, Buck K, et al. Making sense of family medicine resident wellness curricula: a Delphi study of content experts. Fam Med. 2019;51(8):670-676. [CrossRef] [Medline]
- Eckleberry-Hunt J, Van Dyke A, Lick D, Tucciarone J. Changing the conversation from burnout to wellness: physician well-being in residency training programs. J Grad Med Educ. 2009;1(2):225-230. [FREE Full text] [CrossRef] [Medline]
- Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44-56. [CrossRef] [Medline]
- Abbasgholizadeh Rahimi S, Légaré F, Sharma G, Archambault P, Zomahoun HTV, Chandavong S, et al. Application of artificial intelligence in community-based primary health care: systematic scoping review and critical appraisal. J Med Internet Res. 2021;23(9):e29839. [FREE Full text] [CrossRef] [Medline]
- Ethics and governance of artificial intelligence for health: WHO guidance. World Health Organization. 2021. URL: https://www.who.int/publications/i/item/9789240029200 [accessed 2025-03-12]
- Upshaw TL, Craig-Neil A, Macklin J, Gray CS, Chan TCY, Gibson J, et al. Priorities for artificial intelligence applications in primary care: a Canadian deliberative dialogue with patients, providers, and health system leaders. J Am Board Fam Med. 2023;36(2):210-220. [FREE Full text] [CrossRef] [Medline]
- Fowler N, Oandasan I, Wyman R. Preparing our future family physicians: an educational prescription for strengthening health care in changing times. College of Family Physicians of Canada. 2022. URL: https://www.cfpc.ca/CFPC/media/Resources/Education/AFM-OTP-Report.pdf [accessed 2025-03-15]
- Fowler N, Lemire F, Oandasan I, Wyman R. The evolution of residency training in family medicine: a Canadian perspective. Fam Med. 2021;53(7):595-598. [FREE Full text] [CrossRef] [Medline]
- Reznick R, Harris K, Horsley T, Hassani M. Artificial intelligence (AI) and emerging digital technologies. The Royal College of Physicians and Surgeons of Canada. 2020. URL: https://www.royalcollege.ca/content/dam/document/membership-and-advocacy/2020-task-force-report-on-ai-and-emerging-digital-technologies-e.pdf [accessed 2022-06-18]
- Topol E. The Topol review: preparing the health care workforce to deliver the digital future. National Health Service. 2019. URL: https://topol.hee.nhs.uk/wp-content/uploads/HEE-Topol-Review-2019.pdf [accessed 2023-04-25]
- AMA passes first policy recommendations on augmented intelligence. American Medical Association. 2018. URL: https://www.ama-assn.org/press-center/press-releases/ama-passes-first-policy-recommendations-augmented-intelligence [accessed 2023-04-25]
- AI for family medicine. College of Family Physicians of Canada. URL: https://cfpclearn.ca/ai-for-family-medicine [accessed 2025-03-12]
- Austin Z. Competency and its many meanings. Pharmacy (Basel). Apr 22, 2019;7(2):e12402. [FREE Full text] [CrossRef] [Medline]
- Liaw W, Kueper JK, Lin S, Bazemore A, Kakadiaris I. Competencies for the use of artificial intelligence in primary care. Ann Fam Med. 2022;20(6):559-563. [FREE Full text] [CrossRef] [Medline]
- Stabback P. What makes a quality curriculum? UNESCO. 2016. URL: https://unesdoc.unesco.org/ark:/48223/pf0000243975 [accessed 2022-07-10]
- Training tools for curriculum development: developing and implementing curriculum frameworks. UNESCO International Bureau of Education. 2017. URL: https://unesdoc.unesco.org/ark:/48223/pf0000250052 [accessed 2022-07-10]
- Chan KS, Zary N. Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Med Educ. 2019;5(1):e13930. [FREE Full text] [CrossRef] [Medline]
- Tolentino R, Baradaran A, Gore G, Pluye P, Abbasgholizadeh-Rahimi S. Curriculum frameworks and educational programs in AI for medical students, residents, and practicing physicians: scoping review. JMIR Med Educ. 2024;10:e54793. [FREE Full text] [CrossRef] [Medline]
- Tolentino R, Baradaran A, Gore G, Pluye P, Abbasgholizadeh-Rahimi S. Curriculum frameworks and educational programs in artificial intelligence for medical students, residents, and practicing physicians: a scoping review protocol. JBI Evid Synth. 2023;21(7):1477-1484. [CrossRef] [Medline]
- Redwood-Campbell L, Pakes B, Rouleau K, MacDonald CJ, Arya N, Purkey E, et al. Developing a curriculum framework for global health in family medicine: emerging principles, competencies, and educational approaches. BMC Med Educ. 2011;11:46. [FREE Full text] [CrossRef] [Medline]
- Masters K. Artificial intelligence developments in medical education: a conceptual and practical framework. MedEdPublish (2016). 2020;9:239. [FREE Full text] [CrossRef] [Medline]
- Valikodath NG, Cole E, Ting DSW, Campbell JP, Pasquale LR, Chiang MF, et al. Impact of artificial intelligence on medical education in ophthalmology. Transl Vis Sci Technol. 2021;10(7):14. [FREE Full text] [CrossRef] [Medline]
- Obadeji A. Health professions education in the 21st century: a contextual curriculum framework for analysis and development. J Contemp Med Edu. 2019;9(1):34. [CrossRef]
- O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for Reporting Qualitative Research: a synthesis of recommendations. Acad Med. 2014;89(9):1245-1251. [FREE Full text] [CrossRef] [Medline]
- Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984;74(9):979-983. [CrossRef] [Medline]
- Black N, Murphy M, Lamping D, McKee M, Sanderson C, Askham J, et al. Consensus development methods: a review of best practice in creating clinical guidelines. J Health Serv Res Policy. 1999;4(4):236-248. [CrossRef] [Medline]
- Vankova D, Videnova J. Delphi technique for curriculum development. 2019. Presented at: 12th annual International Conference of Education, Research and Innovation; November 11-13, 2019:6167-6171; Seville, Spain. URL: https://library.iated.org/view/VANKOVA2019DEL
- Lewthwaite S, Nind M. Teaching research methods in the social sciences: expert perspectives on pedagogy and practice. Br J Educ Stud. 2016;64(4):413-430. [CrossRef]
- Mangold K, Bartell T, Doobay-Persaud A, Adler M, Sheehan K. Expert consensus on inclusion of the social determinants of health in undergraduate medical education curricula. Acad Med. 2019;94(9):1355-1360. [CrossRef] [Medline]
- Hart A, Romney D, Sarin R, Mechanic O, Hertelendy A, Larson D, et al. Developing telemedicine curriculum competencies for graduate medical education: outcomes of a modified Delphi process. Acad Med. 2022;97(4):577-585. [CrossRef] [Medline]
- Hsu T, Kessler E, Parker I, Dale W, Gajra A, Holmes H, et al. Identifying geriatric oncology competencies for medical oncology trainees: a modified Delphi consensus study. Oncologist. 2020;25(7):591-597. [FREE Full text] [CrossRef] [Medline]
- Evans C. The use of consensus methods and expert panels in pharmacoeconomic studies. Practical applications and methodological shortcomings. Pharmacoeconomics. 1997;12(2 Pt 1):121-129. [CrossRef] [Medline]
- Duncan EAS, Nicol MM, Ager A. Factors that constitute a good cognitive behavioural treatment manual: a delphi study. Behav Cognit Psychother. 1999;32(2):199-213. [CrossRef]
- Mead D, Moseley L. The use of the Delphi as a research approach. Nurse Res. 2001;8(4):4-23. [CrossRef]
- Framework for research ethics. UK Research and Innovation—Economic and Social Research Council. URL: https://www.ukri.org/councils/esrc/guidance-for-applicants/research-ethics-guidance/framework-for-research-ethics/ [accessed 2025-03-25]
- Tri-council policy statement: ethical conduct for research involving humans. Government of Canada. 2018. URL: https://ethics.gc.ca/eng/policy-politique_tcps2-eptc2_2022.html [accessed 2025-04-07]
- Khodyakov D, Hempel S, Rubenstein L, Shekelle P, Foy R, Salem-Schatz S, et al. Conducting online expert panels: a feasibility and experimental replicability study. BMC Med Res Methodol. 2011;11:174. [FREE Full text] [CrossRef] [Medline]
- Hsieh H, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277-1288. [CrossRef] [Medline]
- Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107-115. [CrossRef] [Medline]
- Hamid AYS, Chandra YA, Putri AF, Wakhid A, Falahaini A, Yulianingsih Y. Sustainable disaster risk reduction training model for nurses: a descriptive qualitative approach. Nurse Educ Pract. 2023;69:103616. [CrossRef] [Medline]
- Itchhaporia D. The evolution of the quintuple aim: health equity, health outcomes, and the economy. J Am Coll Cardiol. 2021;78(22):2262-2264. [FREE Full text] [CrossRef] [Medline]
- Nundy S, Cooper LA, Mate KS. The quintuple aim for health care improvement: a new imperative to advance health equity. JAMA. 2022;327(6):521-522. [CrossRef] [Medline]
- Chatterjee D, Corral J. How to write well-defined learning objectives. J Educ Perioper Med. 2017;19(4):E610. [FREE Full text] [Medline]
- Badyal D, Singh T. Learning theories: the basics to learn in medical education. Int J Appl Basic Med Res. 2017;7(Suppl 1):S1-S3. [FREE Full text] [CrossRef] [Medline]
- Kirkpatrick D, Kirkpatrick J. Evaluating Training Programs: The Four Levels. Oakland, CA. Berrett-Koehler Publishers; 2006.
- Stogiannos N, Gillan C, Precht H, Reis CSD, Kumar A, O'Regan T, et al. A multidisciplinary team and multiagency approach for AI implementation: a commentary for medical imaging and radiotherapy key stakeholders. J Med Imaging Radiat Sci. 2024;55(4):101717. [CrossRef] [Medline]
- Knopp MI, Warm EJ, Weber D, Kelleher M, Kinnear B, Schumacher DJ, et al. AI-enabled medical education: threads of change, promising futures, and risky realities across four potential future worlds. JMIR Med Educ. 2023;9:e50373. [FREE Full text] [CrossRef] [Medline]
- Kueper JK, Emu M, Banbury M, Bjerre LM, Choudhury S, Green M, et al. Artificial intelligence for family medicine research in Canada: current state and future directions: report of the CFPC AI Working Group. Can Fam Physician. 2024;70(3):161-168. [FREE Full text] [CrossRef] [Medline]
- Inventory of Artificial Intelligence Resources in Family Medicine Education. Rahimi’s Lab. URL: https://rahimislab.ca/inventory
- Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT—reshaping medical education and clinical management. Pak J Med Sci. 2023;39(2):605-607. [FREE Full text] [CrossRef] [Medline]
- Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare (Basel). 2023;11(6):887. [FREE Full text] [CrossRef] [Medline]
- Ossa LA, Rost M, Lorenzini G, Shaw DM, Elger BS. A smarter perspective: learning with and from AI-cases. Artif Intell Med. 2023;135:102458. [FREE Full text] [CrossRef] [Medline]
- Sauder M, Tritsch T, Rajput V, Schwartz G, Shoja MM. Exploring generative artificial intelligence-assisted medical education: assessing case-based learning for medical students. Cureus. 2024;16(1):e51961. [FREE Full text] [CrossRef] [Medline]
- Weidener L, Fischer M. Teaching AI ethics in medical education: a scoping review of current literature and practices. Perspect Med Educ. 2023;12(1):399-410. [FREE Full text] [CrossRef] [Medline]
- Laupichler MC, Hadizadeh DR, Wintergerst MWM, von der Emde L, Paech D, Dick EA, et al. Effect of a flipped classroom course to foster medical students' AI literacy with a focus on medical imaging: a single group pre-and post-test study. BMC Med Educ. 2022;22(1):803. [FREE Full text] [CrossRef] [Medline]
- Krive J, Isola M, Chang L, Patel T, Anderson M, Sreedhar R. Grounded in reality: artificial intelligence in medical education. JAMIA Open. 2023;6(2):ooad037. [FREE Full text] [CrossRef] [Medline]
- Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, et al. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Intern Med. 2023;183(6):589-596. [FREE Full text] [CrossRef] [Medline]
- Jain A, Way D, Gupta V, Gao Y, de Oliveira Marinho G, Hartford J, et al. Development and assessment of an artificial intelligence-based tool for skin condition diagnosis by primary care physicians and nurse practitioners in teledermatology practices. JAMA Netw Open. 2021;4(4):e217249. [FREE Full text] [CrossRef] [Medline]
- Lee C, Britto S, Diwan K. Evaluating the impact of artificial intelligence (AI) on clinical documentation efficiency and accuracy across clinical settings: a scoping review. Cureus. 2024;16(11):e73994. [CrossRef] [Medline]
- Ahmed MI, Spooner B, Isherwood J, Lane M, Orrock E, Dennison A. A systematic review of the barriers to the implementation of artificial intelligence in healthcare. Cureus. 2023;15(10):e46454. [FREE Full text] [CrossRef] [Medline]
- Hassan M, Kushniruk A, Borycki E. Barriers to and facilitators of artificial intelligence adoption in health care: scoping review. JMIR Hum Factors. 2024;11:e48633. [FREE Full text] [CrossRef] [Medline]
- Nair M, Svedberg P, Larsson I, Nygren JM. A comprehensive overview of barriers and strategies for AI implementation in healthcare: mixed-method design. PLoS One. 2024;19(8):e0305949. [FREE Full text] [CrossRef] [Medline]
- Singh RP, Hom GL, Abramoff MD, Campbell JP, Chiang MF, AAO Task Force on Artificial Intelligence. Current challenges and barriers to real-world artificial intelligence adoption for the healthcare system, provider, and the patient. Transl Vis Sci Technol. 2020;9(2):45. [FREE Full text] [CrossRef] [Medline]
- Terry AL, Kueper JK, Beleno R, Brown JB, Cejic S, Dang J, et al. Is primary health care ready for artificial intelligence? What do primary health care stakeholders say? BMC Med Inform Decis Mak. 2022;22(1):237. [FREE Full text] [CrossRef] [Medline]
- Perkins M, Pregowska A. The role of artificial intelligence in higher medical education and the ethical challenges of its implementation. AIH. 2025;2(1):1-13. [FREE Full text] [CrossRef]
- Jiang L, Wu Z, Xu X, Zhan Y, Jin X, Wang L, et al. Opportunities and challenges of artificial intelligence in the medical field: current application, emerging problems, and problem-solving strategies. J Int Med Res. 2021;49(3):3000605211000157. [FREE Full text] [CrossRef] [Medline]
- Yin J, Ngiam KY, Teo HH. Role of artificial intelligence applications in real-life clinical practice: systematic review. J Med Internet Res. 2021;23(4):e25759. [FREE Full text] [CrossRef] [Medline]
- Paranjape K, Schinkel M, Nannan Panday R, Car J, Nanayakkara P. Introducing artificial intelligence training in medical education. JMIR Med Educ. 2019;5(2):e16048. [FREE Full text] [CrossRef] [Medline]
- Grunhut J, Wyatt AT, Marques O. Educating future physicians in artificial intelligence (AI): an integrative review and proposed changes. J Med Educ Curric Dev. 2021;8:23821205211036836. [FREE Full text] [CrossRef] [Medline]
- Lee J, Wu AS, Li D, Kulasegaram KM. Artificial intelligence in undergraduate medical education: a scoping review. Acad Med. 2021;96(11S):S62-S70. [CrossRef] [Medline]
- Waljee AK, Higgins PDR. Machine learning in medicine: a primer for physicians. Am J Gastroenterol. 2010;105(6):1224-1226. [CrossRef] [Medline]
- Blease C, Kaptchuk TJ, Bernstein MH, Mandl KD, Halamka JD, DesRoches CM. Artificial intelligence and the future of primary care: exploratory qualitative study of UK general practitioners' views. J Med Internet Res. 2019;21(3):e12802. [FREE Full text] [CrossRef] [Medline]
- Irfan F. Artificial intelligence: help or hindrance for family physicians? Pak J Med Sci. 2021;37(1):288-291. [FREE Full text] [CrossRef] [Medline]
- Mousavi M, Shafiee S, Harley JM, Cheung JCK, Abbasgholizadeh Rahimi S. Performance of generative pre-trained transformers (GPTs) in certification examination of the College of Family Physicians of Canada. Fam Med Com Health. May 28, 2024;12(Suppl 1):e002626. [CrossRef] [Medline]
Abbreviations
| AI: artificial intelligence |
| AIFM-ed: Artificial Intelligence Training in Postgraduate Family Medicine Education |
| CanMEDS: Canadian Medical Education Directives for Specialists |
| CFPC: College of Family Physicians of Canada |
| PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews |
| SRQR: Standards for Reporting Qualitative Research |
Edited by B Lesselroth; submitted 24.09.24; peer-reviewed by K Thompson, G Evangelinos; comments to author 12.12.24; revised version received 04.02.25; accepted 25.02.25; published 25.04.25.
Copyright©Raymond Tolentino, Fanny Hersson-Edery, Mark Yaffe, Samira Abbasgholizadeh-Rahimi. Originally published in JMIR Medical Education (https://mededu.jmir.org), 25.04.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on https://mededu.jmir.org/, as well as this copyright and license information must be included.