To understand the educational strategies that were most effective in fostering long-term change in the learners and the influence of the course on the students’ practice, we employed methodological rigor by gathering data from three different sources: (1) precourse survey, (2) five written course assignments, and (3) semistructured interviews.

A precourse survey was administered to capture participants’ baseline perspectives on AI and their expectations for the course. The instrument (Multimedia Appendix 2) included two closed-choice items on participants’ emotional responses toward AI (eg, excitement, enthusiasm, and apprehension) and their prior knowledge and experience with AI tools. These included open-ended response items for participants to share their motivations for enrolling in the course. Participants were also asked to indicate any concerns they had about the course, the broader topic of AI, and the specific areas of AI in HPE they were most interested in exploring. This survey served to contextualize learners’ starting points and to inform course design.

This closed survey was administered online via Google Forms (Google LLC, 2024) through the course learning management system, only to the learners in the course. This was a mandatory survey that needed to be completed as part of the course requirements [28].

Student assignments were included as a key data source to examine how participants engaged with and applied course content in authentic tasks. The assignments required learners to analyze AI models, experiment with GenAI tools, design educational resources, and reflect on leadership and implementation strategies in their own contexts (Multimedia Appendix 1). As these tasks moved beyond knowledge recall to application, synthesis, and reflection, they provided a rich window into how participants made sense of AI concepts and integrated them into their professional roles.

Analyzing these products allowed us to assess not only immediate understanding but also evidence of higher-order thinking, creativity, and transfer of learning. Moreover, the assignments, providing a naturalistic record of learners’ decision-making, challenges, and evolving perspectives, were an essential complement to survey and interview data for evaluating the course’s impact on long-term professional practice.

We conducted 45‐60-minute, semistructured interviews six months after course completion to assess the transfer of learning beyond the immediate course period. This delayed timing allowed us to explore how learners had sustained, adapted, or applied their learning in real-world professional contexts, including educational, administrative, research, and clinical practice (eg, self-reported Kirkpatrick level 3 behavioral changes and expanding connectivism learning networks). The ten semistructured interview questions focused on the research aim and question, including post-course application of learning (Multimedia Appendix 3).

We sent emails to all first cohort participants (nine learners), using IRB-approved recruitment emails that briefly described the interview purpose and research study. Only learners from the first cohort were invited for the interviews as they met the required longitudinal distance of six months since completion of the course.

Each learner voluntarily attended the interview in response to recruitment emails. They were also informed at the beginning of the interview of the option to end the session at any point. At the start of every interview, JGP reviewed the consent process with the participant and clarified the (1) purpose and nature of the interview; (2) voluntary nature of the interview and option to opt-out; (3) maintenance of confidentiality through deidentification and secure storage of interview content; (4) recording of the interviews for auto-transcription and recall purposes, but that it could be deleted if desired; (5) risks associated with the interview; and (6) contact information for the researchers should concerns arise.

Privacy associated with the interviews was achieved through virtual interviews conducted by JGP in a quiet, private office. Interviews were conducted using Google Meet (Google LLC, 2024) with the transcription feature enabled. JGP reviewed the transcripts and recordings for accuracy and clarifications or corrections.

All data sources, survey responses, assignments, and interview transcripts were deidentified by a research assistant and assigned a pseudonym. The data were secured as Google Docs (Google LLC, 2024) files on the USU Google secure server. Members of the team accessed the files on encrypted password-protected computers. This deidentified data was analyzed by the researchers.

The researchers used ATLAS.ti (ATLAS.ti Scientific Software Development GmbH; 2024) to track codes and quotations. The authors acknowledge limited use of multimodal GenAI tools for two discrete tasks, as outlined in Multimedia Appendix 4. The researchers used OpenAI ChatGPT-4o (paid version) to cocreate the subthemes from the researcher-identified codes. JGP used Google NotebookLM (Google LLC and DALL-E-3, Open AI) to cocreate Figure 1, using the manuscript as the source data with author-provided meta-prompting.

The researchers aggregated the survey data into a Google Sheet (Google LLC, 2024) for analysis. The researchers conducted a simple descriptive analysis of the data for the two close-response questions and the open-ended question: How and what AI technologies have you used? The researchers treated the responses to the other three open-ended questions as textual data and thematically analyzed them using the thematic analysis process by Braun and Clarke.

The researchers treated the course assignments and interview transcripts as textual data and thematically analyzed them using the 6-step thematic analysis process by Braun and Clarke. In step 1 of the analysis, authors JGP and JM reviewed the data for familiarization and identified initial patterns within the data. In step 2, JGP and JM generated the initial codes (Table 1). JGP and JM then further refined the codes. JGP, JM, and AS generated, reviewed, and refined the themes in steps 3, 4, and 5 (Table 1). The research team addressed all discrepancies through discussion until they reached consensus. Finally, we selected compelling extracts that were rechecked against the themes and research question. Table 1 details our movement from developing codes, which the authors combined into subthemes, and then themes. This approach was especially effective because it allowed us to follow the thread and gain a deeper understanding of the relationships between these elements [29].

Unlike the survey and assignments, which captured immediate reactions and behavioral applications within course boundaries, the interviews provided insight into longer-term engagement, barriers to implementation, and the integration of AI tools into practice. By focusing on learners’ reflections after an extended interval, these interviews offered a critical perspective on the durability and impact of CPD in fostering meaningful change. Together, these three data sources provided a comprehensive view of the learners’ journey from entry into the course through to longer-term professional application. This multimethod approach allowed for a richer understanding of how CPD in AI influenced Kirkpatrick Level 1‐3 learning in written and oral samples.

To establish the rigor of our findings, we intentionally transitioned away from the positivist concept of data saturation, which is frequently incompatible with interpretivist research. Instead, guided by recent frameworks for assessing qualitative data adequacy [30] and the conceptualization of information power by LaDonna et al [31], the researchers evaluated data adequacy based on the specific characteristics of this study’s design and the resulting artifacts. Sufficiency was assessed iteratively during the analysis. As this study possessed a narrowly focused research question, a highly specific participant sample (educators who completed this exact experiential course), and interview dialogue characterized by rich conceptual depth, the dataset was deemed to have high information power [31]. Through this lens, the research team determined that 105 written assignments from 21 participants and 5 in-depth interviews achieved the theoretical sufficiency needed to address the exploratory research aims. In alignment with this framework, we prioritized the richness, resonance, and conceptual depth of the data over absolute sample size.

By using the experiential learning theory by Kolb as an a priori theoretical lens, we sharpened our analytical focus and enhanced the evidentiary value of this study. This iterative analytical process allowed us to move beyond surface-level descriptions, reaching a point of analytical sufficiency where the findings offer a robust, conceptually thoughtful account of how hands-on experimentation and reflective practice dismantle precourse misconceptions, ultimately fostering the confidence and trust required for institutional AI implementation.

JGP is a nuclear radiologist passionate about AI technologies for educational and clinical practice. He believes AI tools can help educators be more effective and efficient in their teaching efforts, and that these tools, when used ethically and appropriately, can increase creativity and critical thought about complex problems. His views are reflected in the scope, development, and reporting of this study. He helped in the initial development of the first HPE course, but he did not act as an official instructor or grader for either course to reduce research bias and prevent undue influence during the interviews. He performed the interviews described below and participated as one of the coders during the analysis. To reduce bias, he did not review the presurvey or course assignments until independent coding analysis began. Through open-ended, semistructured interview questions, he attempted to mitigate personal influence on the interviews.

JM is an educational psychologist who uses mixed-methods research to explore metacognition and emotion in medical education. Her expertise fundamentally shaped this study’s methodology, analysis, and interpretation to prioritize participants’ metacognitive reflections and emotional responses to AI technologies. She participated as an independent coder during the analysis. She was not involved in the course in any way and only participated in the research phases of this study.

AS is an online educator and researcher. Her research straddles the domains of adult learning, online learning, and health education. We recognize our views have a critical implication on the ethical and effective integration of AI in educational settings. She developed the HPE course and was the primary instructor and grader for both course cohorts. To reduce bias, she did not participate in data collection or initial coding. She participated in a subsequent thematic analysis to avoid influencing the findings of this study.

Overall, 15 of 21 (71%) respondents for the precourse survey noted feeling apprehension about the use of AI, and 14% (n=3) mentioned fear. These feelings support the identified codes aligning with the “AI misconceptions breed fear” theme (Table 1). A total of 52% (n=11) and 67% (n=14) of respondents identified feeling excitement and enthusiasm for the use of AI, respectively. Only 10% (n=2) of participants reported having had no exposure to AI before the course. Few learners (n=1) in the first cohort reported having used AI in some capacity, compared with the second cohort (n=9), resulting in a total of 10 (48%) participants for both cohorts. An additional 39% (n=9) of learners in both cohorts reported basic knowledge about AI. The course was an elective; therefore, participants were interested in AI and curious about it when they enrolled in the course.

In their precourse questionnaire, 2 interviewees reported no prior knowledge of AI, 2 interviewees reported basic limited AI knowledge, and 1 interviewee reported some experience with using AI tools or applications. All demonstrated some enthusiasm or excitement about AI, but 3 interviewees also reported apprehension or uncertainty (Table 1). All reported taking the course to learn more about AI technologies and how they could be used for health care and education, with P6 describing his motivation as “options are either to hide my head in the sand and become obsolete, or develop proficiency and continue to grow along with the practice of medicine.”

Universally, learners remarked on the misconceptions that they had about AI before the course (Table 1). P19 (A) expressed it best as, “if I had been asked frankly what I perceived AI to be, my answer was often drawn to 30-year-old movies like Terminator or Maximum Overdrive.” Misconceptions included the idea that using AI in education and practice was unethical, that AI tools were infallible, that AI would replace them as physicians or educators, that AI would not be useful in academic or clinical practice, and that they were not currently using AI in their daily lives. As P8 (I) stated, “I was so afraid of the ethical dilemmas like ‘Oh my God, if these words aren’t mine, is that okay?’” These misconceptions about AI lead learners to fear AI and avoid its use in their personal and professional lives.

Learners also expressed fears that “the potential risks…of AI in medical education are limitless. The dangers of plagiarism, hallucinations, and bias are worrisome” (P10 (A)). As expected of health professions educators, many expressed fears about learning and education, noting that “trainees would use it to complete assignments, personal statements, etc, essentially making these ‘useless’ in evaluation” (P1 (A)). The learners described AI fears about maintaining work originality, inaccuracies, or “hallucinations,” “deep fakes,” or other nefarious uses of AI technologies, patient safety, bias, inability to keep up with the pace of AI technologies, changes to their professional practice or losing their jobs, an inability to learn something new or technologically challenging, and a fear of the unknown.

Education, however, was able to create a shift in the learners’ perspectives (Table 1). Many learners described that their initial fears and misconceptions were reduced or eliminated due to attending the course: “I’m like, ‘God, I have to engage’ and I’m like, ‘How do I do this?’ And so, it really helped me sort of start my journey” (P2 (I)). As learners engaged experientially with the content in the course, they began to recognize the misconceptions they had previously held about AI technologies. One of the first realizations for many learners was that AI tools are integrated into almost every aspect of modern life. P1 (A) clearly articulated “I erroneously thought that I had almost no interaction with AI…I was surprised to learn that I interacted with AI on a daily basis…”

Many remarked how they would not have imagined that they would have used AI technologies before the course as an expert health care educator, but as they worked hands-on with the tools that had quickly changed. For instance, as P13 (A) reported, as “I reflect on my journey through this course on artificial intelligence (AI) in health professions education, I recognize the significant evolution in both my understanding of AI and my enthusiasm for integrating it into my professional endeavors.” They also realized the imperfections, hallucinations, and errors that the tools produced. For example, participating firsthand in the image creation activities, they realized that AI tools could not draw a perfect hand or an image with text on it. As participant P7 (A) stated, “one interesting thing is that the AI has a lot of trouble with hands…”

Participants actively discussed ethical concerns with faculty, guest lecturers, and peers, and began to describe a shift in understanding about the way they perceived ethical uses of AI in their daily lives, academic work, and clinical practice. It shifted from a concrete view of ethics to a more nuanced perspective on where, when, and how AI tools could be used. Furthermore, many described how their concerns of naivete about AI technologies caused them to be hesitant to learn or use the tools.

They described how the class provided opportunities to remove self-imposed limitations to their use, bravery to use the tools, and the realization that they were in a similar position of novice status relative to their peers and other health care professionals. Learner P6 (I) stated that AI technologies “don’t require other people, they just require me to be brave enough to create an account and yeah, screw up…” Importantly, many learners described a change in their affect and emotional response to AI technologies. The AI fears described before the class gave way to emotions of excitement, enjoyment, and hope about AI technologies and their potential to improve their lives and careers as health care professionals and educators. For instance, P14 (A) noted, “I think the biggest surprise was how much this course served as a paradigm shift for me. How does an entire healthcare system or medical education program shift from what amounts to written rules to algorithms and generative ways of proceeding?” As learners’ perspectives and emotions around AI shifted, they described how they would reach out to their social circles to share, teach, and use their newly developed knowledge about AI.

This excitement in the AI tools shifted to a natural tendency to share that knowledge with their social circles in peer-based approaches (Table 1). P19 (A) described this excitement about sharing with their social circles as follows, “I met each assignment with such excitement and shared my new knowledge with my family and my colleagues at work…I became a weekly feature on our team huddles, sharing my latest knowledge and providing information on how it could be sourced in an organization that was not currently set up for the use of AI.” For many, the closest social circle was their own family members. Learners described teaching their children about the ethical and effective use of AI tools for their schoolwork. They shared how they were able to appropriately modify their response to their children’s use of AI through a more nuanced understanding of the ethical uses of AI. For some individuals, their family members were more knowledgeable about AI technologies, and so they would reach out to learn more from them.

Additionally, learners discussed how they would learn from their peers in the class. They expressed how it was good to brainstorm ideas, discuss challenges, and develop new uses for the AI tools discussed in class. P7 (A) stated that “I certainly was impacted by peer learning and just seeing and reading about some of what my peers were doing with AI.” Learners also described peer-based learning with their trainees. They shared how their own health care trainees would teach them about new AI technologies, and how they would instruct them on effective and ethical AI tool use. In describing this experience, P18 (A) noted, “I believe student collaboration in particular will help me stay abreast of AI development since the younger learners are extremely forward leaning in that direction.” As a component of this social circle-based learning, some learners described the use of social media in helping to develop more knowledge about AI tools. The socialized learning of AI technologies among the learners, their families, and their peers led to a natural progression of technology implementation in their professional careers, development of programmatic and institutional education, and advocacy for AI technologies in their spheres of influence.

As learners learned about, experimented with, and used the AI tools in class, they began to implement the AI tools in their daily lives (Table 1). Many reported initially using the tools for simple administrative tasks, such as writing emails and planning vacations. As educators, they also turned to the implementation of AI tools in basic educational activities, such as building curricula, summarizing clinical information, revising curriculum vitae, developing clinical templates, writing questions, generating learning objectives, developing patient scenarios for education, and developing a second opinion. As P1 (A) explained, “The utility of these within my role is almost endless… from assisting me with drafting initial educational components more efficiently to generating images for educational presentations to more effectively drafting communications I send to staff (ie, emails, newsletters, etc).”

Many learners described how the AI tools helped to reduce the cognitive load and time spent developing and crafting a project or product. They described how AI tools helped reduce the blank page anxiety that comes with starting a new project. As P6 (I) stated, “It was really useful for that, overcoming, what, Picasso described as the tyranny of the blank canvas. Going from ‘How do I even get started on this’ to say ‘Okay, I’ve got a block architecture. I can modify that as needed.’” In reducing the activation barrier for learners, they were able to begin new endeavors more confidently. They noticed how the AI tools made their jobs easier and time-efficient.

P8 (I) described a particularly impactful experience with not knowing what to say or do for a learner who was struggling with mental health concerns. The learner used AI tools to help craft an appropriate message, “we had a resident who was having a really rough time with mental health and I wanted to send them a message and I wasn’t sure of the right words to say in this situation…it was super helpful just helping me come up with a nice message to send.”

Some learners began to develop more complex applications of the AI tools that impacted more than just themselves and their small social circles. They worked with others to develop educational courses, curricula, and tools for clerkship rotations. P21 (A) reported one such presentation: “I recently developed and delivered a grand rounds on the topic of AI. The focus of this presentation was on the evolution of AI, evolution of medicine, and current AI being used within [my organization].” With increasing understanding, confidence, and real-world application of the AI tools, learners felt more empowered to begin advocating for AI technologies in their programs and institutions. Many acknowledged that there were challenges with implementing AI tools in their institutions and in health care practice, but they still sought ways to promote knowledge of AI technologies, the ethics of AI, and the application of AI technologies in daily practice.

Despite institutional challenges, P1 (A and I) described developing and organizing an entire day devoted to discussing the application of AI technologies in education and clinical practice. She worked with AI experts at her institution and educational leaders to build support and invited speakers to come speak about their work with AI technologies. P8 (I) described how she worked with fellow medical student site directors to develop a specific AI tool that would provide improved, targeted, narrative feedback for medical students working on specific entrustable professional activities. These were then distributed to medical student sites nationwide. P6 worked with AI experts at his institution to develop workshops centered on learning about AI technologies and how they could be used in clinical practice.

A summary of these educational advocacy approaches was best conceptualized as follows:

The AI course empowered some educators to learn, explore, and experience AI tools, which they shared socially, building communities that impacted their programs and institutions. As learner P12 (A) indicated, “we all must become AI leaders…”