This paper is in the following e-collection/theme issue:

Reviews in Medical Education (52) Consumer & Patient Education and Shared-Decision Making (636) Continuing Medical Education (CME) for Allied Health Professionals (119) New Methods and Approaches in Medical Education (192)

Published on in Vol 8, No 4 (2022): Oct-Dec

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/42033, first published .
Effectiveness of Shared Decision-making Training Programs for Health Care Professionals Using Reflexivity Strategies: Secondary Analysis of a Systematic Review

Effectiveness of Shared Decision-making Training Programs for Health Care Professionals Using Reflexivity Strategies: Secondary Analysis of a Systematic Review

Effectiveness of Shared Decision-making Training Programs for Health Care Professionals Using Reflexivity Strategies: Secondary Analysis of a Systematic Review

Authors of this article:

Ndeye Thiab Diouf1, 2, 3 Author Orcid Image ;   Angèle Musabyimana1, 2, 3 Author Orcid Image ;   Virginie Blanchette1, 2, 4 Author Orcid Image ;   Johanie Lépine1, 2 Author Orcid Image ;   Sabrina Guay-Bélanger1, 2 Author Orcid Image ;   Marie-Claude Tremblay2, 5, 6 Author Orcid Image ;   Maman Joyce Dogba2, 5, 6 Author Orcid Image ;   France Légaré1, 2, 6 Author Orcid Image
Article Authors Cited by (4) Tweetations (2) Metrics

    Review

    1Canada Research Chair in Shared Decision Making and Knowledge Translation (Tier 1), Quebec, QC, Canada

    2VITAM - Centre de recherche en santé durable, Centre intégré universitaire de santé et services sociaux de la Capitale-Nationale, Quebec, QC, Canada

    3Department of Community Health, Faculty of Nursing and Faculty of Medicine, Université Laval, Quebec, QC, Canada

    4Department of Human Kinetic and Podiatric Medicine, Université du Québec à Trois-Rivières, Trois-Rivières, QC, Canada

    5Office of Education and Continuing Professional Education, Université Laval, Quebec, QC, Canada

    6Department of Family Medicine and Emergency Medicine, Faculty of Medicine, Université Laval, Quebec, QC, Canada

    Corresponding Author:

    France Légaré, BSc, MSc, MD, PhD, CCFP

    VITAM - Centre de recherche en santé durable

    Centre intégré universitaire de santé et services sociaux de la Capitale-Nationale

    2480 chemin de la canardière

    Quebec, QC, G1G 2G1

    Canada

    Phone: 1 418 663 5313

    Email: france.legare@fmed.ulaval.ca


    Background: Shared decision-making (SDM) leads to better health care processes through collaboration between health care professionals and patients. Training is recognized as a promising intervention to foster SDM by health care professionals. However, the most effective training type is still unclear. Reflexivity is an exercise that leads health care professionals to question their own values to better consider patient values and support patients while least influencing their decisions. Training that uses reflexivity strategies could motivate them to engage in SDM and be more open to diversity.

    Objective: In this secondary analysis of a 2018 Cochrane review of interventions for improving SDM by health care professionals, we aimed to identify SDM training programs that included reflexivity strategies and were assessed as effective. In addition, we aimed to explore whether further factors can be associated with or enhance their effectiveness.

    Methods: From the Cochrane review, we first extracted training programs targeting health care professionals. Second, we developed a grid to help identify training programs that used reflexivity strategies. Third, those identified were further categorized according to the type of strategy used. At each step, we identified the proportion of programs that were classified as effective by the Cochrane review (2018) so that we could compare their effectiveness. In addition, we wanted to see whether effectiveness was similar between programs using peer-to-peer group learning and those with an interprofessional orientation. Finally, the Cochrane review selected programs that were evaluated using patient-reported or observer-reported outcome measurements. We examined which of these measurements was most often used in effective training programs.

    Results: Of the 31 training programs extracted, 24 (77%) were interactive, among which 10 (42%) were considered effective. Of these 31 programs, 7 (23%) were unidirectional, among which 1 (14%) was considered effective. Of the 24 interactive programs, 7 (29%) included reflexivity strategies. Of the 7 training programs with reflexivity strategies, 5 (71%) used a peer-to-peer group learning strategy, among which 3 (60%) were effective; the other 2 (29%) used a self-appraisal individual learning strategy, neither of which was effective. Of the 31 training programs extracted, 5 (16%) programs had an interprofessional orientation, among which 3 (60%) were effective; the remaining 26 (84%) of the 31 programs were without interprofessional orientation, among which 8 (31%) were effective. Finally, 12 (39%) of 31 programs used observer-based measurements, among which more than half (7/12, 58%) were effective.

    Conclusions: Our study is the first to evaluate the effectiveness of SDM training programs that include reflexivity strategies. Its conclusions open avenues for enriching future SDM training programs with reflexivity strategies. The grid developed to identify training programs that used reflexivity strategies, when further tested and validated, can guide future assessments of reflexivity components in SDM training.

    JMIR Med Educ 2022;8(4):e42033

    doi:10.2196/42033

    Keywords

    shared decision-making; reflexivity; training; health care professionals; implementation 


    Background

    There is increasing recognition of the ethical imperative to support patients to be engaged in their care, especially in health-related decisions. Shared decision-making (SDM) is a collaborative process whereby health care professionals support patients in making decisions that are informed by the best evidence and by what matters to them [1]. SDM improves the health care experiences of patients and health care professionals and leads to better health care processes, patient outcomes, and lower health costs [2-5]. SDM is the best practice for informed consent and is fundamental to patient- and family-centered care [6]. However, SDM has not yet been widely implemented in clinical practice because of several perceived barriers [7]. In some contexts, SDM implementation is encouraged by health policies, but certain challenges related to patients, patient–health care professional relationships, and organizational factors remain in the way of its concrete adoption [8].

    Reflexivity

    Training programs for health care professionals [9,10] are believed to be crucial to the implementation of SDM. However, SDM training programs for health care professionals are highly heterogeneous [10], and we still do not know what makes them effective [9]. One promising approach is reflexivity, which has been shown to increase health care professionals’ willingness to be more engaged in the health care offer and collaborate with other professionals [11]. SDM training programs that use reflexivity strategies have the potential to be more effective than those that do not [12].

    Reflexivity is a form of learning based on reflection on one’s experiences applied in a professional or interprofessional context [13]. Concepts related to reflexivity are found in various disciplinary fields under different names. Writing about reflective practice (a practice based on reflexivity), Schon [14] states that the concept entails critical thinking concerning the actions and stances one takes [14,15]. In health care practices, being reflexive can mean different things: acknowledging and questioning the power dynamics implicit in a health care encounter; identifying the assumptions that underlie a health care situation; or examining the influences, such as values and beliefs, that shape health care practices [13].

    On the basis of this multiplicity of definitions, Sandars [16] developed a guide that classifies them into 3 main approaches. According to Sandars’s work [16], reflection is a form of learning that is based on three common aims: (1) reflection for learning, (2) reflection to develop a therapeutic relationship, and (3) reflection to develop professional practice [14,15]. Indeed, reflexivity includes questioning the premises of an action, such as the values, norms, and beliefs that a professional may hold, as well as how such actors justify their actions [17]. In sum, reflexivity is a good strategy for motivating both health care professionals and patients to engage in patient care in a collaborative way.

    Reflexivity and SDM

    Reflexivity is appropriate in the context of SDM because the latter emphasizes a partnership between patients and clinicians in making decisions and establishing care plans. SDM aims to reposition the knowledge of patients and clinicians on an equal footing, adjusting the asymmetrical power relationship between patients and health care professionals [18,19]. A prerequisite for implementing SDM in care settings is that health care professionals not only have the knowledge and skills but also the willingness to engage patients in the decision-making process [20].

    Training based on reflexivity may lead health care professionals to question the power issues inherent in a more traditional conception of the health system and of their role and may motivate them to adopt SDM [11]. Thus, we hypothesized that SDM training programs that integrate reflexivity strategies would be more effective in increasing the adoption of SDM than those that do not.

    Interprofessional Training

    Research also shows that SDM training programs developed with an interprofessional orientation are to be encouraged [21]. Interprofessionality is defined by D’amour et al [22] as the development of a cohesive practice between professionals from different disciplines for the care of a single patient. Specifically, interprofessional collaboration involves collegial, authentic, constructive, open and honest communication as well as mutual trust and respect between professionals who are committed to achieving a common set of goals. However, although an interprofessional orientation is highly encouraged in SDM because of the need for better teamwork and professionals’ openness to other forms of practices, few training sessions with an interprofessional orientation are available [10]. In addition, there is little evidence that interprofessionally oriented training is more effective than other training approaches. Given the similarity between the methods used in training programs with an interprofessional orientation and those used in training programs with certain reflexivity strategies (eg, peer-to-peer group learning), it is important to examine how effective both these training types are.

    Outcome Reporting

    In the Cochrane review, one of the criteria for selecting programs was that the type of measurement followed must be a patient-reported outcome measurement (PROM) or an observer-reported outcome measurement (OBOM). A PROM is an instrument used to collect information directly from patients. PROMs do not require amendments or interpretation by a clinician or another observer [23]. An OBOM is any instrument used by a third-party observer to report observable concepts such as signs or behaviors to assess, for example, the decision-making process during an encounter between a patient and family and their health care professional when facing a health treatment or screening decisions [23]. It seemed useful to see what types of measurements were most common, especially among the effective training programs that used reflexivity strategies. This could offer another potential reason for a training program’s effectiveness. As we were interested in interventions targeting health care professionals, it was important to discern whether the type of measurement used to evaluate the intervention was equipped to examine the different levels of effectiveness as defined by Kirkpatrick, the creator of one of the most common evaluation models for assessing training programs targeting health care professionals [24]. The Kirkpatrick model evaluates training programs based on 4 categories: the satisfaction of the participants with the training, improvement of their knowledge, improvement of their care practices, and improvement of patient health. The first category is very important in the evaluation process because, according to Kirkpatrick, learner appreciation is a key factor in motivating participants to learn from training and apply what they learned.

    Therefore, based on our analysis of the Cochrane review of interventions for increasing the use of SDM by health care professionals, we first sought to determine whether the SDM training programs for health care professionals that used reflexivity strategies were more frequently classified as effective than the training programs that did not. Second, if such programs were more effective, we aimed to explore the strategy (peer-to-peer group learning or self-appraisal individual learning) that seemed to be more effective. Third, we aimed to examine whether training programs with an interprofessional orientation tended to be evaluated as effective. Finally, we examined whether the type of measurement used (OBOM or PROM) made it possible to classify the training programs in terms of effectiveness.


    Study Design

    We performed a secondary analysis of a published 2018 Cochrane review that aimed to evaluate the effectiveness of interventions for increasing the use of SDM by health care professionals [7]. As there are no reporting guidelines for the secondary analyses of systematic reviews, we used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 for all applicable items [25]. This study took place from February 2021 to April 2022.

    Data Sources and Search Strategy

    The search strategy for the Cochrane review serving as the basis of the current secondary analysis was launched on June 15, 2017. Details of the Cochrane search strategy and data sources can be found in the published review [7]. The Cochrane review [7] included interventions classified according to the three target categories of the Effective Practice and Organization of Care (EPOC) taxonomy of interventions [26]: (1) interventions targeting patients (eg, patient-mediated interventions), (2) interventions targeting health care professionals (eg, distribution of printed educational material, educational meetings, audit and feedback, reminders, and educational outreach visits), and (3) interventions targeting both patients and health care professionals (eg, a patient-mediated intervention combined with an intervention targeting health care professionals).

    Eligibility Criteria

    In the published Cochrane review, the participants in the training programs could be any type of health care professional (eg, physicians, nurses, pharmacists, or social workers), including professionals in training (eg, resident physicians). Studies that recruited eligible health care professionals along with other types of participants (eg, patients and managers) were also included, as were training programs evaluating a multicomponent intervention (eg, SDM training for health care professionals with the use of patient decision aids). All types of training formats were incorporated into the review (eg, in class, group workshop, web-based training, and synchronous or asynchronous training). A training program was defined as a capacity-building activity conducted live for a group or a single individual, such as a web-based course or a traditional course (ie, a course integrated into an academic program), that used a recognized instructional method such as lectures, workshops, case studies, demonstrations, role plays, and small group discussions [10]. There were no restrictions on the comparison groups, which were all included.

    The same primary outcomes of interest reported in the Cochrane review, namely SDM outcomes, were maintained in this secondary analysis, as were the outcome measurements used [7]. The Cochrane review grouped secondary outcomes into 2 categories: patient outcomes (eg, affective-cognitive outcomes, behavioral outcomes, and health outcomes) and process outcomes (eg, consultation length, costs, and equity). The 4 eligible study designs were randomized controlled trials, nonrandomized controlled trials, controlled before-and-after studies, and interrupted time series.

    In this secondary analysis, we selected all interventions targeting health care professionals from the Cochrane review (43/87, 49%), excluding interventions that only targeted patients (44/87, 51%). Of the 43 that targeted health care professionals, we excluded interventions that were not a training program (12/87, 14%), for example, demonstrations on how to use a decision aid. Therefore, the total number of articles included in this study was 31.

    Study Selection Process

    One of the reviewers (NTD) examined all the content available concerning the interventions used in each of the 87 articles included in the Cochrane review and selected interventions that met the inclusion criteria [7]. Another reviewer (JL) reviewed both the selected and excluded programs to validate the rigor of the selection process. Finally, all the articles involving an eligible training program targeting health care professionals were selected (N=31) (refer to the eligibility criteria section).

    Data Extraction Process

    Data extraction was performed by 2 pairs of independent reviewers (NTD and AM or NTD and VB). Differences between the 2 reviewers were resolved by consensus based on discussion and by referring to the definitions provided in the extraction grid (Multimedia Appendix 1). For the remaining conflicts, a third reviewer (SG-B or MCT) intervened to facilitate a consensus. The articles believed to include reflexivity were submitted to MCT for validation. Data extracted included (1) article and study characteristics—year, country and language of publication, and measurement of study design and type; (2) information on the training program—country, training language, context of care, type of health care professional trained, and training format (eg, unidirectional or interactive); and (3) the use or otherwise of reflexivity strategies for interactive training programs and, among those including reflexivity strategies, the type of strategy selected (peer-to-peer reflective group learning or self-appraisal learning).

    Classification of Articles

    Classification of Training Formats

    First, 2 reviewers (NTD and AM or NTD and VB) classified all the included articles (n=31) according to training format (unidirectional or interactive). The interactive training programs were then classified into 2 groups: programs using reflexivity strategies and programs that do not (refer to the details given in the Reflexivity Strategies Assessment section). On the basis of how reflexivity is defined in the literature, in this analysis, only interactive training programs were considered to have the potential to involve reflexivity. The training programs classified here as unidirectional were those in which the trainer delivered the whole message without asking for learner input other than questions, whereas interactive training programs are delivered in a 2-way manner, requiring the active contribution of learners (ie, the trainer delivers the information to the learners and encourages them to contribute to an exchange process). Interactive training by its very nature has the potential for reflexivity through, for example, role play or case discussions. For the purposes of this study, training programs using reflexivity strategies involve at least a minimal contribution from learners in the reflection process. Second, training programs classified as including reflexivity strategies were also categorized into 2 further groups (peer-to-peer group learning or self-appraisal individual learning) by 2 reviewers (NTD and AM or NTD and VB, validated by MCT).

    Reflexivity Strategies Assessment

    To the best of our knowledge, there is no validated set of criteria that defines the minimal components required to qualify a training program as reflexivity based. Thus, we developed a grid informed by a preliminary rapid literature review that synthesized the most common approaches and concepts related to reflexivity strategies used in health care professional training [27]. The grid contains minimal criteria that a training program must meet to be considered as including reflexivity strategies. The 2 following questions (A and B) from our grid were used to assess whether an interactive training program incorporated reflexivity strategies.

    (A) Does the Training Program Include Any Reflexivity Approaches?

    A reflexivity training approach could be, but is not limited to, the following: group-based reflections with peers (with or without a trainer), self-competence improvement with case-based reflections, electronic platforms with reflective portfolios, reflective journals, Balint groups, on-site reflective writing exercises, and the like. When these approaches were not clearly specified, we looked for the common reflexivity concepts.

    (B) Does the Training Program Include Any Common Reflexivity Concepts?

    The following were considered reflexivity concepts: critical thinking, metacognition, self-reflection, reflective dialogue, reflection-in-action, reflection-in-practice, reflection-on-action reflection-on-practice, reflective practice, reflective learning, reflective approaches, reflective dialogues, critical self-reflection, reflective thinking, reflection on error, and the like.

    Once we identified articles that included reflexivity strategies based on questions A (presence of reflexivity approaches) and B (presence of reflexivity concepts), we further subcategorized them according to 2 types of strategies: peer-to-peer group learning or self-appraisal individual learning.

    Peer-to-Peer Group Learning

    The main objective of peer-to-peer group learning (small or large group) is to stimulate interaction between participants. In peer-to-peer groups, the participants learn from each other’s reflections while being supported by experienced trainers or facilitators. This strategy can be organized in different ways, for example, a few days of practice followed by a day of reflection among peers or presentation of a topic followed by a group reflection among professionals during which they discuss their practice experience. Various approaches such as reflective writing exercises or groups with colleagues can be incorporated. Reflections may be based on real cases (ie, cases seen in practice) or fictitious ones. Everyone is called upon to give their point of view, and lessons are learned as the reflection progresses [28]. During and at the end of the exercise, the trainer or facilitator reframes the interactions, guides the discussion, and corrects errors or discrepancies resulting from the reflection [28].

    Self-appraisal Individual Learning

    A self-appraisal individual learning strategy is any individual learning process in which the learner is subjected to reflection exercises or cases to be solved. The learner can also be questioned on their practice. The exercise might involve traps to allow learners to detect their own errors. This exercise is performed individually, for example, in a reflective journal where professionals might write down reflections on their practice such as all the events (positive or negative) experienced, what these events meant for them, and what they learned from these experiences [29]. They may then reflect on how such experiences could help them in similar future circumstances. This exercise can also be performed during a group training session but where participants reflect individually on a case (fictitious or real) [30]. A self-appraisal individual learning strategy can also be applied during web-based courses. In a self-appraisal individual learning process, exercises such as self-reflection; reflective learning; reflection on one’s own values, beliefs, and thoughts; and metacognition are often used.

    Assessing Training Effectiveness

    We determined whether the included training programs were classified as effective in the Cochrane review [7]. The same outcomes of interest reported in the Cochrane review, that is, SDM outcomes, were maintained in this secondary analysis.

    Analysis

    First, unidirectional and interactive training programs were compared to see what percentage of each was classified as effective by the Cochrane review. Second, among the interactive programs, those that used reflexivity strategies were compared with those that did not to see which were more frequently effective. Third, reflexivity strategies (peer-to-peer group learning and self-appraisal individual learning) were compared. After these 3 steps, we carried out an additional analysis to see whether there were elements that could explain why a training program was effective or otherwise, apart from those cited earlier. For example, we compared the proportion of training programs based on interprofessional orientation that were effective with the proportion of programs using reflexivity that were effective. In addition, to see how SDM training programs can be better evaluated, we classified training programs according to the type of measurement they incorporated and whether they were classified as effective.


    General Characteristics of Results

    Main Characteristics of the Included Studies

    All the 87 studies included in the Cochrane review were evaluated [7]. Of these, 43 (49%) interventions targeted health care professionals, and 44 (51%) targeted only patients. Among the 43 interventions targeting health care professionals, 31 (72%) were found to be training programs (Figure 1) and were analyzed in this study.

    ‎
    Figure 1. Selection process of the included studies. HCP: health care professional.
    View this figure

    Of the 31 articles included, 10 (32%) were published between 2002 and 2010 (the first publication of the Cochrane review) [31-40], 14 (45%) were published between 2011 and 2014 (the first update of the Cochrane review) [41-54], and the other 7 (23%) were published between 2015 and 2017 (the most recent update) [55-62]. A total of 11 (35%) included articles were published in the United States [38, 43, 46, 50, 51, 53-55, 58, 60, 61, 63], followed by 8 (26%) in Germany [3,32,35,37,39,44,47,57,64-66] (Table 1). All the studies included were published in English, and 29 (94%) of them were RCTs. For the primary outcome assessment, 15 (48%) of the 31 studies were evaluated using PROMs alone, 12 (39%) were evaluated using OBOMs, and 3 (10%) were evaluated using both. Information about the 1 (3%) study [35] that used health care professional–reported outcome measurements (HCPROMs) was directly collected from the article. Cochrane did not include outcomes measured by HCPROMs in its analysis [7]. However, seeing that this paper [35] was included in the analysis of some secondary outcomes in the Cochrane review, we examined the results directly from the article to analyze information related to our variable of interest (SDM).

    Table 1. Characteristics of included studies (N=31).
    CharacteristicsValues, n (%)
    Year of publication

    		2002-201010 (32)

    		2011-201414 (45)

    		2015-20177 (23)
    Country

    		United States11 (35)

    		Germany8 (26)

    		United Kingdom4 (13)

    		Canada3 (10)

    		Norway2 (6)

    		Other (Netherlands, Belgium, and Switzerland)3 (10)
    Language

    		English31 (100)
    Study design

    		Randomized controlled trials29 (94)

    		Nonrandomized controlled trials1 (3)

    		Before-and-after studies1 (3)
    Outcome measure assessors

    		PROMa15 (48)

    		OBOMb12 (39)

    		PROM and OBOM3 (10)

    		PROM and HCPROMc1 (3)

    aPROM: patient-reported outcome measurement.

    bOBOM: observer-reported outcome measurement.

    cHCPROM: health care professional–reported outcome measurement.

    Main Characteristics of the Training Programs

    Of the 31 training programs, 11 (35%) were developed in the United States, and 8 (26%) in Germany (Table 2). A total of 22 (71%) programs were in English, followed by 4 (13%) in German and 2 (6%) in Dutch. Of the remaining programs, 1 (3%) was in French, 1 (3%) was in both English and Spanish, and language was not reported for one of the programs. Of the 31 training programs, 20 (65%) were developed in a primary health care context, and 11 (35%) were developed in specialized care. Regarding the type of health care professionals trained, 18 (58%) targeted physicians, and 26 (84%) were developed for fully trained health care professionals (Table 2). Only 5 (16%) training programs out of the 31 were developed with an interprofessional orientation, that is, with the promotion of interprofessionality as one of its training objectives.

    A total of 24 (77%) of the 31 programs had an interactive format [3,32-39,41-45,51-60,64], and the remaining 7 (23%) were unidirectional training programs [40,46-50,61]. Among the 24 interactive training programs, 7 (29%) were developed using reflexivity strategies [32-34,41,42,55,56,67,68], among which 5 (71%) were classified as peer-to-peer group learning [32,33,41,42,56,64,69] and 2 (29%) as self-appraisal individual learning [34,55,62]. Details on how the training programs were classified according to the reflexivity strategy used are reported in Table 3.

    Table 2. Training program characteristics (N=31).
    CharacteristicsValues, n (%)
    Country

    		United States11 (35)

    		Germany8 (26)

    		United Kingdom4 (13)

    		Canada3 (10)

    		Norway2 (7)

    		Australia, Netherlands, Switzerland, and Germany (collaboration)1 (3)

    		Others (Netherlands and Belgium)2 (7)
    Language

    		English22 (71)

    		German4 (13)

    		Dutch2 (7)

    		French1 (3)

    		English and Spanish1 (3)

    		Not reported1 (3)
    Context of care

    		Primary care20 (65)

    		Specialized care11 (36)
    Types of professional trained

    		Physiciansa18 (58)

    		Nurses and geneticists8 (26)

    		Nurses2 (7)

    		Physicians and nurses2 (7)

    		Physicians and midwives1 (3)

    		Fully trained26 (84)

    		Not fully trained5 (16)
    Interprofessional orientation5 (16)

    aOne training program was designed for oncologists and gynecologists, and the other included medical residents.

    Table 3. Quotations illustrating how reflexivity strategies are reported.
    ArticlesMain quotationsTypes of reflexivity strategies
    Krones et al [32], 2008
    • “Practical communication strategies”
      • ‎
    • “After role-play feedback was given by their peers”a
      • ‎
    • “Educational outreach...members were invited to moderate the sessions”
      • ‎
    • “Using the script-like decision aid was practiced through role playing”
      • ‎
    • “Participants received feedback from peers in their groups”
      • ‎
    		Peer-to-peer reflective group learning
    Murray et al [33], 2010
    • “Self and peer appraisal during role play”
      • ‎
    • “Participants will evaluate decision-support skills to self-appraise their own and workshop peers’ quality of decision support during the case studies and role-play activities”
      • ‎
    • “To train nurses and medical residents in self-appraisal”
      • ‎
    		Peer-to-peer reflective group learning
    Sanders et al [56], 2017
    • “The training was based on the learning principles described by Kolb. In the training sessions, group discussion, theory, role-playing, and reflections on personal behaviour were alternated. This tool was generated in the first session when the GPs reflected on their training experiences”
      • ‎
    		Peer-to-peer reflective group learning
    Fossli et al [41], 2011
    • “The course consisted of a 50/50 mix of theory and 45 min group sessions (3-7 participants and two teachers per group) including role-plays, with plenary debriefs after each group”
      • ‎
    • “Our course was based on the same content as the 5-day course Communication Skills Intensive offered by Kaiser Permanente”
      • ‎
    • “At the conclusion of the course, all participants received a one-sheet overview of the Four Habits to carry in their pockets as reminder in everyday work”
      • ‎
    		Peer-to-peer reflective group learning
    Kennedy et al [42], 2013
    • “Skills to encourage a structured approach to self-care support in consultations. Interactive role play (small groups) techniques to help deal with difficult issues during consultations. Interactive role play (small groups). Brief presentation with discussion. DVD exemplar of use plus manual involving (whole group). Explanatory models to encourage discussion about the causes and consequences of long-term conditions... Presentation with discussion. DVD exemplar of use plus manual (involving whole group)... As a practice–develop skills to solve problems that come up in the work of the practice. Problem-solving techniques involving whole practice systems within practice to improve self-care support for patients.”
      • ‎
    • “Problem-solving techniques involving whole practice ways to engage patients with self-care support.”
      • ‎
    		Peer-to-peer reflective group learning
    Elwyn et al [34], 2004
    • “Practitioners attended two workshops. During the first workshop, the background literature on SDM was outlined and participants were asked to debate its relevance to clinical practice. The skills of SDM were described and demonstrated using simulated consultations. This provided opportunities for all the participants to comment on the method, using an observational competence checklist. Simulated patients were also encouraged to comment. Participants were asked to consult with the simulated patients using preprepared scenarios involving the study conditions. At the second workshop, participants were asked to consider the competences in more depth. By the end of the workshop, all participants had conducted and received feedback from at least one consultation with a simulated patient”
      • ‎
    		Self-appraisal individual learning
    Epstein et al [55], 2017
    • “A 2-session in-office physician training (1.75 hours) using a brief video, feedback from standardized patients portraying roles of patients with advanced cancer, audio recorded study patient visits, and (2) (...), plus up to 3 follow-up phone calls (Table 1).”
      • ‎
    		Self-appraisal individual learning

    aText in italics illustrates possible reflexivity approaches and concepts (related to questions A and B of our criteria grid).

    Effectiveness of the Training Programs in Connection With Different Variables

    Training Formats and Effectiveness

    Based on the Cochrane review classification of the effectiveness of the included interventions, 10 (42%) of the 24 interactive training programs were deemed effective, as opposed to 1 (14%) of the 7 unidirectional programs (Table 4).

    Table 4. Effectiveness of the training programs according to the measures reported.
    Training format and articlesSMDa, EMDb, or RDc (95% CI)Narrative resultsEffectived
    Interactive format (including reflexivity strategies)

    		Krones et al [32], 2008; Hirsch et al [64], 20100.40 (0.28 to 0.52)N/AeYes

    		Murray et al [33], 20103.75 (2.46 to 5.03)N/AYes

    		Sanders et al [56], 20170.85 (0.54 to 1.16); 0.93 (0.62 to 1.25)N/AYes

    		Elwyn et al [34], 2004; Edwards et al [67], 2004; Longo et al [68], 2006−0.30 (−1.19 to 0.59); 0.05 (−0.17 to 0.27)N/ANo

    		Fossli et al [41], 20110.38 (−0.17 to 0.94)N/ANo

    		Kennedy et al [42], 2013; Kennedy et al [69], 2010−0.05 (−0.12 to 0.01)N/ANo

    		Epstein et al [55], 2017; Butow et al [62], 20150.00 (−0.24 to 0.24)N/ANo
    Interactive format (not including reflexivity strategies)

    		Bieber et al [35], 2006; Bieber et al [65], 2007N/A“An ANOVA for repeated measurements comparing the SDM group with the information group revealed that patients’ appraisal of the interaction quality was higher in the SDM group”Yes

    		Stacey et al [36], 20082.07 (1.26 to 2.87)N/AYes

    		Loh et al [37], 2007N/A“In the intervention group, significantly higher patient participation from pre- to postintervention was found.”Yes

    		Haskard et al [38], 2008N/A“Training significantly improved physicians’ health behaviour counseling of their patients.”Yes

    		Deinzer et al [39], 2009N/A“The degree of SDM was significantly higher in the SDM group at baseline and after one-year visits. The results of the SDM sum score on actually practiced SDM exhibited in both groups significantly increased, but the control group did not reach the score of the study group after one year.”Yes

    		Feng et al [43], 2013N/A“Significant difference in favour of the intervention group, high risk of bias.”Yes

    		Tinsel et al [44], 20130.32 (0.17 to 0.46)N/AYes

    		Hamann et al [3], 20070.16 (−0.28 to 0.61)N/ANo

    		Bernhard et al [45], 2011; Butow et al [62], 2015N/A“There was no effect for this variable for SGAf doctors (estimated population mean difference 0.52, SE 1.39, ESg=0.04; P=.71)” “After the training workshop, doctors in the experimental group within the ANZh cohort displayed more behaviours designed to establish the SDM framework than doctors in the control group (estimated population mean difference=3.42, SE 1.50, ES=0.30, P=.03). However, the ES was small” “There was considerable variation in patient outcomes between the SGA and ANZ cohorts and no substantial training effect”No

    		Cooper et al [54], 20110.11 (−0.30 to 0.51); 0.03 (−0.15 to 0.20); 0.16 (−0.23 to 0.56)N/ANo

    		Légaré et al [52], 2012; Allaire et al [70], 2012; Légaré et al [71], 20130.01 (−0.03 to 0.06)N/ANo

    		Cooper et al [53], 20130.70 (0.30 to 1.90)5N/ANo

    		Wilkes et al [51], 2013−0.13 (−0.32 to 0.05)N/ANo

    		Härter et al [57]i, 2015; Bieber et al [66], 20180.54 (0.35 to 0.74); −0.07 (−0.26 to 0.12); 0.11 (−0.10 to 0.31)N/ANo

    		Tai-Seale et al [58]j, 2016; Dillon et al [63], 20170.35 (−0.53 to 1.24); 0.51 (0.19 to 0.84); −0.29 (−1.17 to 0.60); 0.00 (−0.32 to 0.32)N/ANo

    		Ampe et al [59], 2017−0.10 (−0.96 to 0.76)N/ANo

    		Cox et al [60], 20170.11 (−0.21 to 0.42)N/ANo
    Unidirectional

    		Hess et al [46], 20122.82 (2.43 to 3.21)N/AYes

    		O’Cathain et al [40], 2002−0.02 (−0.05 to 0.01)N/ANo

    		Koerner et al [47], 2014−0.08 (−0.26 to 0.11)N/ANo

    		Mathers et al [48], 2012−0.09 (−0.23 to 0.05)N/ANo

    		Rise et al [49], 2012; Rise et al [72], 20160.13 (−0.32 to 0.58)N/ANo

    		Sheridan et al [50], 2014−0.17 (−0.35 to 0.00)N/ANo

    		Coylewright et al [61], 20160.51 (−0.05 to 1.07)N/ANo

    aSMD: standardized mean difference.

    bEMD: effect size mean difference.

    cRD: risk difference.

    dScored as “Yes” if the 95% CI reported in the Cochrane review did not include 0 for the SMD, RD, and MD values or when the 95% CI did not include 1 for the OR values. In some studies, with ≥2 scales, we referred to the conclusion of the authors.

    eN/A: not applicable.

    fSGA: Switzerland, Germany, and Australia.

    gES: effect size.

    hANZ: Australia and New Zealand.

    iThis study found no effect of shared decision-making training on the primary outcomes, which were similar between both the groups. However, training did contribute to improved observer-rated shared decision-making skills in physicians and reduced anxiety and depression in patients, particularly in women with breast cancer.

    jThe primary outcome measure was CollaboRATE, a patient-reported experience with care. While the odds ratios (ORs) from the ASK (Ask Share Know) clinic (OR 1.417) and the OpenComm plus ASK clinic (OR 1.134) were greater than 1, their 75% CIs included 1, which suggests no difference from the usual care clinic. Our findings suggest that something could be done to improve the patient experience. We view the results as promising evidence of the intervention’s efficacy and as meaningful signals of its likely effects on patient experience.

    Reflexivity Strategies and Effectiveness

    Regarding the effectiveness of the programs, 3 (43%) of the 7 programs including reflexivity strategies were deemed effective [32,33,56]. The number of effective programs among programs with reflexivity strategies was similar to that among interactive programs without reflexivity strategies. Concerning the latter, 7 (41%) out of 17 were deemed effective (Table 4) [35-39,43,44].

    Table 4 shows that 3 (60%) of the 5 training programs using a peer-to-peer group learning strategy were effective [32,33,56], whereas none of those using a self-appraisal individual learning strategy were effective (Table 4).

    Interprofessional Approach and Effectiveness

    Of the 5 training programs developed with an interprofessional orientation, 3 (60%) were classified by the Cochrane review as effective [32,33,46], and 2 (67%) of these included reflexivity strategies [32,33] (Table 5). In other words, the 2 programs with an interprofessional orientation that included reflexivity strategies were both deemed effective.

    Table 5. Interprofessional orientation and type of measurement.
    StudiesEffectiveIPa approachOutcome assessorsTraining format
    Krones et al [32], 2008; Hirsch et al [64], 2010YesYes
    • PROMb
      • ‎
    		Interactive format (including reflexivity strategies)
    Murray et al [33], 2010YesYes
    • OBOMc
      • ‎
    		Interactive format (including reflexivity strategies)
    Sanders et al [56], 2017YesNo
    • OBOM
      • ‎
    • OBOM
      • ‎
    		Interactive format (including reflexivity strategies)
    Elwyn et al [34], 2004; Edwards et al [67], 2004; Longo et al [68], 2006NoNo
    • OBOM
      • ‎
    • PROM
      • ‎
    		Interactive format (including reflexivity strategies)
    Fossli et al [41], 2011NoNo
    • OBOM
      • ‎
    		Interactive format (including reflexivity strategies)
    Kennedy et al [42], 2013; Kennedy et al [69], 2010NoNo
    • PROM
      • ‎
    		Interactive format (including reflexivity strategies)
    Epstein et al [55], 2017; Butow et al [62], 2015NoNo
    • PROM
      • ‎
    		Interactive format (including reflexivity strategies)
    Bieber et al [35], 2006; Bieber et al [65], 2007YesNo
    • PROM
      • ‎
    • HCPROM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Stacey et al [36], 2008YesNo
    • OBOM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Loh et al [37], 2007YesNo
    • PROM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Haskard et al [38], 2008YesNo
    • OBOM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Deinzer et al [39], 2009YesNo
    • OBOM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Feng et al [43], 2013YesNo
    • OBOM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Tinsel et al [44], 2013YesNo
    • PROM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Hamann et al [3], 2007NoNo
    • PROM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Bernhard et al [45], 2011; Butow et al [62], 2015NoNo
    • OBOM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Cooper et al [54], 2011NoNo
    • PROM
      • ‎
    • PROM
      • ‎
    • PROM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Légaré et al [52], 2012; Allaire et al [70], 2012; Légaré et [71], 2013NoNo
    • PROM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Cooper et al [53], 2013NoNo
    • PROM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Wilkes et al [51], 2013NoNo
    • PROM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Härter et al [57], 2015; Bieber et al [66], 2018NoNo
    • PROM
      • ‎
    • OBOM
      • ‎
    • OBOM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Tai-Seale et al [58], 2016; Dillon et al [63], 2017NoNo
    • OBOM
      • ‎
    • PROM
      • ‎
    • OBOM
      • ‎
    • PROM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Ampe et al [59], 2017NoNo
    • OBOM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Cox et al [60], 2017NoYes
    • OBOM
      • ‎
    		Interactive format (not including reflexivity strategies)
    Hess et al [46], 2012YesYes
    • OBOM
      • ‎
    		Unidirectional
    O’Cathain et al [40], 2002NoNo
    • PROM
      • ‎
    		Unidirectional
    Koerner et al [47], 2014NoYes
    • PROM
      • ‎
    		Unidirectional
    Mathers et al [48], 2012NoNo
    • PROM
      • ‎
    		Unidirectional
    Rise et al [49], 2012; Rise et al [72], 2016NoNo
    • PROM
      • ‎
    		Unidirectional
    Sheridan et al [50], 2014NoNo
    • PROM
      • ‎
    		Unidirectional
    Coylewright et al [61], 2016NoNo
    • OBOM
      • ‎
    		Unidirectional

    aIP: interprofessional.

    bPROM: patient-reported outcome measure.

    cOBOM: observer-reported outcome measure.

    Outcome Assessors and Effectiveness

    Among the selected articles, based on the Cochrane review, 8 (67%) out of the 12 programs using OBOMs were classified as effective [33,36,38,39,43,46,56], while 3 (21%) of the 14 programs using PROMs were classified as effective [32,44]. Meanwhile, 2 (67%) of the 3 training programs that included reflexivity strategies and were classified as effective were assessed using OBOMs. The only effective program in the unidirectional category was assessed using OBOMs (Table 5).

    The main findings related to the different elements analyzed in the study are summarized in Table 6.

    Table 6. Summary of main findings (N=31).
    Level of analysis and training categoriesTraining programsEffective programIPa approachEffective program with IP approachType of assessment


    		Values, NValues, n (%)Values, NValues, n (%)Values, NValues, n (%)Values, NValues, n (%)

    		Level 1


    		Format


    		Interactive3124 (7)2410 (42)53 (60)32 (67)OBOMb: 10/24 (42); PROMc: 10/24 (42); PROM/OBOM: 3/24 (12); PROM/HCPROM: 1/24 (4)


    		Unidirectional317 (23)71 (14)52 (40)21 (50)OBOM: 2/7 (29); PROM: 5/7 (71)
    Level 2

    		Interactive


    		Reflexivity strategies—yes247 (30)73 (43)32 (67)32 (67)OBOM: 3/7 (43); PROM: 3/7 (43); PROM/OBOM: 1/7 (14)


    		Reflexivity strategies—no2417 (70)247 (30)31 (33)10 (0)OBOM: 7/17 (41); PROM: 7/17 (41); PROM/OBOM: 2/17 (12); PROM/HCPROM: 1/17 (6)
    Level 3

    		Reflexivity strategies—yes


    		Peer-to-peer group learning75 (71)53 (60)22 (100)22 (100)OBOM: 3/5 (60); PROM: 2/5 (40)


    		Self-appraisal learning72 (29)50 (0)20 (0)N/AdN/APROM: 1/2 (50); PROM/OBOM: 1/2 (50)

    aIP: interprofessional

    bOBOM: observer-reported outcome measure.

    cPROM: patient-reported outcome measure.

    dN/A: not applicable.

    Risk of Bias Assessment

    The risk of bias assessment was reported in the initial Cochrane review, which used criteria for EPOC reviews [73] and the Cochrane Handbook for Systematic Reviews of Interventions [74] for interrupted time series designs.

    Certainty of Evidence

    For our variable of interest (the primary outcome), the certainty of evidence was assessed in the Cochrane review according to the GRADE (Grading of Recommendations Assessment, Development and Evaluation) guidelines and methods described in Chapter 12 of the Cochrane Handbook for Systematic Reviews of Interventions [75]. The method proposed in EPOC worksheets was used to determine which secondary outcomes should be assessed [76].


    Principal Findings

    This secondary analysis of a 2018 Cochrane review on interventions for increasing the use of SDM by health care professionals aimed to identify training programs that included reflexivity and to ascertain how effective they were. Our study is the first to evaluate the effectiveness of SDM training programs that include reflexivity strategies. Among the 31 SDM training programs for health professionals, 23% (n=7) included reflexivity and 77% (n=24) did not. More of those that included reflexivity were deemed effective as a percentage of the whole (3/7, 43%) than those that did not (8/24, 33%). Among the interactive training programs, there was little difference in effectiveness between those that used reflexivity strategies (3/7, 43%) and those that did not (7/17, 41%). However, when comparing interactive training programs with unidirectional ones, there were a great many more programs deemed effective among the former (10/24, 42%) than among the latter (1/7, 14%).. Among the training programs that included reflexivity (n=7), most programs using a peer-to-peer group learning strategy were found to be effective (3/5, 60%), whereas those using a self-appraisal individual learning strategy were not (0/2, 0%). The training programs with an interprofessional orientation were more frequently classified as effective (3/5, 60%) than those without (2/5, 40%). Finally, the percentage of effective training programs in studies using OBOMs to assess training was higher than the percentage in studies using PROMs. These findings led to the observations that follow.

    A Larger Percentage of Reflexivity-Based Training Programs Were Deemed Effective

    Our results confirm the findings of Leyland et al [77] and Chaffey et al [12] that in general, programs that include reflexivity have more positive effects than those that do not [12,77]. Training using reflexivity strategies has been shown to increase medical students’ ability to integrate alternative sources of knowledge and critically reflect on their own practices [12]. According to Chaffey et al [12], it is difficult to assess reflexivity alone, but when applied in an intervention, it yields more positive results. Although only a few SDM training programs include reflexivity strategies to date, these results suggest that reflexivity could be a core component of effective training and, as such, may be understood as an effective implementation strategy for change. However, further studies with a larger sample are needed to confirm this hypothesis. For Kolb [78], learning is a complex process driven partly by individuals’ ability to find a sense for themselves in operationalizing change in their day-to-day practices. As our results suggest, training that uses reflexivity strategies could provide health care professionals with that personal sense of the purpose of SDM and the motivation needed to implement SDM in their practices. However, future research could more precisely evaluate the impact of reflexivity strategies on SDM uptake by health care professionals if more programs were explicit in their use or promotion of reflexivity. The training programs included in this review were not designed to use or promote reflexivity. An analysis that compared SDM training programs that were explicitly based on reflexivity with those that were not would be more useful and appropriate. Nevertheless, our results suggest that SDM training for health care professionals based on reflexivity strategies is effective in motivating trainees to adopt SDM in a manner that facilitates positive patient experiences in health care systems.

    A Peer-to-Peer Group Learning Strategy Would Appear to Be More Effective Than a Self-appraisal Learning Strategy

    Our findings also showed that more training programs using peer-to-peer group learning were classified as effective than those using self-appraisal individual learning. These findings suggest that a self-appraisal strategy can be more effective than training focused on an individual if it is part of a group learning process. Interaction among learners seems to be a powerful strategy for encouraging reflection, even self-reflection. Research could further compare the types of reflexivity learning strategies that are most effective in SDM training for health care professionals.

    Training Using Reflexivity Strategies and an Interprofessional Orientation Can Lead to Better Results

    Our findings indicated that training programs with an interprofessional orientation have proven to be more effective than those without this orientation. All programs that used interprofessional orientation and reflexivity strategies were also deemed effective. In addition, these programs used a peer-to-peer group learning strategy in which participants were encouraged to learn from each other’s reflections by sharing their points of view and experiences [28]. One of the main goals of interprofessional SDM is to encourage the recognition of other professionals’ values and competencies [21,79], in other words, to recognize other members of the care team as peers [22]. Leyland et al [77] also considered that reflexivity is needed for health care professionals to recognize each other’s professional skills. This suggests that the peer-to-peer group learning strategy is especially appropriate for implementing an interprofessional approach. In addition, Tremblay et al [27] defined the goal of “formative reflexivity” (ie, reflexive practices for learning) as developing new visions of professional experience and working in collaboration. Thus, blending these 2 goals (interprofessional orientation and reflexivity) is a promising avenue for fostering the uptake of interprofessional SDM in health practice. Furthermore, encouraging reflection and questioning of one’s practice can help to correct the paternalistic way in which SDM still seems to be undertaken [19,77].

    Measuring SDM Training for Health Care Professionals Only With OBOM or PROM, a Limiting Approach

    The Cochrane review used only OBOMs and PROMs (or both) to evaluate intervention effectiveness, eschewing programs that used HCPROMs. In the review, most programs classified as effective were assessed using OBOMs (8/11, 73%) rather than PROMs (3/11, 27%). OBOMs are considered a more rigorous assessment option than self-reported measures because they are more independent [23]. Based on this justification, one can say that the apparent objectivity of OBOM assessment yields better results than the potential subjectivity arising when patients report an outcome. Furthermore, our findings showed that 2 (67%) of the 3 training programs using reflexivity and classified as effective were assessed with OBOMs, as opposed to only 1 (33%) with PROMs. However, neither PROMs nor OBOMs respect the first Kirkpatrick model criterion, which is the satisfaction of the participants with the training. According to Kirkpatrick, when participants like a training program, they may be willing to adopt what they have learned from it. Based on the importance of this first level of evaluation, we suggest that the perspective of the training beneficiaries (health care professionals) should be considered in evaluations, as is the case with HCPROMs. This does not exclude the use of PROM or OBOM measures to assess Kirkpatrick’s other 3 levels of improvement: knowledge improvement, practice improvement, and health-related outcome improvement (the levels considered in the Cochrane review). Furthermore, a reflexivity-based approach focuses not only on outcomes but also on the process or experience of health care, which may be as important to patients as outcomes [80]. If trainees reflect on their own practice and question their own values, they will better consider patients’ values and experiences in the health care-seeking process. Therefore, another relevant measure of effectiveness would be patient-reported experience measures (PREMs), which help improve patient care [81]. In a 2019 systematic review, Müller et al [9] analyzed the methods used by 41 studies that assessed SDM training programs for health care professionals and concluded that the diversity of assessment methods limits the ability to compare training program effectiveness and is a barrier to conclusive evidence. Therefore, it now seems important to develop a harmonized SDM training assessment measure that includes all the 4 perspectives to enable better comparison.

    Multiplicity of Elements Did Not Facilitate the Analysis

    In this review, some interventions involved multiple components, for example, they included a workshop, a web-based tutorial, and a decision aid tool or a workshop with audit and feedback. In these cases, it was difficult to evaluate the effects of the components separately. In addition, some of the training programs included patients. If the training includes patients, it can be difficult to know whether this inclusion has an additional impact on effectiveness. Another example is the diversity of the comparators. The articles included in the review used different types of comparators (usual care or another differing intervention). Finally, even if the primary outcome (SDM) was our focal interest, it may be defined differently from one study to another. For example, while some focus their analysis on the uptake of SDM as a whole, others may analyze only one component (eg, decision conflict or decision regret) or separate them. In some articles, analysis used ≥2 scales, which made our judgment difficult, especially if effectiveness-related results were different. To classify these types of articles in terms of effectiveness, we referred to the conclusions of the authors. Based on all these observations, we suggest using a core set of assessment methods and validated outcomes for all learning levels in SDM training programs to identify the most effective strategies and better compare them. Future research could explore methods to specifically assess reflexivity strategies included in SDM training programs targeting health care professionals for their inclusion in this core model. Our reflexivity grid, when validated and published, along with the results from this analysis, can be considered as the first step in assessing SDM training that uses reflexivity strategies, but further work is needed to guide training approaches in this field.

    Limitations

    This study has a few limitations. First, the systematic review included evidence only up to June 2017, and another update has not been performed since then. The Cochrane review has been updated twice, with each update including all the studies in the earlier versions; yet, its conclusions regarding effectiveness have varied little since the first review in 2010. At its first publication, it included only 5 RCTs, but in 2018, with 84 RCTs, it still concluded that “a great variety of activities exist to increase shared decision-making by health care professionals, but we cannot be confident about which of these activities work best because the certainty (or the confidence) of the evidence has been assessed as very low.”

    Second, seeing that the training programs included in the systematic review were not explicit about promoting reflexivity, the selection of those that included what could be accurately described as reflexivity strategies was not an easy task. Nevertheless, using our grid based on a preliminary rapid review, we identified 7 programs. This was a small number for further analysis, although not surprising, as reflexivity is a new concept in the context of health professional education about SDM. It is possible that we missed a few programs that included reflexivity, as we relied solely upon published data and did not contact the authors for additional information. To minimize this risk, we analyzed all additional materials cited in the articles that were linked with the training programs and considered the most recent publications to discuss our results. Finally, the grid used in the analysis was developed for the purpose of this study, and although it is an important contribution to the field of implementation, its validation will require further use and assessment.

    Conclusions

    Our results suggest that SDM training programs for health care professionals using reflexivity strategies could increase SDM implementation. Our study is the first to evaluate the effectiveness of SDM training that includes reflexivity and raises several questions: (1) Are peer-to-peer learning strategies more effective than self-appraisal strategies? (2) How can reflexivity and interprofessional orientation strategies best complement each other? (3) Are OBOMs and PROMs the only appropriate means of evaluating SDM training programs? The grid developed for identifying reflexivity strategies in training programs, including reflexivity-related approaches and concepts, will be a useful guide for developing reflexivity training and is to be validated in future studies.

    Acknowledgments

    The authors wish to thank Louisa Blair, scientific editor, for the language assessment and scientific editing, along with Anne McBryde of Abaca Traduction for revision services. They also thank Lionel Adisso, PhD candidate in epidemiology, for orienting the data analysis.

    Authors' Contributions

    NTD and FL were involved in the conceptualization. NTD and JL were involved in article selection. NTD, VB, and AM were involved in the classification of training programs and data extraction. NTD and JL were involved in data analysis. NTD and FL were involved in the investigation. NTD, M-CT, FL, MJD, and SG-B were involved in the development of the collection grid. FL was involved in resource collection. FL, MJD, and M-CT were involved in supervision. NTD wrote the first draft, and JL edited it. NTD, JL, FL, MJD, and M-CT were involved in writing, reviewing, and editing.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Extraction grid for training program based on reflexivity strategies.

    XLSX File (Microsoft Excel File), 32 KB
    1. Légaré F, Witteman HO. Shared decision making: examining key elements and barriers to adoption into routine clinical practice. Health Aff (Millwood) 2013 Feb;32(2):276-284. [CrossRef] [Medline]
    2. Hibbard JH, Greene J. What the evidence shows about patient activation: better health outcomes and care experiences; fewer data on costs. Health Aff (Millwood) 2013 Feb;32(2):207-214. [CrossRef] [Medline]
    3. Hamann J, Langer B, Winkler V, Busch R, Cohen R, Leucht S, et al. Shared decision making for in-patients with schizophrenia. Acta Psychiatr Scand 2006 Oct;114(4):265-273. [CrossRef] [Medline]
    4. Shay LA, Lafata JE. Where is the evidence? A systematic review of shared decision making and patient outcomes. Med Decis Making 2015 Jan;35(1):114-131 [FREE Full text] [CrossRef] [Medline]
    5. Veroff D, Marr A, Wennberg DE. Enhanced support for shared decision making reduced costs of care for patients with preference-sensitive conditions. Health Aff (Millwood) 2013 Feb;32(2):285-293. [CrossRef] [Medline]
    6. Moulton B, Collins PA, Burns-Cox N, Coulter A. From informed consent to informed request: do we need a new gold standard? J R Soc Med 2013 Oct;106(10):391-394 [FREE Full text] [CrossRef] [Medline]
    7. Légaré F, Adekpedjou R, Stacey D, Turcotte S, Kryworuchko J, Graham ID, et al. Interventions for increasing the use of shared decision making by healthcare professionals. Cochrane Database Syst Rev 2018 Jul 19;7(7):CD006732 [FREE Full text] [CrossRef] [Medline]
    8. Blumenthal-Barby J, Opel DJ, Dickert NW, Kramer DB, Tucker Edmonds B, Ladin K, et al. Potential unintended consequences of recent shared decision making policy initiatives. Health Aff (Millwood) 2019 Nov;38(11):1876-1881 [FREE Full text] [CrossRef] [Medline]
    9. Müller E, Strukava A, Scholl I, Härter M, Diouf NT, Légaré F, et al. Strategies to evaluate healthcare provider trainings in shared decision-making (SDM): a systematic review of evaluation studies. BMJ Open 2019 Jun 21;9(6):e026488 [FREE Full text] [CrossRef] [Medline]
    10. Diouf NT, Menear M, Robitaille H, Painchaud Guérard G, Légaré F. Training health professionals in shared decision making: update of an international environmental scan. Patient Educ Couns 2016 Nov;99(11):1753-1758 [FREE Full text] [CrossRef] [Medline]
    11. Beyene LS, Severinsson E, Hansen BS, Rørtveit K. Shared decision-making-balancing between power and responsibility as mental health-care professionals in a therapeutic milieu. SAGE Open Nurs 2018 Jan 11;4:2377960817752159 [FREE Full text] [CrossRef] [Medline]
    12. Chaffey LJ, de Leeuw EJ, Finnigan GA. Facilitating students' reflective practice in a medical course: literature review. Educ Health (Abingdon) 2012;25(3):198-203 [FREE Full text] [CrossRef] [Medline]
    13. Alexander SA, Jones CM, Tremblay MC, Beaudet N, Rod MH, Wright MT. Reflexivity in health promotion: a typology for training. Health Promot Pract 2020 Jul;21(4):499-509 [FREE Full text] [CrossRef] [Medline]
    14. Schön DA. The Reflective Practitioner: How Professionals Think In Action. London, UK: Routledge; 1992.
    15. Tremblay MC, Parent AA. Reflexivity in PHIR: let's have a reflexive talk!. Can J Public Health 2014 May 09;105(3):e221-e223 [FREE Full text] [CrossRef] [Medline]
    16. Sandars J. The use of reflection in medical education: AMEE Guide No. 44. Med Teach 2009 Aug;31(8):685-695. [CrossRef] [Medline]
    17. Priddis L, Rogers SL. Development of the reflective practice questionnaire: preliminary findings. Reflect Pract 2018;19(1):89-104 [FREE Full text] [CrossRef]
    18. Elwyn G, Fisher E. Higher integrity health care: evidence-based shared decision making. Circ Cardiovasc Qual Outcomes 2014 Nov;7(6):975-980. [CrossRef] [Medline]
    19. Berger Z, Galasinski D, Scalia P, Dong K, Blunt HB, Elwyn G. The submissive silence of others: examining definitions of shared decision making. Patient Educ Couns 2022 Jul;105(7):1980-1987. [CrossRef] [Medline]
    20. Grim K, Rosenberg D, Svedberg P, Schön UK. Shared decision-making in mental health care-a user perspective on decisional needs in community-based services. Int J Qual Stud Health Well-being 2016 May 9;11:30563 [FREE Full text] [CrossRef] [Medline]
    21. Dogba MJ, Menear M, Stacey D, Brière N, Légaré F. The evolution of an interprofessional shared decision-making research program: reflective case study of an emerging paradigm. Int J Integr Care 2016 Jul 19;16(3):4 [FREE Full text] [CrossRef] [Medline]
    22. D'Amour D, Oandasan I. Interprofessionality as the field of interprofessional practice and interprofessional education: an emerging concept. J Interprof Care 2005 May;19 Suppl 1:8-20. [CrossRef] [Medline]
    23. Velentgas P, Dreyer NA, Nourjah P, Smith SR, Torchia MM. Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide. Rockville, MD, USA: Agency for Healthcare Research and Quality; 2013.
    24. Kirkpatrick DL. Evaluating Training Programs: The Four Levels. Oakland, CA, USA: Berrett-Koehler; 1994.
    25. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev 2021 Mar 29;10(1):89 [FREE Full text] [CrossRef] [Medline]
    26. EPOC Taxonomy. Cochrane Effective Practice Organisation of Care.   URL: https://epoc.cochrane.org/epoc-taxonomy [accessed 2022-11-14]
    27. Tremblay MC, Garceau L, Thiab Diouf N, Guichard A, Quinty J, Gravel C, et al. Improving understanding of reflexivity in family medicine: development of an educational tool based on a rapid review. MedEdPublish 2021;10(1):237-243 [FREE Full text] [CrossRef]
    28. Al-Imari L, Yang J, Pimlott N. Peer-support writing group in a community family medicine teaching unit: facilitating professional development. Can Fam Physician 2016 Dec;62(12):e724-e730 [FREE Full text] [Medline]
    29. Rozental L, Meitar D, Karnieli-Miller O. Medical students' experiences and needs from written reflective journal feedback. Med Educ 2021 Apr;55(4):505-517. [CrossRef] [Medline]
    30. Bethune C, Brown JB. Residents' use of case-based reflection exercises. Can Fam Physician 2007 Mar;53(3):471 [FREE Full text] [Medline]
    31. Hamann J, Cohen R, Leucht S, Busch R, Kissling W. Do patients with schizophrenia wish to be involved in decisions about their medical treatment? Am J Psychiatry 2005 Dec;162(12):2382-2384. [CrossRef] [Medline]
    32. Krones T, Keller H, Sönnichsen A, Sadowski EM, Baum E, Wegscheider K, et al. Absolute cardiovascular disease risk and shared decision making in primary care: a randomized controlled trial. Ann Fam Med 2008;6(3):218-227 [FREE Full text] [CrossRef] [Medline]
    33. Murray MA, Stacey D, Wilson KG, O'Connor AM. Skills training to support patients considering place of end-of-life care: a randomized control trial. J Palliat Care 2010;26(2):112-121. [Medline]
    34. Elwyn G, Edwards A, Hood K, Robling M, Atwell C, Russell I, Study Steering Group. Achieving involvement: process outcomes from a cluster randomized trial of shared decision making skill development and use of risk communication aids in general practice. Fam Pract 2004 Aug;21(4):337-346. [CrossRef] [Medline]
    35. Bieber C, Müller KG, Blumenstiel K, Schneider A, Richter A, Wilke S, et al. Long-term effects of a shared decision-making intervention on physician-patient interaction and outcome in fibromyalgia. A qualitative and quantitative 1 year follow-up of a randomized controlled trial. Patient Educ Couns 2006 Nov;63(3):357-366. [CrossRef] [Medline]
    36. Stacey D, Taljaard M, Drake ER, O'Connor AM. Audit and feedback using the brief Decision Support Analysis Tool (DSAT-10) to evaluate nurse-standardized patient encounters. Patient Educ Couns 2008 Dec;73(3):519-525. [CrossRef] [Medline]
    37. Loh A, Simon D, Wills CE, Kriston L, Niebling W, Härter M. The effects of a shared decision-making intervention in primary care of depression: a cluster-randomized controlled trial. Patient Educ Couns 2007 Aug;67(3):324-332. [CrossRef] [Medline]
    38. Haskard KB, Williams SL, DiMatteo MR, Rosenthal R, White MK, Goldstein MG. Physician and patient communication training in primary care: effects on participation and satisfaction. Health Psychol 2008 Sep;27(5):513-522. [CrossRef] [Medline]
    39. Deinzer A, Veelken R, Kohnen R, Schmieder RE. Is a shared decision-making approach effective in improving hypertension management? J Clin Hypertens (Greenwich) 2009 May;11(5):266-270 [FREE Full text] [CrossRef] [Medline]
    40. O'Cathain A, Walters SJ, Nicholl JP, Thomas KJ, Kirkham M. Use of evidence based leaflets to promote informed choice in maternity care: randomised controlled trial in everyday practice. BMJ 2002 Mar 16;324(7338):643 [FREE Full text] [CrossRef] [Medline]
    41. Fossli Jensen B, Gulbrandsen P, Dahl FA, Krupat E, Frankel RM, Finset A. Effectiveness of a short course in clinical communication skills for hospital doctors: results of a crossover randomized controlled trial (ISRCTN22153332). Patient Educ Couns 2011 Aug;84(2):163-169. [CrossRef] [Medline]
    42. Kennedy A, Bower P, Reeves D, Blakeman T, Bowen R, Chew-Graham C, Salford National Institute for Health Research Gastrointestinal programme Grant Research Group. Implementation of self management support for long term conditions in routine primary care settings: cluster randomised controlled trial. BMJ 2013 May 13;346:f2882 [FREE Full text] [CrossRef] [Medline]
    43. Feng B, Srinivasan M, Hoffman JR, Rainwater JA, Griffin E, Dragojevic M, et al. Physician communication regarding prostate cancer screening: analysis of unannounced standardized patient visits. Ann Fam Med 2013;11(4):315-323 [FREE Full text] [CrossRef] [Medline]
    44. Tinsel I, Buchholz A, Vach W, Siegel A, Dürk T, Buchholz A, et al. Shared decision-making in antihypertensive therapy: a cluster randomised controlled trial. BMC Fam Pract 2013 Sep 11;14:135 [FREE Full text] [CrossRef] [Medline]
    45. Bernhard J, Butow P, Aldridge J, Juraskova I, Ribi K, Brown R. Communication about standard treatment options and clinical trials: can we teach doctors new skills to improve patient outcomes? Psychooncology 2012 Dec;21(12):1265-1274. [CrossRef] [Medline]
    46. Hess EP, Knoedler MA, Shah ND, Kline JA, Breslin M, Branda ME, et al. The chest pain choice decision aid: a randomized trial. Circ Cardiovasc Qual Outcomes 2012 May;5(3):251-259. [CrossRef] [Medline]
    47. Koerner M, Wirtz M, Michaelis M, Ehrhardt H, Steger AK, Zerpies E, et al. A multicentre cluster-randomized controlled study to evaluate a train-the-trainer programme for implementing internal and external participation in medical rehabilitation. Clin Rehabil 2014 Jan;28(1):20-35. [CrossRef] [Medline]
    48. Mathers N, Ng CJ, Campbell MJ, Colwell B, Brown I, Bradley A. Clinical effectiveness of a patient decision aid to improve decision quality and glycaemic control in people with diabetes making treatment choices: a cluster randomised controlled trial (PANDAs) in general practice. BMJ Open 2012 Nov 5;2(6):e001469 [FREE Full text] [CrossRef] [Medline]
    49. Rise MB, Eriksen L, Grimstad H, Steinsbekk A. The short-term effect on alliance and satisfaction of using patient feedback scales in mental health out-patient treatment. A randomised controlled trial. BMC Health Serv Res 2012 Oct 03;12:348 [FREE Full text] [CrossRef] [Medline]
    50. Sheridan SL, Golin C, Bunton A, Lykes JB, Schwartz B, McCormack L, et al. Shared decision making for prostate cancer screening: the results of a combined analysis of two practice-based randomized controlled trials. BMC Med Inform Decis Mak 2012 Nov 13;12:130 [FREE Full text] [CrossRef] [Medline]
    51. Wilkes MS, Day FC, Srinivasan M, Griffin E, Tancredi DJ, Rainwater JA, et al. Pairing physician education with patient activation to improve shared decisions in prostate cancer screening: a cluster randomized controlled trial. Ann Fam Med 2013;11(4):324-334 [FREE Full text] [CrossRef] [Medline]
    52. Légaré F, Labrecque M, Cauchon M, Castel J, Turcotte S, Grimshaw J. Training family physicians in shared decision-making to reduce the overuse of antibiotics in acute respiratory infections: a cluster randomized trial. CMAJ 2012 Sep 18;184(13):E726-E734 [FREE Full text] [CrossRef] [Medline]
    53. Cooper LA, Ghods Dinoso BK, Ford DE, Roter DL, Primm AB, Larson SM, et al. Comparative effectiveness of standard versus patient-centered collaborative care interventions for depression among African Americans in primary care settings: the BRIDGE study. Health Serv Res 2013 Feb;48(1):150-174 [FREE Full text] [CrossRef] [Medline]
    54. Cooper LA, Roter DL, Carson KA, Bone LR, Larson SM, Miller 3rd ER, et al. A randomized trial to improve patient-centered care and hypertension control in underserved primary care patients. J Gen Intern Med 2011 Nov;26(11):1297-1304 [FREE Full text] [CrossRef] [Medline]
    55. Epstein RM, Duberstein PR, Fenton JJ, Fiscella K, Hoerger M, Tancredi DJ, et al. Effect of a patient-centered communication intervention on oncologist-patient communication, quality of life, and health care utilization in advanced cancer: the VOICE randomized clinical trial. JAMA Oncol 2017 Jan 01;3(1):92-100 [FREE Full text] [CrossRef] [Medline]
    56. Sanders AR, Bensing JM, Essed MA, Magnée T, de Wit NJ, Verhaak PF. Does training general practitioners result in more shared decision making during consultations? Patient Educ Couns 2017 Mar;100(3):563-574. [CrossRef] [Medline]
    57. Härter M, Buchholz A, Nicolai J, Reuter K, Komarahadi F, Kriston L, et al. Shared decision making and the use of decision aids. Dtsch Arztebl Int 2015 Oct 02;112(40):672-679 [FREE Full text] [CrossRef] [Medline]
    58. Tai-Seale M, Elwyn G, Wilson CJ, Stults C, Dillon EC, Li M, et al. Enhancing shared decision making through carefully designed interventions that target patient and provider behavior. Health Aff (Millwood) 2016 Apr;35(4):605-612. [CrossRef] [Medline]
    59. Ampe S, Sevenants A, Smets T, Declercq A, Van Audenhove C. Advance care planning for nursing home residents with dementia: influence of 'we DECide' on policy and practice. Patient Educ Couns 2017 Jan;100(1):139-146. [CrossRef] [Medline]
    60. Cox ED, Jacobsohn GC, Rajamanickam VP, Carayon P, Kelly MM, Wetterneck TB, et al. A family-centered rounds checklist, family engagement, and patient safety: a randomized trial. Pediatrics 2017 May;139(5):e20161688 [FREE Full text] [CrossRef] [Medline]
    61. Coylewright M, Dick S, Zmolek B, Askelin J, Hawkins E, Branda M, et al. PCI choice decision aid for stable coronary artery disease: a randomized trial. Circ Cardiovasc Qual Outcomes 2016 Nov;9(6):767-776. [CrossRef] [Medline]
    62. Butow P, Brown R, Aldridge J, Juraskova I, Zoller P, Boyle F, et al. Can consultation skills training change doctors' behaviour to increase involvement of patients in making decisions about standard treatment and clinical trials: a randomized controlled trial. Health Expect 2015 Dec;18(6):2570-2583 [FREE Full text] [CrossRef] [Medline]
    63. Dillon EC, Stults CD, Wilson C, Chuang J, Meehan A, Li M, et al. An evaluation of two interventions to enhance patient-physician communication using the observer OPTION measure of shared decision making. Patient Educ Couns 2017 Oct;100(10):1910-1917. [CrossRef] [Medline]
    64. Hirsch O, Keller H, Albohn-Kühne C, Krones T, Donner-Banzhoff N. Satisfaction of patients and primary care physicians with shared decision making. Eval Health Prof 2010 Sep;33(3):321-342. [CrossRef] [Medline]
    65. Bieber C, Müller KG, Blumenstiel K, Schuller-Roma B, Richter A, Hochlehnert A, et al. Partizipative Entscheidungsfindung (PEF) mit chronischen Schmerzpatienten. Der Patient als Partner im medizinischen Entscheidungsprozess. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2004 Oct;47(10):985-991. [CrossRef] [Medline]
    66. Bieber C, Nicolai J, Gschwendtner K, Müller N, Reuter K, Buchholz A, et al. How does a Shared Decision-Making (SDM) intervention for oncologists affect participation style and preference matching in patients with breast and colon cancer? J Cancer Educ 2018 Jun;33(3):708-715 [FREE Full text] [CrossRef] [Medline]
    67. Edwards A, Elwyn G, Hood K, Atwell C, Robling M, Houston H, Study Steering Group. Patient-based outcome results from a cluster randomized trial of shared decision making skill development and use of risk communication aids in general practice. Fam Pract 2004 Aug;21(4):347-354. [CrossRef] [Medline]
    68. Longo MF, Cohen DR, Hood K, Edwards A, Robling M, Elwyn G, et al. Involving patients in primary care consultations: assessing preferences using discrete choice experiments. Br J Gen Pract 2006 Jan;56(522):35-42 [FREE Full text] [Medline]
    69. Kennedy A, Chew-Graham C, Blakeman T, Bowen A, Gardner C, Protheroe J, et al. Delivering the WISE (Whole Systems Informing Self-Management Engagement) training package in primary care: learning from formative evaluation. Implement Sci 2010 Jan 29;5:7 [FREE Full text] [CrossRef] [Medline]
    70. Allaire AS, Labrecque M, Giguere A, Gagnon MP, Légaré F. What motivates family physicians to participate in training programs in shared decision making? J Contin Educ Health Prof 2012;32(2):98-107. [CrossRef] [Medline]
    71. Légaré F, Guerrier M, Nadeau C, Rhéaume C, Turcotte S, Labrecque M. Impact of DECISION + 2 on patient and physician assessment of shared decision making implementation in the context of antibiotics use for acute respiratory infections. Implement Sci 2013 Dec 26;8:144 [FREE Full text] [CrossRef] [Medline]
    72. Rise MB, Eriksen L, Grimstad H, Steinsbekk A. The long-term effect on mental health symptoms and patient activation of using patient feedback scales in mental health out-patient treatment. A randomised controlled trial. Patient Educ Couns 2016 Jan;99(1):164-168. [CrossRef] [Medline]
    73. What study designs should be included in an EPOC review and what should they be called? Cochrane Effective Practice. 2017.   URL: https:/​/epoc.​cochrane.org/​sites/​epoc.cochrane.org/​files/​public/​uploads/​EPOC%20Study%20Designs%20About.​pdf [accessed 2022-11-14]
    74. Higgins J, Altman DG, Sterne JA. Chapter 8: Assessing risk of bias in included studies. In: Higgins JP, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. London, UK: The Cochrane Collaboration; 2011:243-296.
    75. Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, et al. GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes. J Clin Epidemiol 2013 Feb;66(2):173-183. [CrossRef] [Medline]
    76. EPOC Worksheets for preparing a Summary of Findings (SoF) table using GRADE. Cochrane Effective Practice Organisation of Care. 2017.   URL: https:/​/epoc.​cochrane.org/​sites/​epoc.cochrane.org/​files/​public/​uploads/​Resources-for-authors2017/​worksheets_for_preparing_a_summary_of_findings_using_grade.​docx [accessed 2022-11-14]
    77. Leyland R, Heath M, Neve H, Maynard V. Structured reflection on shared decision making. Clin Teach 2021 Feb;18(1):55-61. [CrossRef] [Medline]
    78. Kolb B. Functions of the frontal cortex of the rat: a comparative review. Brain Res 1984 Nov;320(1):65-98. [CrossRef] [Medline]
    79. Légaré F, Stacey D, Brière N, Fraser K, Desroches S, Dumont S, et al. Healthcare providers' intentions to engage in an interprofessional approach to shared decision-making in home care programs: a mixed methods study. J Interprof Care 2013 May;27(3):214-222 [FREE Full text] [CrossRef] [Medline]
    80. Coulter A. Measuring what matters to patients. BMJ 2017 Feb 20;356:j816. [CrossRef] [Medline]
    81. Patient experience in Canadian hospitals, 2022. Canadian Institute for Health Information. 2022.   URL: https://www.cihi.ca/en/patient-experience-in-canadian-hospitals-2022 [accessed 2022-10-03]

    ‎
    EPOC: Effective Practice and Organization of Care
    HCPROM: health care professional–reported outcome measurement
    OBOM: observer-reported outcome measurement
    PREM: patient-reported experience measure
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    PROM: patient-reported outcome measurement
    RCT: randomized controlled trial
    SDM: shared decision-making

    Edited by N Zary, T Leung; submitted 19.08.22; peer-reviewed by K Moens; comments to author 13.09.22; revised version received 05.10.22; accepted 31.10.22; published 07.12.22

    Copyright

    ©Ndeye Thiab Diouf, Angèle Musabyimana, Virginie Blanchette, Johanie Lépine, Sabrina Guay-Bélanger, Marie-Claude Tremblay, Maman Joyce Dogba, France Légaré. Originally published in JMIR Medical Education (https://mededu.jmir.org), 07.12.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on https://mededu.jmir.org/, as well as this copyright and license information must be included.