This paper is in the following e-collection/theme issue:

Reviews in Medical Education (54) Continuing Medical Education (CME) for Doctors (120) New Methods and Approaches in Medical Education (199) Communities of Practice (22) Continuing Medical Education (CME) for Allied Health Professionals (121) Evaluation of Medical Education (62) New Resources for Medical Education (54) Peer-Learning in Medical and Health Education (10) Models and Frameworks in Medical Education (25) Teamwork (48)

Published on in Vol 8, No 2 (2022): Apr-Jun

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/36948, first published .
Use of the CPD-REACTION Questionnaire to Evaluate Continuing Professional Development Activities for Health Professionals: Systematic Review

Use of the CPD-REACTION Questionnaire to Evaluate Continuing Professional Development Activities for Health Professionals: Systematic Review

Use of the CPD-REACTION Questionnaire to Evaluate Continuing Professional Development Activities for Health Professionals: Systematic Review

Authors of this article:

Gloria Ayivi-Vinz1, 2, 3 Author Orcid Image ;   Felly Bakwa Kanyinga1, 2, 3 Author Orcid Image ;   Lysa Bergeron2 Author Orcid Image ;   Simon Décary4 Author Orcid Image ;   Évèhouénou Lionel Adisso1, 2, 3 Author Orcid Image ;   Hervé Tchala Vignon Zomahoun1, 5 Author Orcid Image ;   Sam J Daniel6 Author Orcid Image ;   Martin Tremblay6 Author Orcid Image ;   Karine V Plourde1, 2 Author Orcid Image ;   Sabrina Guay-Bélanger1, 2 Author Orcid Image ;   France Légaré1, 2, 5, 7 Author Orcid Image
Article Authors Cited by (6) Tweetations (5) Metrics

    Review

    1VITAM – Centre de Recherche en Santé Durable, Centre Intégré Universitaire de Santé et de Services Sociaux de la Capitale-Nationale, Université Laval, Quebec, QC, Canada

    2Tier 1 Canada Research Chair in Shared Decision Making and Knowledge Translation, Université Laval, Quebec, QC, Canada

    3Department of Social and Preventive Medicine, Faculty of Medicine, Université Laval, Quebec, QC, Canada

    4School of Rehabilitation, Faculty of Medicine and Health Sciences, Université de Sherbrooke, Sherbrooke, QC, Canada

    5Unité de Soutien SSA Québec, Université Laval, Quebec, QC, Canada

    6Direction du Développement Professionnel Continu, Fédération des Médecins Spécialistes du Québec, Montreal, QC, Canada

    7Department of Family Medicine and Emergency Medicine, Faculty of Medicine, Université Laval, Quebec, QC, Canada

    Corresponding Author:

    France Légaré, BSc, MSc, MD, PhD, FCMF

    VITAM – Centre de Recherche en Santé Durable, Centre Intégré Universitaire de Santé et de Services Sociaux de la Capitale-Nationale

    Université Laval

    2480 Chemin de la Canardière

    Quebec, QC, G1J 2G1

    Canada

    Phone: 1 418 663 5919

    Email: France.Legare@mfa.ulaval.ca


    Background: Continuing professional development (CPD) is essential for physicians to maintain and enhance their knowledge, competence, skills, and performance. Web-based CPD plays an essential role. However, validated theory–informed measures of their impact are lacking. The CPD-REACTION questionnaire is a validated theory–informed tool that evaluates the impact of CPD activities on clinicians’ behavioral intentions.

    Objective: We aimed to review the use of the CPD-REACTION questionnaire, which measures the impact of CPD activities on health professionals’ intentions to change clinical behavior. We examined CPD activity characteristics, ranges of intention, mean scores, score distributions, and psychometric properties.

    Methods: We conducted a systematic review informed by the Cochrane review methodology. We searched 8 databases from January 1, 2014, to April 20, 2021. Gray literature was identified using Google Scholar and Research Gate. Eligibility criteria included all health care professionals, any study design, and participants’ completion of the CPD-REACTION questionnaire either before, after, or before and after a CPD activity. Study selection, data extraction, and study quality evaluation were independently performed by 2 reviewers. We extracted data on characteristics of studies, the CPD activity (eg, targeted clinical behavior and format), and CPD-REACTION use. We used the Mixed Methods Appraisal Tool to evaluate the methodological quality of the studies. Data extracted were analyzed using descriptive statistics and the Student t test (2-tailed) for bivariate analysis. The results are presented as a narrative synthesis reported according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

    Results: Overall, 65 citations were eligible and referred to 52 primary studies. The number of primary studies reporting the use of CPD-REACTION has increased continuously since 2014 from 1 to 16 publications per year (2021). It is available in English, French, Spanish, and Dutch. Most of the studies were conducted in Canada (30/52, 58%). Furthermore, 40 different clinical behaviors were identified. The most common CPD format was e-learning (34/52, 65%). The original version of the CPD-REACTION questionnaire was used in 31 of 52 studies, and an adapted version in 18 of 52 studies. In addition, 31% (16/52) of the studies measured both the pre- and postintervention scores. In 22 studies, CPD providers were university-based. Most studies targeted interprofessional groups of health professionals (31/52, 60%).

    Conclusions: The use of CPD-REACTION has increased rapidly and across a wide range of clinical behaviors and formats, including a web-based format. Further research should investigate the most effective way to adapt the CPD-REACTION questionnaire to a variety of clinical behaviors and contexts.

    Trial Registration: PROSPERO CRD42018116492; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=116492

    JMIR Med Educ 2022;8(2):e36948

    doi:10.2196/36948

    Keywords

    CPD-REACTION; behavior; intention; education medical; continuing; health care professionals; questionnaire; web-based; continuing professional development 


    Continuing professional development (CPD) encompasses the multiple educational and developmental activities that health care professionals undertake to maintain and enhance their knowledge, skills, performance, and relationships in the provision of health care. The ultimate goal of CPD is to enhance the quality and safety of patient care and enhance both patient experience and health outcomes [1]. In recent years, web-based CPD has increased exponentially, and the recent COVID-19 pandemic has emphasized the need for more effective web-based CPD. Health professional behavior change (adoption or abandonment of a practice) is a long and complex process [2]. The Kirkpatrick model conceptualizes a framework for CPD assessment that measures four distinct outcome levels: satisfaction; knowledge, skills, or attitudes; transfer of learning to practice (ie, behavior); and organizational outcomes such as productivity and quality [3].

    The lack of validated instruments informed by behavior change theories for assessing CPD outcomes has slowed the advancement of the CPD knowledge base [4]. In 2011, a consortium of CPD providers from the Province of Quebec, Canada, developed a tool to assess Kirkpatrick level 3 outcomes (transfer of learning to practice) based on an integrated model explaining behavior change among health professionals [5,6]. This model posits that intention is a strong predictor of behavior, and that behavioral intention, in turn, is influenced by beliefs about capabilities, beliefs about consequences, moral norms, and social influences [5]. The resulting tool, the CPD-REACTION questionnaire, is a comprehensive, theory-based, validated instrument for assessing the impact of accredited CPD activities on clinical behavioral intention [7,8]. During the past 10 years, it has been used in regular evaluations of the effects of CPD activities on behavior change by major CPD providers such as the Federation of Medical Specialists of Quebec (Fédération des Médecins Spécialistes du Québec) and to assess training for a wide variety of other health care professionals [9-12].

    However, the current range of CPD-REACTION use remains unknown. Moreover, the clinical topics of CPD activities evaluated using the tool, the types of clinical behaviors sought, how often it has been used to evaluate web-based CPD, what kind of health care professionals are targeted by such CPD activities, and how the results shown by CPD-REACTION in terms of behavior change intentions are used, are also unknown. Although tool validity has been demonstrated in the Canadian context [8], other evidence on its cross-cultural validity and psychometric properties is still lacking. Therefore, we aimed to systematically review studies that have used the CPD-REACTION questionnaire.

    Our research questions were as follows: (1) What are the characteristics of CPD activities in studies using CDP-REACTION? (2) What are the ranges of behavioral change intentions, mean scores, and distribution of scores across all studies that used CPD-REACTION? (3) What are the psychometric properties of CPD-REACTION?


    Ethics Approval and Consent to Participate

    As this research was based on published studies, ethics approval was not required for this systematic review. The protocol was registered in the PROSPERO (International Prospective Register of Systematic Reviews) registry under the number CRD42018116492 on December 4, 2018. The main change to the protocol was the inclusion of references to studies reported in a language other than English and French.

    Study Design

    Informed by the Cochrane review methodology [13], we conducted a systematic review and followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 statement [14].

    Eligibility Criteria

    Informed by the PICOS (Population, Intervention, Comparator, Outcomes, Study design) model [15], the inclusion criteria were as follows: (1) Population- the target population considered for this review included all individuals working in health fields who completed an original, translated, or adapted version of the CPD-REACTION questionnaire before, after, or before and after an activity. There was no age restriction or restriction of health care professions (eg, physician, nurse, or any other health professional). They could be working in the public or private sector, in the process of training, or have already graduated; (2) Intervention- not specified; (3) Comparator- not specified; (4) Outcomes- The original, adapted, or translated version of the 12-item CPD-REACTION was used to assess the intention to change either a clinical practice or a health behavior. On the basis of the Godin integrated model, this tool is a questionnaire composed of 12 items that measure behavioral intention and some of its predictors; that is, beliefs about capabilities, beliefs about consequences, moral norms, and social influences [5,7]. The 5 constructs of the CPD-REACTION questionnaire have been validated, with Cronbach coefficients for the constructs varying from 0.77 to 0.85 [7,8]; (5) Study Design-Any study design was considered: randomized clinical trials (individual, group, or cluster, including stepped-wedge), before-and-after studies, translation studies, ecology studies, qualitative studies, or any mixed-study design (if they included the use of CPD-REACTION). Only primary studies were considered for inclusion in this systematic review. Therefore, we did not include any systematic reviews. These articles could be reported in any language.

    Information Sources

    The literature search was performed using eight databases: Embase, MEDLINE/PubMed, Web of Science, ERIC-EBSCO, PsycINFO-ovid, CINAHL, Social Sciences Full Text-EBSCO, and Academic Search Premier EBSCO. A temporal filter was applied from January 1, 2014, to April 20, 2021, because CPD-REACTION was published in 2014 [7]. We also performed a forward citation search using Google Scholar and Research Gate to identify studies citing the 3 main studies on the development and validation of CPD-REACTION (Multimedia Appendix 1).

    Search Strategy

    The first phase of developing the search strategy was carried out on PubMed and reviewed by the authors to ensure that the concepts covered all research questions. This strategy was then translated into expressions that were adapted to each database. A documentary research expert revised the search strategy and the final version was based on three key concepts: “continuing education,” “CPD-REACTION questionnaire,” and “questionnaires.” These key concepts were searched using a combination of controlled vocabulary (MeSH [Medical Subject Headings] terms) and free-text search queries (Multimedia Appendix 1).

    Selection Process

    Duplicates were identified using EndNote ×9 [16] and manual checking. First, reviewers (GA-V, FBK, LB, and LS) performed an independent selection based on the title and abstract. Second, all relevant references were considered for selection by full text (GA-V, FBK, LL, LB, and LS). An internet-based system, Covidence [17], was used to complete this step. The 2 reviewers then discussed and resolved any disagreement to obtain a consensus on study selection according to the eligibility criteria and, if necessary, consulted a third author (KVP). The reasons for exclusion of articles were documented.

    Data Extraction

    A coding guide and corresponding extraction grid were developed and tested by the reviewers. The reviewers (GA-V, FBK, LL, LB, and LS) individually extracted data from the included studies. The reviewers discussed and resolved any disagreement.

    Qualitative and quantitative data were extracted. The main groups of variables were (1) study characteristics, including author names, study design, study objectives, country, and type of CPD activity; (2) characteristics of the study participants, such as profession, setting (eg, hospital or university), average age, sex, study population (eg, single profession, mixed professions, and patients included); (3) CPD activity characteristics, such as country in which it was used, health field, duration of CPD activity, when tool was used (eg, pre- or post-CPD activity), format of CPD activity (eg, web-based), title of CPD activity, clinical behavior change targeted; (4) CPD-REACTION version used (original or adapted), adaptations to the questionnaire, eg, translations; (5) score values (mean, median, SD, minimum, and maximum) for all constructs measured, that is, behavioral intention, beliefs about capabilities, social influence, moral norm, and beliefs about consequences; (6) psychometric properties (Cronbach α, κ, or Cohen d).

    Methodological Quality Assessment of Individual Studies

    Two examiners (GA-V and FBK) assessed the quality of each identified study using the Mixed Methods Appraisal Tool (MMAT), a validated tool for evaluating the quality of qualitative, quantitative, and mixed methods studies [18]. For each type of study design, 5 criteria were evaluated and each was rated “yes,” “can’t tell,” or “no.” The tool guideline discourages the calculation of an overall score, instead suggesting presenting detailed ratings for each criterion [18] (Multimedia Appendix 2).

    Data Synthesis

    Given the large variety of behavior changes targeted by studies (clinical practice behaviors and others) and the methodological and statistical heterogeneity of studies, we performed a narrative synthesis using descriptive statistics. For the CPD-REACTION score values, we did not calculate the average scores for the construct if CPD-REACTION did not evaluate the same behavior. Instead, we summarized the construct scores based on the timing of the evaluation, that is, if it was a pre-post, only pre-evaluation, or only postevaluation. Descriptive statistics were computed using STATA (version 11; StataCorp). To summarize the target behaviors of the included studies, we performed a thematic analysis. After the analysis, we organized and summarized the main behaviors based on the emerging themes, namely, “shared decision-making,” “decision aids or toolkit,” and “others.”


    Study Selection

    We described the selection process in a PRISMA flowchart (Figure 1). A total of 9504 records were identified and 3330 (duplicates or ineligible) were removed. After screening, 65 records matched the eligibility criteria and referred to 61 publications and 52 unique studies [7-12,19-70] (Figure 1).

    ‎
    Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 flowchart.
    View this figure

    Study Characteristics

    Since 2014, the number of published studies using CPD-REACTION has increased from 1 to 16 publications in 2021 (Multimedia Appendix 3). Of all the studies, 69% (36/52) were published between 2019 and 2021 [9,32-68]. Furthermore, 58% (30/52) were located in Canada [7,8,10-12,23-25,28,31,35,38-41,44-51,53,54,56,60,63,65,67] and the rest in the United States (n=6), the United Kingdom (n=4), Australia (n=2) [30,34], Iran (n=2) [55,64], Argentina (n=1) [52], Indonesia (n=1) [32], Germany (n=1) [58], Sweden (n=1) [21], the Netherlands (n=1) [59], and Burkina Faso (n=1) [37]. In addition, two multicountry studies were reported: 1 from Brazil-China-France-Japan-Mali [68] and 1 from Canada-Vietnam [19] (Table 1; Figure 2). There were no exclusive qualitative studies (Table 1). Most study designs were mixed methods (24/52, 46%) [8,9,12,18,23,26,35,42,​43,45,47,50,55,56,61,64,67,71,72], followed by cross-sectional studies (9/52, 17%) [22,32,39,42,56,59].

    Table 1. Study and intervention characteristics (N=52).
    Study and intervention characteristicsNumber of studies, n (%a)
    Study location

    		Canada30 (58)

    		United States6 (12)

    		United Kingdom4 (8)

    		Australia2 (4)

    		Iran2 (4)

    		Argentina1 (2)

    		Burkina Faso1 (2)

    		Germany1 (2)

    		Indonesia1 (2)

    		Netherlands1 (2)

    		Sweden1 (2)

    		Canada and Vietnam1 (2)

    		China-Brazil-France-Mali-Canada-Japan1 (2)
    Study design

    		Mixed methods study24 (46)

    		Cross-sectional study9 (17)

    		Baseline and follow-up or before-after or comparative study7 (13)

    		Randomized trial4 (8)

    		Quasi-experimental study3 (6)

    		Validation study2 (4)

    		Cohort study2 (4)

    		Intervention study1 (2)
    Clinical setting

    		Multicenter academic hospitals3 (6)

    		Multicenter community hospitals11 (21)

    		Multicenter both academic and community11 (21)

    		Single-center academic hospital4 (8)

    		Single-center community hospital5 (10)

    		Not a clinical setting13 (25)

    		Not reported or not applicable5 (10)
    Type of CPDb activities

    		Course or workshop31 (60)

    		Conference1 (2)

    		Otherc CPD activities4 (8)

    		No activity pertaining to CPDd3 (6)

    		Not specified or not applicable13 (25)
    CPD activity Format

    		Web-based34 (65)

    		In person13 (25)

    		Not specified5 (10)
    Version of questionnaire used

    		Adapted18 (35)

    		Original31 (60)

    		Not specified3 (6)
    When CPD-REACTION was used

    		Preactivity6 (12)

    		Postactivity11 (21)

    		Pre- and postactivity16 (31)

    		Not specified or not applicable19 (37)
    Delivery mode of CPD-REACTION questionnaire

    		Digital platform or web-based19 (37)

    		Web-based and paper1 (2)

    		Paper copy16 (31)

    		Not specified16 (31)
    Language of CPD questionnaire used

    		Dutch1 (2)

    		Spanish1 (2)

    		English28 (54)

    		French14 (27)

    		English and French3 (6)

    		Not reported5 (10)
    Type of CPD provider

    		Government1 (2)

    		Hospital13 (25)

    		Private company6 (12)

    		University22 (42)

    		Not specified or not applicable7 (19)

    aAll percentages may not add up to 100%.

    bCPD: continuing professional development.

    cTraining or workshop combined with activities such as face-to-face meetings, media interviews, minutes documenting interactions, conferenced meetings, annual national collaboration meeting, and team meeting to watch video.

    dGuidelines application, outreach sessions.

    ‎
    Figure 2. Distribution of published studies worldwide that used the CPD-REACTION questionnaire.
    View this figure

    Characteristics of the Study Participants

    In total, CPD-REACTION was administered to 4886 participants. Even when age was mentioned, it was not possible to properly report on age because of the heterogeneity of age ranges. The sex of the participants was not reported for all studies. The authors mostly defined participants based solely on their profession. Physicians were the most represented health profession (1843/4886, 37.72%). Furthermore, 7 studies included residents or unlicensed health professionals [8,36,38,39,46,55,64]. (Table 2). In most studies, participants in CPD activities consisted of interprofessional groups (30/52, 60%) [7,9-11,19,21,27,28,31,35,37,41,43,44,46,47,50,51,53, 55-61,63,64,66-68]. Professions included nurses (5/52, 10%) [26,29,30,48,54], physicians (4/52, 8%) [38,39,49,52], social workers or other health professionals, namely occupational therapists, physiotherapists, dietitians, behavioral counselors, nutritionists, health researchers (4/52, 8%) [23-25,34,36], specialist physicians (3/52, 6%) [32,40,65], and pharmacists (2/52, 4%) [22,42]. The presence of managers or decision-makers among participants was reported in 7 (13%) out of 52 studies. The number of participants per study ranged from 8 to 489 (Figure 3).

    The largest proportion of CPD providers reported was university-based (22/52, 42%), whereas others were based in hospitals (13/52, 25%), private companies (6/52, 12%), or government (1/52, 2%).

    Table 2. Professional profiles of study participants.
    Population characteristicsFrequency, n (%a)
    Population of interest, in the studies (n=52)

    		Physicians8 (15.4)

    		Interprofessional groups31 (59.6)

    		Nurses5 (9.6)

    		Other health professionsb8 (15.4)
    Number of participants per professional group (n=4886)

    		Interprofessional groups1843 (37.7)

    		Nurses1568 (32.1)

    		Social workers and other health professionals1053 (21.6)

    		Not specified422 (8.6)
    Presence of managers or decision-makers among participants (n=52)

    		Yes7 (13.5)

    		No45 (86.5)
    Presence of residents or unlicensed health professionals among participants (n=52)

    		Yes7 (13.5)

    		No45 (86.5)

    aAll percentages may not add up to 100%.

    bPharmacists, physical therapists, physiotherapists, providers of radiation therapy, midwives, and social workers.

    ‎
    Figure 3. Boxplot of number of participants by health profession present at each continuing professional development activity.
    View this figure

    CPD Activity Characteristics

    The questionnaire was administered in four languages: French (14/52, 27%) or English (28/52, 54%) [8-11,19,21-24,​26-32,34-37,39-51,53-55,57,59-68], Spanish (1/52, 2%) [52] and Dutch (1/52, 2%) [58]. The median number of CPD activities targeting behavior change per study was 1 and varied between 1 and 9 CPD activities per study. One-quarter (11/52, 21%) of the studies used CPD-REACTION to measure behavioral change intention but were not linked to a specific CPD activity [21,22,29,31,38,40,45,46,51,53,59] (Table 1). The most common format for CPD activities was web-based or e-learning based (34/52, 65%). The duration of CPD activities ranged from 30 to 225 minutes, with an average of 115 (SD 67) minutes.

    Targeted Clinical Behavior and Scoring of CPD-REACTION

    The evaluations targeted 39 different clinical behaviors [7-12,19-70]. Thematic analysis showed that 7 (18%) out of the 39 pertained to shared decision-making [9,35,50,54,56,64,67] and 5 pertained to decision aids or toolkit implementation [53,60,61,63,66] (Table 3).

    Regarding studies reporting mean scores after the intervention (n=33) [8,12,22,25,29,30,33-36,40-44,47-49,55,56,59,60,63,​64,66], 9 studies (27%) reported both pre- and postactivity scores [8,22,25,29,42,49,56,60,66]. The scores were all higher after the intervention. Furthermore, in all 9 studies, the pre-post score ranges (2.5-5.7) were higher than in studies measuring prescores only (2.6-5.2) or postscores only (1.8-4.8; Table 4). The average difference between the pre- and postintention scores was 0.54 SD 0.13. Among the 5 CPD-REACTION constructs, social influence scored the lowest (43; Table 4).

    Table 3. Main behaviors targeted in included studies (n=39).

    		Main clinical behavior targeted in included studiesTopic theme


    		SDMaDecision aids or toolkitOthers
    1.To prescribe spirometry and to interpret the result [38,39]−b−+c
    2.To actively engage with and invite patients who are underserved for Medicine Use Reviews (MURS) [20,42,43]−−+
    3.To adopt SDM [53]+−−
    4.To engage older patients living with dementia and their caregivers in decision-making about choosing a health intervention, based on the TPB [10]+−−
    5.To use Decision Box to explain to patients the benefits and harms of the options, based on the TPB [10]−+−
    6.To use a decision aid in clinical practice after completing the web-based program “MyDiabetesPlan” [63]−+−
    7.To implement developmental coordination disorder (DCD) best practices [23]−−+
    8.To provide medical abortion [40]−−+
    9.To use COSTARS (pan-Canadian Oncology Symptom Triage and Remote Support) practice guides [45]−−+
    10.To use of 15 evidence-informed symptom practice guides for providing telephone or in-home nursing services to clients with cancer [45]−−+
    11.To engage in IP-SDM (interprofessional shared decision-making) [50]+−−
    12.To use patient decision aids [12,56,64]−+−
    13.To counsel patients regarding HIV prep therapy [22]−−+
    14.To use IP-SDM [50,55]+−−
    15.To apply the disclosure guidelines to my practice [44]−−+
    16.To apply the Situation-Background-Assessment-Recommendation (SBAR) to my practice [44]−−+
    17.To apply quality improvement strategies to solve challenges in my practice [44]−−+
    18.To practice the person-centered approach (PCA—MACHIP 2) in maternal health [37]−−+
    19.“Utiliser l’outil d’évaluation du risque de violence” (To use the Risk of Violence evaluation tool) [31]−−+
    20.To collaboratively work with and actively involve children and young people who self-harm in their care [29]−−+
    21.To use the evidence of implementing FREEDOM [46]−−+
    22.To implement the STEADI toolkit [61]−+−
    23.To report research translation and impact on the CVd [51]−−+
    24.To use SDM [35]+−−
    25.To prescribe no pharmacological treatments [36]−−+
    26.To use SDM with their next patient facing a preference-sensitive decision [56]+−−
    27.To apply a systematic framework to identify and manage patients with dementia [34]−−+
    28.To change and improve practice based on the interventions, that is, to order pneumococcal vaccines [41]−−+
    29.To use research evidence in rheumatology [21]−−+
    30.To successfully plan and implement evidence-based practice changes in health facility [27]−−+
    31.To consider probiotic recommendation in infants and toddler patients [32]−−+
    32.To perform SDM (action) among health professionals in any clinical setting [64]+−−
    33.To use an app to decide about prenatal screening [9,54]−+−
    34.To formulate a violence risk assessment and management plan [30]−−+
    35.To use de-escalation techniques during escalating aggression [30]−−+
    36.To use breakaway techniques when responding to a violent person [30]−−+
    37.To change their practice about compassion fatigue education [57]−−+
    38.To implement the 5A method training in the area of physical activity promotion [58]−−+
    39.To care for children and young people admitted to hospital with self-harm [29]−−+

    aSDM: shared decision-making.

    bNot related to theme.

    cRelated to theme.

    dCV: Curriculum Vitae.

    Table 4. Summary of pre- and postscores for all constructs of CPD-REACTION.

    		Value, nPre-CPDa activity (range)Post-CPD activity (range)
    Interventions with pre- and post-CPD scores9

    		Intention94.5-6.55.7-6.8

    		Social influence92.5-5.63.8-5.8

    		Beliefs about capabilities93.2-65.4-6.4

    		Moral norm75.51-6.76.2-6.9

    		Beliefs about consequences95.73-6.66.2-6.8
    Interventions with only prescores—b

    		Intention52.9-6.6

    		Social influence52.6-6

    		Beliefs about capabilities52.4-6.6

    		Moral norm54.3-6.8

    		Beliefs about consequences45.2-6.7
    Interventions with only postscores—

    		Intention19
    		3.4-7

    		Social influence18
    		1.8-6.3

    		Beliefs about capabilities18
    		3.9-6.8

    		Moral norm18
    		4.6-6.9

    		Beliefs about consequences18
    		4.8-4.8

    aCPD: continuing professional development.

    bAuthor did not report or measure mean scores.

    CPD-REACTION Adaptations and Psychometric Properties

    One-third (18/52, 35%) of the included studies reported having adapted CPD-REACTION [19,21,22,27,28,32,39,40,53,​55,56,58,63,66-68,73] (Table 1). Adaptations to the questionnaire reported were reformulated items (n=3) [19,28,49], using only certain construct scales (n=3 studies) [22,53,55], adding or dropping some items without reformulating the original items (n=3) [39,56,67], reporting percentages instead of score values ranging from 1 to 7 (n=2) [23,27], translation of the questionnaire into other languages (n=2) [52,58], and using a 5-point instead of 7-point Likert scale (n=1) [40]. Furthermore, more than 80% of all studies (48/52, 92%) reported the psychometric parameters of the original version of CPD-REACTION or else stated it was a validated tool [8,10,23,25,26,29-32,34-37,40-57,60,61,64-68,74]. In addition, 4 studies reported the psychometric properties of their adapted versions [28,39,52,69], with the Cronbach α of the included constructs ranging from 0.62 to 0.91.

    Risk of Bias in Studies

    Although none of the studies fully met all MMAT criteria, none were rated “no” for any criteria (Multimedia Appendix 2). In the 4 quantitative randomized trials, only the criterion “randomization appropriately performed” was met by all 4 studies [10,50,54,63], and in all 17 mixed method studies, only the criterion “adequate rationale for using a mixed methods design” was met. In all the design groups, all the criteria not rated “yes” were rated “not sure.”


    Principal Findings

    We found 61 publications of 52 unique studies that reported the use of the CPD-REACTION questionnaire to assess changes in behavioral intention among health professionals. Although the tool is aged <10 years (2014), we observed the most rapid increase in its use in the past 3 years, mostly in Canada, where it was developed. However, its use has spread to many other countries, including lower- and middle-income countries, and it is found in numerous languages (our finding of only 4 is an underrepresentation, as the team that produced the tool has agreed to translations into 8 languages) [75]. Since its inception, CPD-REACTION has been used by close to 5000 participants to target 39 clinical behaviors. The participants included 8 types of health professionals, with physicians and nurses being the most reported. Two-thirds of the studies included interprofessional clinical teams, including one in which 10 managers or decision-makers were CPD activity participants. The tool appeared to be mostly used for evaluating e-learning (n=34). In many cases, users adapted the questionnaire, such as using only certain construct scales or adding or dropping some items. The psychometric properties of CPD-REACTION reported in included studies showed that Cronbach α scores were very good, ranging from 0.62-0.91. However, few studies were designed to assess changes in intention (ie, scoring both pre- and postactivity), thus limiting the evidence regarding the responsiveness of the tool. Regarding behavioral intention to change, the mean difference of intention score was 0.54 SD 0.13 in the pre-post studies and the distribution of scores across all studies using CPD-REACTION ranged from 1.8-7. Although none of the studies fully met all MMAT criteria, none were rated “no” for any criteria. In all the design groups, all the criteria not rated “yes” were rated “not sure.”

    Significance and Comparison With Prior Work

    First, the rapid adoption of CPD-REACTION across time, countries, and languages suggests that this instrument addresses the needs of CPD developers and that they seek not only validated assessment tools but also those that are informed by behavior change theories. Recent literature on this topic tends to suggest an increasing penetration of behavior change theories in the CPD developer community [2,76,77]. The use of behavior change theory has been frequently linked to effectiveness in systematic reviews of behavioral change interventions [76,77]. More recently, strategies have also focused not only on adopting new behaviors but also on abandoning low-value or harmful behaviors. However, few behavioral theories distinguish between behavior adoption and abandonment, including the theories on which CPD-REACTION is based [78]. Future research should distinguish between the two and develop theories that support both types of behavior change [79,80].

    Second, physicians and nurses were the most represented health professionals. Most groups of participants engaging in CPD activities were interprofessional clinical teams, and 1 in 10 studies included managers or decision-makers among participants. This suggests that CPD designers are increasingly creating multidisciplinary training experiences to be shared with other stakeholders and professionals to enhance the relevance and impact of CPD [2,81]. Previous research has highlighted that including peer groups seems to be an effective approach to enhancing CPD activities and moving forward with professional practice change [82]. Future research should determine the effects of interprofessional participant groups or peer groups on CPD effectiveness.

    Third, studies using an adapted version of CPD-REACTION reported Cronbach α ranging from .62 to .91, indicating that modified instruments perform well in terms of their psychometric properties. Other studies have reported psychometric values mentioned in the original version of CPD-REACTION. We observed that overall, the behavioral change intention scores reported ranged from 2.9 to 7. In pre-post studies, the mean difference in intention scores was 0.54 SD 0.13, and the distribution of scores across all constructs ranged from 1.8 to 7. Lower scores were observed when CPD-REACTION was used either only preactivity or only postactivity. Dissemination of the user manual will aid in the use of the tool to its best advantage. A lower score could also be because of the CPD topic being more controversial and thus less likely to be implementable. Overall, the adapted versions of CPD-REACTION reported Cronbach α values, indicating that the questionnaire had good internal consistency reliability. Furthermore, our results suggest that CPD-REACTION is adaptable to digital platforms, as two-thirds of the activities were web-based.

    Fourth, using CPD-REACTION to measure construct scores, both pre- and post-CPD activity, is a helpful demonstration of the effect of CPD activities on behavioral intention and explanatory constructs. However, measuring learning outcomes for levels 3 and 4 of the Kirkpatrick model remains challenging. CPD-REACTION uses intention as a measure of behavioral intention; however, other measurement strategies are needed to directly measure behavior change. Although other outcomes such as “satisfaction of participants” were reported, the studies did not correlate these with the CPD-REACTION measures. In some studies, participants were contacted after 3 months or more to self-assess their behavior change [8]. The purpose of CPD-REACTION was not to measure its effects on patient outcomes, which is another important outcome of CPD. CPD-REACTION could be followed up by participant surveys to assess the longer-term impacts of participants’ behaviors on their practices or institutions and should use patient-reported measures. Some studies suggest that CPD programs should compare self-assessments, such as CPD-REACTION, with continuous formal participant multisource assessment by peers [73].

    Limitations

    Our systematic review used diverse strategies to find studies that had used CPD-REACTION. However, we relied on the published results and did not contact the authors of the included studies. Thus, it is possible that we may have missed studies that were not published as well as items of interest in those we included. Owing to the large number of included studies, we had to organize the information into broad categories to increase the interpretability of the data.

    Conclusions

    The CPD-REACTION questionnaire is a simple, relevant, and easy-to-use tool for assessing the effectiveness of CPD activities on health professionals’ behavioral intention and, as we have observed, to identify barriers and facilitators of behavior change. This tool has been used to evaluate CPD activities in a wide range of clinical topics and behaviors. However, most users do not measure intention both before and after the activity. Dissemination of a user manual will aid in the use of the tool to its best advantage. Further research should investigate the most effective way to adapt the CPD-REACTION questionnaire to various uses and contexts.

    Acknowledgments

    The authors wish to thank Nathalie Rheault for her dedicated assistance with the design and search strategy and Louisa Blair for language revision and editing. The authors also thank Laure Siebierski, Samy Bouderba, Angèle Musabyimana, Laura Langevin, and Yuxi Wang for their assistance with the selection process. The following bodies funded the development of “CPD-REACTION questionnaire”: The Tier 1 Canada Research Chair in Shared Decision-Making and Knowledge Translation, the Fédération des médecins spécialistes du Québec (FMSQ; Federation of Medical Specialists of Quebec), and the Conseil Québécois de développement professionnel continu des médecins (CQDPCM; Quebec Council of Continuing Professional Development for Physicians).

    Data Availability

    The data sets used and analyzed during this study are available from the corresponding author upon reasonable request.

    Authors' Contributions

    GA-V, LB, KVP, SG-B, SD, ÉLA, HTVZ, SJD, MT, and FL designed the research protocol. GA-V, FB, LB, KVP, and SD conducted study selection and data extraction. GA-V and FB performed the quality assessment of the studies. GA-V conducted data synthesis, which was revised by KVP, SG-B, MT, HTVZ, and FL. GA-V drafted the manuscript overseen by KVP, SG-B, HTVZ, LB, SD, ÉLA, SJD, MT, and FL. SJD and MT provided expertise as a continuing professional development activities developer. All authors contributed to the key intellectual content and provided consent for this version of the review to be published.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Search strategy.

    DOCX File , 31 KB
    ‎
    Multimedia Appendix 2

    Mixed Methods Appraisal Tool criteria quality assessment in the included studies.

    PDF File (Adobe PDF File), 74 KB
    ‎
    Multimedia Appendix 3

    Evolution of the number of published studies using CPD-REACTION questionnaire, 2014-2021.

    PPTX File , 68 KB
    1. Sargeant J, Bruce D, Campbell CM. Practicing physicians' needs for assessment and feedback as part of professional development. J Contin Educ Health Prof 2013;33 Suppl 1:S54-S62. [CrossRef] [Medline]
    2. Chichirez CM, Purcărea VL. Health marketing and behavioral change: a review of the literature. J Med Life 2018;11(1):15-19 [FREE Full text] [Medline]
    3. Kirkpatrick DL, Kirkpatrick JD. Evaluating training programs: the four levels. 3rd edition. San Francisco, CA, USA: Berrett-Koehler Publishers; 2006.
    4. Légaré F, Borduas F, Jacques A, Laprise R, Voyer G, Boucher A, et al. Developing a theory-based instrument to assess the impact of continuing professional development activities on clinical practice: a study protocol. Implement Sci 2011 Mar 07;6:17 [FREE Full text] [CrossRef] [Medline]
    5. Godin G, Bélanger-Gravel A, Eccles M, Grimshaw J. Healthcare professionals' intentions and behaviours: a systematic review of studies based on social cognitive theories. Implement Sci 2008 Jul 16;3:36 [FREE Full text] [CrossRef] [Medline]
    6. Ajzen I. From intentions to actions: a theory of planned behavior. In: Kuhl J, Beckmann J, editors. Action control: from cognition to behavior. Berlin, Germany: Springer; 1985:11-39.
    7. Légaré F, Borduas F, Freitas A, Jacques A, Godin G, Luconi F, CPD-KT team. Development of a simple 12-item theory-based instrument to assess the impact of continuing professional development on clinical behavioral intentions. PLoS One 2014;9(3):e91013 [FREE Full text] [CrossRef] [Medline]
    8. Légaré F, Freitas A, Turcotte S, Borduas F, Jacques A, Luconi F, et al. Responsiveness of a simple tool for assessing change in behavioral intention after continuing professional development activities. PLoS One 2017;12(5):e0176678 [FREE Full text] [CrossRef] [Medline]
    9. Abbasgholizadeh Rahimi S, Archambault PM, Ravitsky V, Lemoine ME, Langlois S, Forest JC, et al. An analytical mobile app for shared decision making about prenatal screening: protocol for a mixed methods study. JMIR Res Protoc 2019;8(10):e13321 [FREE Full text] [CrossRef] [Medline]
    10. Giguere AM, Lawani MA, Fortier-Brochu É, Carmichael PH, Légaré F, Kröger E, et al. Tailoring and evaluating an intervention to improve shared decision-making among seniors with dementia, their caregivers, and healthcare providers: study protocol for a randomized controlled trial. Trials 2018 Jun 25;19(1):332 [FREE Full text] [CrossRef] [Medline]
    11. Légaré F, Brière N, Stacey D, Lacroix G, Desroches S, Dumont S, et al. Implementing shared decision-making in interprofessional home care teams (the IPSDM-SW study): protocol for a stepped wedge cluster randomised trial. BMJ Open 2016 Nov 24;6(11):e014023 [FREE Full text] [CrossRef] [Medline]
    12. Stacey D, Taljaard M, Smylie J, Boland L, Breau RH, Carley M, et al. Implementation of a patient decision aid for men with localized prostate cancer: evaluation of patient outcomes and practice variation. Implement Sci 2016 Jul 02;11(1):87 [FREE Full text] [CrossRef] [Medline]
    13. Chandler J, Hopewell S. Cochrane methods--twenty years experience in developing systematic review methods. Syst Rev 2013 Sep 20;2:76 [FREE Full text] [CrossRef] [Medline]
    14. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015 Jan 02;350:g7647 [FREE Full text] [CrossRef] [Medline]
    15. Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res 2014 Nov 21;14:579 [FREE Full text] [CrossRef] [Medline]
    16. EndNote. Clarivate Analytics. 2018.   URL: https://endnote.com/ [accessed 2022-04-12]
    17. Covidence systematic review software. Veritas Health Innovation.   URL: https://www.covidence.org/ [accessed 2022-04-12]
    18. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract 2018 Jun;24(3):459-467. [CrossRef] [Medline]
    19. Pyke SD. The perceptions of commune health workers about an education strategy received for counselling fathers about infant and breastfeeding involvement. Brock University. 2015.   URL: https:/​/dr.​library.brocku.ca/​bitstream/​handle/​10464/​8684/​Brock_Pyke_Stephanie_2015.​pdf?sequence=1&isAllowed=y [accessed 2022-04-12]
    20. Latif A, Pollock K, Anderson C, Waring J, Solomon J, Chen LC, et al. Supporting underserved patients with their medicines: a study protocol for a patient/professional coproduced education intervention for community pharmacy staff to improve the provision and delivery of Medicine Use Reviews (MURs). BMJ Open 2016 Dec 09;6(12):e013500 [FREE Full text] [CrossRef] [Medline]
    21. Neher M, Stahl C, Festin K, Nilsen P. Implementation of evidence-based practice in rheumatology: what sociodemographic, social cognitive and contextual factors influence health professionals' use of research in practice? J Rheum Dis Treat 2016;2(3):1510039. [CrossRef]
    22. Unni EJ, Lian N, Kuykendall W. Understanding community pharmacist perceptions and knowledge about HIV preexposure prophylaxis (PrEP) therapy in a Mountain West state. J Am Pharm Assoc (2003) 2016;56(5):527-32.e1. [CrossRef] [Medline]
    23. Camden C, Rivard LM, Hurtubise K, Héguy L, Berbari J. Can a community of practice improve physical therapists' self-perceived practice in developmental coordination disorder? Phys Ther 2017 Jul 01;97(7):746-755. [CrossRef] [Medline]
    24. Demers I, Maltais DB, Lamontagne ME, Camden C, Moffet H, Turcotte S. Développement et évaluation d’une activité de transfert de connaissance pour les physiothérapeutes visant à soutenir la participation aux activités physiques chez les enfants ayant un trouble développemental de la coordination. Quebec Provincial Adaptation-Rehabilitation Research Network. 2017.   URL: http:/​/old.​repar.ca/​Fichiers/​Subvention/​7707_Demande_de_subvention_OPPQ-REPAR_Demers2017_copiefinale.​pdf [accessed 2022-04-20]
    25. Ludwig C, Bennis C, Carley M, Gifford W, Kuziemsky C, Lafreniere-Davis N, et al. Managing symptoms during cancer treatments: barriers and facilitators to home care nurses using symptom practice guides. Home Health Care Manag Pract 2017 Jun 14;29(4):224-234. [CrossRef]
    26. Manning JC, Carter T, Latif A, Horsley A, Cooper J, Armstrong M, et al. 'Our Care through Our Eyes'. Impact of a co-produced digital educational programme on nurses' knowledge, confidence and attitudes in providing care for children and young people who have self-harmed: a mixed-methods study in the UK. BMJ Open 2017 May 04;7(4):e014750 [FREE Full text] [CrossRef] [Medline]
    27. Paulo M. Implementing for success and sustainability. The University of San Francisco. 2017.   URL: https://repository.usfca.edu/cgi/viewcontent.cgi?article=1110&context=dnp [accessed 2022-04-12]
    28. Portocarrero L, Esther M. Identification des croyances saillantes des femmes enceintes sur l'utilisation d'un outil d'aide à la décision dans le contexte du dépistage prénatal de la trisomie 21 pour l'élaboration et validation d'un questionnaire. Université Laval. 2017.   URL: https://corpus.ulaval.ca/jspui/handle/20.500.11794/27801 [accessed 2022-04-12]
    29. Carter T, Latif A, Callaghan P, Manning JC. An exploration of predictors of children's nurses' attitudes, knowledge, confidence and clinical behavioural intentions towards children and young people who self-harm. J Clin Nurs 2018 Jul;27(13-14):2836-2846. [CrossRef] [Medline]
    30. Lamont S, Brunero S. The effect of a workplace violence training program for generalist nurses in the acute hospital setting: a quasi-experimental study. Nurse Educ Today 2018 Sep;68:45-52. [CrossRef] [Medline]
    31. Souli I, Vandyk A, Versailles D, Marcoux I, Salvador A, Peterson WE, et al. Les obstacles et les facilitateurs à l’utilisation de l’évaluation du risque de violence auprès des personnes ayant des troubles mentaux: perceptions du personnel soignant. Rech Soins Infirm 2018 Jun(133):45-57. [CrossRef] [Medline]
    32. Wagiu Basrowi R, Krisnamurti D, Wibowo Y, Vandenplas Y. Factors influencing probiotics recommendation among pediatricians in Indonesia. Integr Food Nutr Metab 2019;6(5):1-4. [CrossRef]
    33. Bentley M, Kerr R, Ginger M, Karagoz J. General practice training Tasmania research note: the social influence on behavioural intentions of doctors and nurses to dementia care in general practice. Soc Influ 2019;5:8-7 [FREE Full text]
    34. Bentley MW, Kerr R, Ginger M, Karagoz J. Behavioural change in primary care professionals undertaking online education in dementia care in general practice. Aust J Prim Health 2019 May 29;25(3):244. [CrossRef] [Medline]
    35. Boland L, Lawson ML, Graham ID, Légaré F, Dorrance K, Shephard A, et al. Post-training shared decision making barriers and facilitators for pediatric healthcare providers: a mixed-methods study. Acad Pediatr 2019;19(1):118-129. [CrossRef] [Medline]
    36. Booth LS. Provider attitudes on using nonpharmacological methods for pain management in acute care - post an educational intervention. University of Arizona. 2019.   URL: https://repository.arizona.edu/handle/10150/634386 [accessed 2022-04-12]
    37. Cossou-Gbeto I. Évaluation de la formation portant sur l'approche centrée sur la personne dans le cadre du projet d'amélioration de la santé des mères et des enfants. Université Laval. 2019.   URL: https://corpus.ulaval.ca/jspui/handle/20.500.11794/37536 [accessed 2019-12-12]
    38. Desjardins A. Évaluation de l'intention des médecins de famille enseignants et des résidents en médecine familiale de prescrire et d'interpréter la spirométrie: une étude descriptive transversale. Université Laval. 2019.   URL: https://corpus.ulaval.ca/jspui/handle/20.500.11794/33776 [accessed 2022-04-12]
    39. Desjardins A, Boulay ME, Gagné M, Simon M, Boulet LP. Family medicine physician teachers and residents' intentions to prescribe and interpret spirometry: a descriptive cross-sectional study. J Asthma 2020 Feb;57(2):149-159. [CrossRef] [Medline]
    40. Devane C, Renner RM, Munro S, Guilbert É, Dunn S, Wagner MS, et al. Implementation of mifepristone medical abortion in Canada: pilot and feasibility testing of a survey to assess facilitators and barriers. Pilot Feasibility Stud 2019;5:126 [FREE Full text] [CrossRef] [Medline]
    41. Dominguez MJ. Increase in pneumococcal immunization in adults over 65 years of age in a federally qualified health center. Liberty University. 2019.   URL: https://digitalcommons.liberty.edu/doctoral/1992/ [accessed 2022-04-12]
    42. Latif A, Gulzar N, Abbasi N, Choudhary S. Improving provision of Medicine Use Reviews (MURs) to medically under-served groups: a baseline/follow-up comparison study investigating the impact of a co-produced digital learning resource. Int J Pharm Pract 2019 Apr;27(S2):31-32. [CrossRef]
    43. Latif A, Waring J, Chen LC, Pollock K, Solomon J, Gulzar N, et al. Supporting the provision of pharmacy medication reviews to marginalised (medically underserved) groups: a before/after questionnaire study investigating the impact of a patient-professional co-produced digital educational intervention. BMJ Open 2019 Sep 17;9(9):e031548 [FREE Full text] [CrossRef] [Medline]
    44. Luconi F, Boillat M, Mak S, Chartrand D, Korah N, Daly M, et al. Patient safety and quality of care are everybody's business: evaluating the impact of a continuing professional development program beyond satisfaction. MedEdPublish 2019;8:46. [CrossRef]
    45. Ludwig C, Renaud J, Barbera L, Carley M, Henry C, Jolicoeur L, et al. Factors influencing the use by radiation therapists of cancer symptom guides: a mixed-methods study. Curr Oncol 2019 Feb;26(1):56-64 [FREE Full text] [CrossRef] [Medline]
    46. Miller KJ, Park JE, Ramanathan K, Abel J, Zhao Y, Mamdani A, et al. Examining coronary revascularization practice patterns for diabetics: perceived barriers, facilitators, and implications for knowledge translation. Can J Cardiol 2020 Aug;36(8):1236-1243. [CrossRef] [Medline]
    47. Norman WV, Munro S, Brooks M, Devane C, Guilbert E, Renner R, et al. Could implementation of mifepristone address Canada's urban-rural abortion access disparity: a mixed-methods implementation study protocol. BMJ Open 2019 Apr 20;9(4):e028443 [FREE Full text] [CrossRef] [Medline]
    48. Stacey D, Ludwig C, Truant T, Carley M, Bennis C, Gifford W, et al. Implementing practice guides to improve cancer symptom management in homecare: a comparative case study. Home Health Care Manag Pract 2018;31(3):139-146. [CrossRef]
    49. Understanding Pain: From Cells to Brain to Individual Perceptions. Canadian Journal of Pain 2019 Apr 02;3(1):A73-A189. [CrossRef]
    50. Adekpedjou R, Haesebaert J, Stacey D, Brière N, Freitas A, Rivest LP, et al. Variations in factors associated with healthcare providers' intention to engage in interprofessional shared decision making in home care: results of two cross-sectional surveys. BMC Health Serv Res 2020 Mar 12;20(1):203 [FREE Full text] [CrossRef] [Medline]
    51. Boland L, Brosseau L, Caspar S, Graham ID, Hutchinson AM, Kothari A, et al. Reporting health research translation and impact in the curriculum vitae: a survey. Implement Sci Commun 2020;1:20 [FREE Full text] [CrossRef] [Medline]
    52. Fraguas L, Vietto V, Arceo D, Vázquez Peña F, Durante E. Adaptation and validation of the Argentine version of an instrument to assess continuing professional development activities. Arch Argent Pediatr 2020 Apr;118(2):125-129 [FREE Full text] [CrossRef] [Medline]
    53. Giguere AM, Bogza LM, Coudert L, Carmichael PH, Renaud JS, Légaré F, et al. Development of the IcanSDM scale to assess primary care clinicians' ability to adopt shared decision-making. medRxiv. Preprint posted online on July 2, 2020. [CrossRef]
    54. Poulin Herron A, Agbadje TT, Cote M, Djade CD, Roch G, Rousseau F, et al. Web-based training for nurses on shared decision making and prenatal screening for down syndrome: protocol for a randomized controlled trial. JMIR Res Protoc 2020 Oct 29;9(10):e17878 [FREE Full text] [CrossRef] [Medline]
    55. Keshmiri F, Rezai M, Tavakoli N. The effect of interprofessional education on healthcare providers' intentions to engage in interprofessional shared decision-making: perspectives from the theory of planned behaviour. J Eval Clin Pract 2020 Aug;26(4):1153-1161. [CrossRef] [Medline]
    56. Lawani MA, Côté L, Coudert L, Morin M, Witteman HO, Caron D, et al. Professional training on shared decision making with older adults living with neurocognitive disorders: a mixed-methods implementation study. BMC Med Inform Decis Mak 2020 Aug 12;20(1):189 [FREE Full text] [CrossRef] [Medline]
    57. McWilliams S. Addressing compassion fatigue of a family caregiver of adolescents in a mental health setting. Int J Nurs Health Care Res 2020;3:1206. [CrossRef]
    58. Peters S, Pfeifer K. Gesundheitsfachleute als Anbieter bewegungsbezogener Interventionen in der gesundheitlichen Versorgung. Deutsche Digitale Bibliothek. 2020.   URL: https://www.deutsche-digitale-bibliothek.de/item/NF5TOQ7AFIDA6ZW6NYGNXYCN266S7RK3 [accessed 2022-04-12]
    59. Slootweg YM, Walg C, Koelewijn JM, Van Kamp IL, De Haas M. Knowledge, attitude and practices of obstetric care providers towards maternal red-blood-cell immunization during pregnancy. Vox Sang 2020 Apr;115(3):211-220 [FREE Full text] [CrossRef] [Medline]
    60. Stacey D, Taljaard M, Breau RH, Baba N, Blackmore T, Boland L, et al. A patient decision aid for men with localized prostate cancer: a comparative case study of natural implementation approaches. Cancer Nurs 2020;43(1):E10-E21. [CrossRef] [Medline]
    61. Urban K, Wright PB, Hester AL, Curran G, Rojo M, Tsai PF. Evaluation of an education strategy versus usual care to implement the STEADI algorithm in primary care clinics in an academic medical center. Clin Interv Aging 2020;15:1059-1066 [FREE Full text] [CrossRef] [Medline]
    62. Urban KL. Facilitating implementation of fall prevention in primary care practice. University of Arkansas for Medical Sciences. 2020.   URL: https:/​/www.​proquest.com/​openview/​7a20fbfa6b30d5b3261d69f6d9c3bd76/​1?pq-origsite=gscholar&cbl=18750&diss=y [accessed 2022-04-12]
    63. Yu C, Choi D, Bruno BA, Thorpe KE, Straus SE, Cantarutti P, et al. Impact of MyDiabetesPlan, a web-based patient decision aid on decisional conflict, diabetes distress, quality of life, and chronic illness care in patients with diabetes: cluster randomized controlled trial. J Med Internet Res 2020 Sep 30;22(9):e16984 [FREE Full text] [CrossRef] [Medline]
    64. Abbasgholizadeh Rahimi S, Rodriguez C, Croteau J, Sadeghpour A, Navali A, Légaré F. Continuing professional education of Iranian healthcare professionals in shared decision-making: lessons learned. BMC Health Serv Res 2021 Mar 12;21(1):225 [FREE Full text] [CrossRef] [Medline]
    65. Bergeron L. Identification des facteurs influençant l’intention des médecins spécialistes d’adopter un comportement clinique ciblé par des formations de développement professionnel continu. Université Laval. 2021.   URL: https://corpus.ulaval.ca/jspui/handle/20.500.11794/67901?locale=fr [accessed 2022-04-12]
    66. Kepper MM, Walsh-Bailey C, Brownson RC, Kwan BM, Morrato EH, Garbutt J, et al. Development of a health information technology tool for behavior change to address obesity and prevent chronic disease among adolescents: designing for dissemination and sustainment using the ORBIT model. Front Digit Health 2021;3:648777 [FREE Full text] [CrossRef] [Medline]
    67. Lawani MA, Turgeon Y, Côté L, Légaré F, Witteman HO, Morin M, et al. User-centered and theory-based design of a professional training program on shared decision-making with older adults living with neurocognitive disorders: a mixed-methods study. BMC Med Inform Decis Mak 2021 Feb 17;21(1):59 [FREE Full text] [CrossRef] [Medline]
    68. Ridde V, Gautier L, Dagenais C, Chabrol F, Hou R, Bonnet E, et al. Learning from public health and hospital resilience to the SARS-CoV-2 pandemic: protocol for a multiple case study (Brazil, Canada, China, France, Japan, and Mali). Health Res Policy Syst 2021 May 06;19(1):76 [FREE Full text] [CrossRef] [Medline]
    69. Fraguas L, Vietto V, Arceo D, Vázquez Peña F, Durante E. Adaptación y validación de la versión argentina de un instrumento para evaluar actividades de desarrollo profesional continuo. Arch Argent Pediatr 2020;118(2):125-129. [CrossRef]
    70. Bentley MW, Kerr R, Ginger M, Karagoz J. DCTEP Research Note_June2019.pdf. ResearchGate. 2019 Jun.   URL: https://www.researchgate.net/publication/333678470_DCTEP_Research_Note_June2019pdf [accessed 2022-04-20]
    71. Martinengo L, Yeo NJ, Markandran KD, Olsson M, Kyaw BM, Car LT. Digital health professions education on chronic wound management: a systematic review. Int J Nurs Stud 2020 Apr;104:103512. [CrossRef] [Medline]
    72. Ngenzi JL, Scott RE, Mars M. Information and communication technology to enhance continuing professional development (CPD) and continuing medical education (CME) for Rwanda: a scoping review of reviews. BMC Med Educ 2021 Apr 29;21(1):245 [FREE Full text] [CrossRef] [Medline]
    73. Ramani S, McMahon GT, Armstrong EG. Continuing professional development to foster behaviour change: from principles to practice in health professions education. Med Teach 2019 Sep;41(9):1045-1052. [CrossRef] [Medline]
    74. Sinclair PM, Kable A, Levett-Jones T, Booth D. The effectiveness of Internet-based e-learning on clinician behaviour and patient outcomes: a systematic review. Int J Nurs Stud 2016 May;57:70-81. [CrossRef] [Medline]
    75. Légaré F. Le développement professionnel continu en partenariat pour l'application des connaissances. Université Laval. 2017.   URL: http://www.decision.chaire.fmed.ulaval.ca/recherche/5d8a3c8fb4956133050a2bad [accessed 2022-04-12]
    76. Colquhoun HL, Squires JE, Kolehmainen N, Fraser C, Grimshaw JM. Methods for designing interventions to change healthcare professionals' behaviour: a systematic review. Implement Sci 2017 Mar 04;12(1):30 [FREE Full text] [CrossRef] [Medline]
    77. Dalgetty R, Miller CB, Dombrowski SU. Examining the theory-effectiveness hypothesis: a systematic review of systematic reviews. Br J Health Psychol 2019 May;24(2):334-356. [CrossRef] [Medline]
    78. Patey AM, Hurt CS, Grimshaw JM, Francis JJ. Changing behaviour 'more or less'-do theories of behaviour inform strategies for implementation and de-implementation? A critical interpretive synthesis. Implement Sci 2018 Oct 29;13(1):134 [FREE Full text] [CrossRef] [Medline]
    79. Park YS, Zaidi X, O'Brien BC. RIME foreword: what constitutes science in educational research? Applying rigor in our research approaches. Acad Med 2020 Nov;95(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 59th Annual Research in Medical Education Presentations):Si-Sv. [CrossRef] [Medline]
    80. Chen W, Berry A, Drowos J, Lama A, Kleinheksel AJ. Improving the evaluation of faculty development programs. Acad Med 2021 Oct 01;96(10):1496. [CrossRef] [Medline]
    81. Hilty DM, Liu HY, Stubbe D, Teshima J. Defining professional development in medicine, psychiatry, and allied fields. Psychiatr Clin North Am 2019 Sep;42(3):337-356. [CrossRef] [Medline]
    82. François P, Philibert AC, Esturillo G, Sellier E. [Peer groups: a model for the continuous professional development in general practice]. Presse Med 2013 Jan;42(1):e21-e27. [CrossRef] [Medline]

    ‎
    CPD: continuing professional development
    MeSH: Medical Subject Headings
    MMAT: Mixed Methods Appraisal Tool
    PICOS: Population, Intervention, Comparator, Outcomes, Study design
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    PROSPERO: International Prospective Register of Systematic Reviews

    Edited by G Eysenbach, T Leung; submitted 31.01.22; peer-reviewed by M Yordy, L McNamara, S Pesälä; comments to author 25.02.22; revised version received 11.03.22; accepted 21.03.22; published 02.05.22

    Copyright

    ©Gloria Ayivi-Vinz, Felly Bakwa Kanyinga, Lysa Bergeron, Simon Décary, Évèhouénou Lionel Adisso, Hervé Tchala Vignon Zomahoun, Sam J Daniel, Martin Tremblay, Karine V Plourde, Sabrina Guay-Bélanger, France Légaré. Originally published in JMIR Medical Education (https://mededu.jmir.org), 02.05.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on https://mededu.jmir.org/, as well as this copyright and license information must be included.