We performed a cross-sectional study of a convenience sample of specialist physicians who had completed 1 of 5 different web-based CPD courses [30]. We report data according to the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) reporting guidelines for cross-sectional studies [31]. Data were collected between November 2015 and April 2019 following completion of the CPD courses by participants using a web-based interactive platform (MÉDUSE) designed by the Federation of Medical Specialists of Québec (FMSQ) [32]. The FMSQ consists of 35 medical associations and represents 59 medical specialties in the province of Quebec. Its members include more than 10,000 medical specialists [33].

Approval for this study was obtained from the research ethics boards of the Centre intégré universitaire de santé et de services sociaux (CIUSSS) de la Capitale-Nationale (Project 2020-1889_SPPL).

To be eligible, the physicians had to have completed (1) one of the 5 available FMSQ CPD web-based courses and (2) the CPD-Reaction questionnaire.

The 5 web-based courses were all accredited by the Royal College of Physicians and Surgeons of Canada [34] and targeted the following five behaviors: (1) to adapt the frequency of cytological exams for gynecological patients 25-45 years old to new human papillomavirus recommendations; (2) to use recommended lung cancer treatment and monitoring algorithms; (3) to use a systematic leadership approach in community health endeavors (eg, preventing instances of suicide from a bridge in Montreal); (4) to respect best practices in record keeping; and (5) to identify patients who meet the criteria for identifying a potential organ donor. Courses were free of commercial support (paid by FMSQ members’ annual contributions). They were secured and accessible 24/7 by Quebec specialist physicians. The courses lasted from 90 to 120 minutes, and participants could stop or reinitiate courses at any time. Course objectives were based on Godin’s integrated model for health professionals’ behavior and aimed to encourage physicians to adopt new behaviors (or cease old ones). Each targeted behavior was designed according to 3 of the TACT principles: “target,” “action,” and “context” (“time” was excluded, as the targeted behaviors were not dependent on a specific time frame) [35]. The courses were also designed to develop core competencies as described in the CanMEDS Competency Framework [36]. CanMEDS is a framework created by the Royal College to ensure that CPD courses, regardless of their specialist content, allow physicians to develop one or more of the following core roles: medical expert, communicator, collaborator, leader, health advocate, scholar, or professional. We inserted the relevant learning objectives into each of the 5 CPD-Reaction questionnaires and attached them to the end of each respective course (Figure S1 in Multimedia Appendix 1).

Data were collected in 2 separate databases by the FMSQ, one for the sociodemographic variables of those attending each course (henceforth referred to as “participants”) and another for those who had completed CPD-Reaction (henceforth referred to as “respondents”). Individual participant sociodemographic data could not be linked to individual respondent CPD-Reaction questionnaire scores and were analyzed at the level of the CPD course. For sociodemographic variables and variables collected at the course level (content and format of the CPD courses), the same values were then attributed to all respondents in the same course.

The data set had a hierarchical structure consisting of 2 levels, which were respondents and CPD courses. Individual participant sociodemographic data were in a distinct database and could not be linked to individual respondent CPD-Reaction questionnaire scores. Therefore, analysis focused primarily on variables at the level of CPD courses. These variables were only retained if they could be collected for all 5 courses. All variables retained at the course level had fewer than 1% missing values. At the respondent level (n=259), only the dependent variable, intention, and the 4 independent psychosocial variables (social influences, beliefs about capabilities, moral norm, and beliefs about consequences) were accessible and analyzed. At the CPD course level (n=5 CPD courses), data analyzed included the 5 sociodemographic variables of participants, 8 course format variables, and 9 course content variables.

We used descriptive statistics and frequency counts to describe all variables. We performed an ANOVA single-factor analysis to assess whether the topic of the courses had an impact on intention. We also computed the intraclass coefficient (ICC) to assess the percentage of variance in behavioral intention and its psychosocial determinants attributable to the CPD course [40]. We performed exploratory bivariate analysis using Spearman correlations for each one of our independent variables at the level of the CPD courses to assess their association with physicians’ intention scores. Lastly, we performed bivariate and multivariate analysis on the 4 psychosocial determinants of intention at the respondent level (n=259) to explore their impact on intention scores. We used a linear model and introduced a random effect for the CPD courses. A threshold of .05 was set for statistical significance. We verified all assumptions for the linear regression model [41]. All analyses were performed with SAS, version 9.4 (SAS Institute Inc).

Table 1 shows the characteristics of participants from across Quebec attending each course.

Table 2 shows details of course characteristics, including course formatting, content, presence of Godin’s constructs, and Bloom’s learning levels. Three courses lasted 90 minutes and 2 lasted 120 minutes. Two out of 5 courses focused on the CanMEDS role of medical expert, 1 on the role of leader, 1 on the role of professional, and 1 on several of the roles at once (classified as “not applicable”). Moreover, 3 course characteristics showed no variability—all contained slides with a quiz (format variable), slides on beliefs about capabilities, and slides on role and identity (content variables; Table 2).

Of the 400 participants, 259 (65%) respondents fully completed the CPD-Reaction questionnaire. Table 3 shows details of respondents’ mean intention and psychosocial determinants and SD scores for each course. The behavioral intention score was medium to high and varied depending on the course undertaken (intention score between 5.37, SD 1.17 and 6.60, SD 0.88). ANOVA analysis of the variable intention showed significant differences between courses (F value=12.50, P<.001). The ICC (0.17) indicated that 17% of the total variation in the behavioral intention of physicians to adopt new behaviors could be explained at the level of the course in which they had registered. Some courses showed significantly different means of intention, that is, respondents shared more intracourse similarities than extracourse similarities (ie, within CPD courses vs between courses).

Only one of the course variables, psychomotor learning level (Bloom’s taxonomy), was significantly associated with the physicians’ intention to change their behavior, and this was the case in all 5 courses (R=0.89, P=.04) (data not shown). Bivariate regression analysis of psychosocial determinants showed that all 4 variables were significantly associated with intention (P<.001) (Table 4). Multivariate regression analysis of the same variables showed 3 out of the 4 were significantly correlated with intention, namely beliefs about capabilities (0.49, P<.001), moral norm (0.37, P<.001), and beliefs about consequences (0.40, P<.001) (Table 4). When we analyzed the courses separately, we found similar results (Multimedia Appendix 2).

We identified factors associated with physicians’ intention to adopt new behaviors following the completion of 5 different web-based CPD courses. Behavioral intention scores were average to high but differed significantly from one course to another. The differences between CPD courses (higher level in our hierarchical database) explained a significant proportion of this variance in intention (ICC=0.17). We saw no influence of course characteristics (content-wise or format-wise) on intention except the targeting of Bloom’s psychomotor learning level. Finally, we observed that together, beliefs about capabilities, moral norm, and belief about consequences (3 of the psychosocial variables included in Godin’s integrated model for health professional behavior change) partially explained physicians’ behavioral intentions.

First, we found that behavioral intention scores were average to high but varied by course. Some courses seemed to be associated with higher physician intention to adapt their practice compared with others. Course 3, on using a systematic leadership approach in community health endeavors, had the lowest intention score of all—although this course also targets more complex and ambiguous outcomes than the others, and leadership skills are difficult to develop in 120 minutes. We also found that the variance in intention explained by the difference in CPD courses had significant magnitude. We obtained an ICC of 0.17; thus, the intergroup variance represented 17% of the total variance. Interestingly, higher ICCs are more often seen in studies in specialty settings than in primary care studies [42]. It is possible that specialist physicians have more in common with each other, even diverse specialist physicians attending the same course, than do general practitioners. Some studies have observed that medical professional culture ensures there is more similarity than diversity within specific medical specialties [43-45].

Second, in bivariate analyses at the CPD course level, the only variable significantly correlated with the intention to adopt new behaviors was targeting Bloom’s psychomotor learning level. This level of learning, unlike the cognitive or affective levels, is more closely related to physical changes in behavior. However, this variable was not retained in our final model, suggesting that as an influence on adoption of new behaviors, it does not supersede the psychosocial variables included in the integrated model [17]. Regarding the other nonsignificant variables, previous studies have also found little significant association between sociodemographic characteristics and intention to adopt new behaviors [46,47]. Our results validate the assumption of the integrated model for health professional behavior change: modifiable psychosocial factors are the variables most likely to explain behavior change, and CPD courses should therefore focus on these factors to be more effective.

Third, we found that the 3 variables most significantly associated with intention to adopt a behavior among respondents were all psychosocial factors included in our integrated theoretical framework—beliefs about capabilities, moral norm, and beliefs about consequences (ie, their confidence about adopting the behavior, its ethical acceptability, and their perception that the behavior would be useful and beneficial). Based on our results, CPD courses should use behavior change techniques that focus on these 3 variables [48,49]. To improve beliefs about capabilities, courses could provide more experience to give participants confidence in their abilities, such as identifying barriers and management strategies, providing feedback, and encouraging monitoring of future actions (eg, noting and recording when the new behaviors have been adopted) [17,50]. To improve beliefs about consequences, courses could provide information about the proven benefits of the behavior and personalized information about its consequences. Regarding moral norm, courses could emphasize the felt obligation to adopt behaviors or help participants focus on moral considerations such as being aware of others’ needs [51]. In addition, according to Godin’s theory, when people hold two ideas that are not psychologically consistent, to reduce cognitive dissonance, they do all in their power to change them until they become consistent [17]. One way to reduce cognitive dissonance is to solicit arguments from the subject in favor of the behavior to be adopted even if they are against it [52]. While producing such arguments may cause discomfort, the subject will ultimately adjust their initial attitude to be more consistent with the arguments they fabricated in favor of the behavior. Surprising as it may seem, when we are led to act contrary to our convictions, we tend to justify our actions, and we adapt our opinions to our behavior. Other work on “provisional selves” suggests that playing a role with which one is unfamiliar, or even against which one resists, opens new moral possibilities and can help one envisage adapting one’s current role or adopting new ones [53,54]. Including this technique in a CPD course would be an interesting challenge.

This was a cross-sectional study, which limited our interpretation to assuming that attending the CPD courses improved intention scores. Indeed, we are unaware if respondents already had moderate-to-high intention to adopt these behaviors before completing the CPD courses—using the CPD-Reaction questionnaire both before and after the course would have better indicated a change in intention due to the course topic. A future study with a more robust study design (eg, pre-post controlled trial) could further verify the impact of courses [55]. Moreover, to increase power, we brought data from all 5 CPD courses (each targeting a different clinical behavior) into one hierarchical data set. Although aggregating data on distinct behaviors is not always advisable [56,57], this limitation was mitigated by respect for the theory archetypes that structure the study. In addition, our sensitivity analysis (Multimedia Appendix 2) showed similar results to those obtained with the aggregated database. The literature suggests that at least 30 units at each level of analysis are needed to reach sufficient power [58]. New ways to assess CPD courses are needed as few individual CPD courses recruit hundreds of participants. Moreover, sociodemographic data collected at the group level could not be applied to the respondent level (ie, to the individual level). Inferring results of analysis at the upper level (where determinants and outcomes are related at the group [course] level) to the individual level (ecological fallacy) or the reverse (atomistic fallacy) can result in bias [59]. Finally, intention is recognized as a limited proxy for behavior. Meta-analytic syntheses have found that intention accounts, on average, for only about 25% of the variance in behavior [35,60], although finding other reliable measures of behavior is challenging [19,56]. While a 2006 review by Eccles et al [61] “provide[d] encouragement for the contention that there is a predictable relationship between the intentions of a health professional and their subsequent behaviour,” CPD activities making use of the determinants of intention as dependent variables should also integrate methods to close the intention-behavior gap such as audit and feedback, eliciting of implementation intentions (“if-then” plans), commitment to change statements, and supervision to support clinicians in following through on their intentions [62-64].

Beliefs about capabilities, moral norm, and belief about consequences partially explained physicians’ behavioral intention. To address these beliefs, CPD activities could focus on building physicians’ confidence about overcoming obstacles and on strategies for helping them align moral values with new behaviors, as well as providing information about their proven benefits.

As mentioned in our previous work [12], the use of CPD-Reaction helps CPD developers reflect on the nature of their training objectives in relation to the impact they seek. This study provides insights as to how to optimize physicians’ intention to adopt a new behavior as a result of web-based CPD activities. CPD-Reaction contains the relevant theory-informed and validated items needed to assess intention and its determinants for CPD developers targeting clinical behavior change.