Development and Evaluation of the Online Addiction Medicine Certificate: Free Novel Program in a Canadian Setting

Background : Despite the enormous burden of disease attributable to drug and alcohol addiction, there remain major challenges in implementing evidence-based addiction care and treatment modalities. This is partly due to a persistent lack of accessible, specialized training in addiction medicine. In response, a new online certificate in addiction medicine has been established in Vancouver, Canada, free of charge to participants globally. Objectives : To evaluate and examine changes in knowledge acquisition among health care professionals before and after completion of an online certificate in addiction medicine. Methods: Learners enrolled in a 17-module certificate program and completed pre- and post-knowledge tests using online multiple-choice questionnaires. Knowledge acquisition was then evaluated using a repeated measures t-test of mean test scores before and after the online course. Following certificate completion, a subset of learners completed online course evaluation form. Results: Of the total 6985 participants who registered for the online course between May 15, 2017 and February 22, 2018, 3466 (49.6%) completed the online pre-test questionnaire. A total of 1010 participants completed the full course, achieving the required 70% scores. Participants self-reported working in a broad range of health-related fields, including nursing (371), medicine (92), counselling or social work (69), community health (44), and pharmacy (34). The median graduation year was 2010 (n = 363, interquartile range 2002-2015). Knowledge of addiction medicine increased significantly post-certificate (mean difference 28.21; 95% Confidence Interval 27.32-29.10; p <0.001). Physicians scored significantly higher on the pre-test than any other health discipline ( p <0.01), while the greatest improvement in scores was seen in the counselling professions ( p <0.05) and community outreach ( p <0.01). Conclusions : This free, online, open-access certificate in addiction medicine was found to improve knowledge of learners from a variety of disciplines and backgrounds. Scaling up “low threshold” learning opportunities may further advance addiction medicine training, thereby helping to narrow the evidence-to-practice gap. settings included hospitals, mental health facilities, pharmacies, group homes and more.


INTRODUCTION
Approximately 29 million people are affected by substance use disorders (SUDs) annually [1], with an estimated 21 billion dollars lost in associated productivity costs in the U.S. alone [2]. In recent years, a sharp rise in the number of opioid-related overdose deaths has led to declaration of public health emergencies around North America, and the number of individuals affected by alcohol use disorders continues to increase [3][4][5]. Still, uptake of evidence-based SUD treatment is low, and people with SUDs often receive inadequate care [6]. This is despite recent progress in addiction science, which has highlighted the important role of skilled health care providers and the efficacy of established psychosocial and pharmacological therapies in improving treatment outcomes [1,[6][7][8]. Moreover, most health care providers frequently report feeling unprepared to effectively identify and treat SUDs, and the stigma towards persons with SUDs persists among the health professions [9][10][11].
To address these gaps in practice, health professionals need better education and training in addiction medicine. However, such specialized training programs in addiction medicine can be inaccessible due to a limited number of available places, inability to compensate participants for their time, and the location of training (often big, urban centers) [12][13][14][15]. In addition, many of these programs accept only a particular subset of health care professionals -for instance, only physicians or social workers-and have limited resources for expansion. At the same time, effectively training a variety of health care providers in evidence-based treatment of SUDs is critical for providing quality care, as well as curbing the current opioid epidemic. One solution to overcome these challenges is through the expansion of online training programs in addiction medicine, which can reach a large number of participants, train professionals from a wide range of health disciplines, can be accessed from a wide variety of settings, and can be delivered at a relatively low cost [16,17]. Although the literature suggests that online continuing professional development (CPD) courses in health care can be equally effective as traditional classroom-based courses [17][18][19], this has not been demonstrated for addiction medicine. Therefore, we sought to evaluate changes in knowledge acquisition before and after completion of a new, comprehensive and accredited online certificate in addiction medicine, accessible and free of charge to learners worldwide.

Course description
The Online Addiction Medicine Certificate is a free online course and certificate program, is open for anyone wishing to improve their knowledge of substance use though is targeted at health professionals who are interested in learning more about care for patients with SUDs. The course involves 17 modules related to the identification, management, and treatment of SUDs. Box 1 lists the course learning objectives and topics covered. Each module consists of a 20-to 75minute video lecture (including a slideshow with spoken voiceover) by an expert in the relevant topic. Each module (with the exception of modules 1 and 17) is then followed by a brief multiplechoice test of the module material. All multiple-choice questions were Single Best Answer with four answer options. Those looking to obtain a formal certificate and/or a Continued Medical Education (CME) certificate must complete all of the modules, achieving a minimum passing score of 70% on all post-module tests. Overall, the online course takes approximately 16 hours to complete; however, it is self-paced, so learners can complete each module in their own time.
Modules do not have to be completed in chronological order, allowing learners to select the topics that most interest or benefit them. Once a learner completes the course, they must pass through an online evaluation form (with one mandatory question on potential bias in course content) in order to receive a digital certificate.
Prior to release, this course was accredited via review and approval by the University of British Columbia Division of Continuing Professional Development. This required a formal committee with representation of the target audience, a high degree of evidence and referencing throughout, and no industry bias or involvement. The course also met the certification criteria of the College of Family Physicians of Canada, as well as Maintenance of Certification Program of The Royal College of Physicians and Surgeons of Canada.

Course development and background
Based on informal and formal consultation with a range of stakeholders by the course's planning committee, it was clear that there was general lack of awareness of the range of evidencebased treatments for SUDs. Specifically, there was an unmet need for specific education and training for health care professionals on SUDs that was accessible and provided at no cost. In response to this need, the British Columbia Centre on Substance Use (BCCSU) developed this structured course in partnership with content experts throughout British Columbia. This included physicians in the areas of psychiatry, public health, internal medicine and family medicine. While the target audience of the program is primarily prescribers, modules were created to be accessible to all allied health disciplines, including nurses, social workers, psychologists, and were also accessible for a general audience.
The course design utilizes a case-based learning structure, with theoretical and academic context nested in the introductions to the chapter. The topics, content, and speakers were selected through recommendations and advice from the planning committee. Each chapter is taught by a faculty member that has extensive experience with the particular topic and would be considered an expert in the province. Physician members of the planning committee were instrumental in ensuring that all materials were evidence-based and relevant to physicians practicing in primary care environments. Prior to beginning their presentations, the course's lead author (EW) and the planning committee required faculty members to submit detailed chapter outlines. These were thoroughly reviewed and feedback was returned to presenters for incorporation into their lecture and presentation slides. The finalized presentations were then reviewed to ensure validity and objectivity of content. The lectures were recorded between August 2015 -May 2016. The course was hosted on the host center's website using a Wordpress content management system.
The course leads and Advisory Committee guide ongoing, year-round recruitment activities. Participant recruitment strategies include disseminating advertisements through email mailing lists, posters, brochures, descriptions and links in our website, lay media advertisements, conference participation, newsletters, social media, and word of mouth. Additionally, the course was promoted through in-person seminars focused on substance use across Canada.

Procedures
In order to take the online course, participants first had to register using the online registration form. Here, learners were given space to fill in their full name and email address, and were asked to select their home province from a drop-down menu (all Canadian provinces and 'Other' were listed). Providing province information became mandatory after the first month of the course. In the registration form, learners were also asked to select their professional discipline from a given list. Following registration, learners completed a multiple-choice knowledge test (the "pre-test") to evaluate baseline knowledge of course content. As well, following each module, learners completed multiple-choice knowledge tests (the "post-tests") to evaluate understanding of the material just taught.
Because two of the modules (1 and 17, the introduction and conclusion) lacked post-tests, in order to pass the online course, learners had to complete a total of 15 post-tests. When all posttests were completed with a minimum 70% score, learners were then given the option to complete an online evaluation form, gauging their satisfaction with the program and the applicability of course material to their clinical practice. The evaluation form also asked for further demographic information, such as professional discipline, the year that learners completed their professional degrees, and the health care settings in which they provide services.
We measured the effectiveness of the course using: (1)  We also examined differences in test scores between participants in different health-related fields (e.g., nursing, pharmacy). Health professionals with significantly greater pre-test scores were taken to have greater baseline knowledge than other health professionals taking the course.
As well, health professionals with a significantly greater difference in scores were taken to have had greater knowledge benefits than other health professionals taking the course. Linear regression and paired t-tests were used to test statistical significance of these differences with SAS 9.4 (SAS, Cary, NC). All p-values were two-sided.  Table 1).

Registration
There was an overall difference in the province of origin between those who did and did not complete the course (p=0.013), with a higher proportion of Ontarians and Prairie residents among completers versus non-completers (Table 1). had not yet completed the course at the time of data collection, and participants who completed the course but did not pass. b SK=Saskatchewan, MB=Manitoba, AB=Alberta c NB=New Brunswick, NL=Newfoundland, NS=Nova Scotia, PEI=Price Edward Island d This includes participants from international settings as well as those with missing location data. e Only participants who passed the course filled in this information in the satisfaction form; totals do not add up to 475 since participants could select more than one service setting f Other settings included hospitals, mental health facilities, pharmacies, group homes and more.

Knowledge of addiction medicine
Attempting (regardless of pass/fail status) all 15 modules with post-tests was a study inclusion criterion. However, since all learners who attempted every module also passed the course, a total of 1,010 "attempters" were included in analyses.
Post-certificate, knowledge of addiction medicine increased significantly (mean difference = 28.2; 95% Confidence Interval 27.3-29.1; p<0.001) ( Table 2). Physicians scored significantly higher on the pre-test than any other health discipline (p<0.01), while the greatest improvement in scores was seen in the counselling (p<0.05) and community outreach (p < 0.01) professions (Table   3).   either agreed or strongly agreed that the course successfully met their learning needs. In addition, the majority of participants rated the course's relevance to their practice (83%) and incorporation of evidence-based research (93%) as "above average" or "excellent" (Figure 2).

DISCUSSION
We sought to evaluate changes in knowledge acquisition before and after completion of a comprehensive online certificate in addiction medicine. Following the course, knowledge of addiction medicine increased significantly, with a completion rate of 14.5% (percentage of participants who attempted all 15 post-tests out of the total number of participants registered), success rate of 100% (percentage of participants who successfully passed all 15 post-tests out of the total number of participants who attempted all 15 post-tests), and commitment rate of 29.1% (percentage of participants who passed all 15 post-tests out of the total number of participants who attempted at least one post-test). Physicians scored significantly higher on the pre-test than any other health discipline, while the greatest improvement in scores was seen in the counselling professions and community outreach. The large majority of participants reported that the course was effective in meeting their learning needs, was relevant to their practice, and well-incorporated evidence-based research.
This course is a novel online training program in addiction medicine -a field in urgent need of expanded educational opportunities for health care providers [9,10,20]. As of February 2018, the course had nearly 7000 registrants, confirming the strength of low threshold, online models for facilitating rapid scale-up of evidence-informed training in addiction medicine, and a high demand [21][22][23]. This demand was particularly notable among nurses who composed the largest proportion of health providers in our sample and saw the sharpest increase in registrations.
As reported in previous articles [24], completion rates of open online courses are often lower than those of traditional in-person training, and may be more a product of participants' individual preferences and/or needs, rather than the course material or structure. As such, the observed completion rate of this course is in line with previous studies of open online courses, which report completion rates from 0.9 -36.1% (median 6.5%) [24,25]. It is also important to note that the online certificate in addiction medicine was intentionally structured for participants to select the modules most relevant to them.
Similar to previous studies of online courses on SUDs, we found a significant increase in knowledge of addiction medicine post-certificate [22,23,26,27]. In addition, this study highlighted which health professions may derive the most benefit from such a course. Aligned with prior literature, which has shown that education in SUD care is often lacking in social work and counselling curricula [22,28], participants in counselling/social work and community outreach demonstrated the greatest improvement in scores. For example, in one study of university-level counselling and social work programs, it was found that just 69% of masters-level counselling programs, 3% of bachelors-level social work programs, and 2% of masters-level social work programs required a course in SUD care [29]. Although it is clear that professionals in counseling/social work and community outreach should be targeted for similar training in the future, the online format can present unique barriers -including time-and schedule-constraintsas noted by previous studies of online training in SUDs [30].
Several limitations may reduce generalizability of our findings. First, due to inconsistencies in the data, for some participants, it was impossible to link their course registration data and test scores. Second, the large number of participants from western Canada may have introduced bias into the results, as health care professional' knowledge and training in addiction medicine may vary by setting. Promoting the course to a more international audience could improve training and highlight the needs of healthcare professionals in a wider range of contexts. Third, the selfselection of registrants for the course may mean that study participants were more likely to have a higher level of interest and/or experience in addiction medicine--it is likely that practitioners who seek specialized training are more prone to positive attitudes towards, and learning experiences with people who have SUDs [31]. Fourth, as participants were able to attempt each post-module test as many times as they liked, and a minimum score of 70% was required to pass each test, our overall post-test value may have been positively skewed. This limitation was carefully considered prior to analyses, balancing perspectives that a participant's final passing attempt at each post-test (1) represented new retained knowledge, and (2) was the product of selecting the correct answer by chance. Therefore, we averaged the scores from all attempts by a single participant at each posttest. This mean score was then used to calculate a total group mean score for the course. Finally, we did not capture a corresponding change in provider behaviour following the course. Future research examining the impact of the current course in addiction care settings would be valuable.
In the present study, over 6000 participants embarked on and over 1000 participants completed online training in addiction medicine. Overall, our analyses suggest that the course can feasibly increase knowledge in addiction medicine to a wide range of health care providers. Scaling up "low threshold" learning opportunities may further advance addiction medicine training, thereby helping to narrow the evidence-to-practice gap.
17 Cases consolidating knowledge (35 mins) Note. Lecture times do not include the time to complete pre-and post-tests. a Each topic corresponds to a single module in the online course; post-tests were only provided for chapters 2-16.

Box 2. Sample Question and Format
Question: Evidence-based first line anti-craving and relapse prevention therapies for the treatment of alcohol use disorder include: a) Naltrexone 50 mg once daily b) Acamprosate 666 mg once daily c) Gabapentin 300 mg once daily d) Celexa 40 mg once daily Correct answer: A Explanation: Naltrexone is typically provided 50 mg once daily and has a number needed to treat to prevent a return to any drinking of 20. Acamprosate has a number needed to treat to prevent a return to any drinking of 12 and is an alternative first line agent but it is dosed 666 mg three times per day rather than once daily. Though less studied, Gabapentin appears to be an effective anti-craving agent but the optimal studied dose was 600 mg three times per day. Celexa is not a pharmacotherapy for alcohol use disorder.