This paper is in the following e-collection/theme issue:

Health Professionals' Training in eHealth, Digital Medicine, Medical Informatics (104) e-Learning and Medical Education (691) Telehealth and Telemonitoring (1671) Innovations in Exercise Rehabilitation (213) Internet for Rehabilitation (75) Knowledge Translation and Implementation Science (398) Clinical Communication, Electronic Consultation and Telehealth (590) Telerehabilitation (159)

Published on in Vol 7, No 4 (2021): Oct-Dec

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/30378, first published .
An e-Learning Program for Physiotherapists to Manage Knee Osteoarthritis Via Telehealth During the COVID-19 Pandemic: Real-World Evaluation Study Using Registration and Survey Data

An e-Learning Program for Physiotherapists to Manage Knee Osteoarthritis Via Telehealth During the COVID-19 Pandemic: Real-World Evaluation Study Using Registration and Survey Data

An e-Learning Program for Physiotherapists to Manage Knee Osteoarthritis Via Telehealth During the COVID-19 Pandemic: Real-World Evaluation Study Using Registration and Survey Data

Authors of this article:

Ana Elisa Serafim Jorge1 Author Orcid Image ;   Kim Louise Bennell2 Author Orcid Image ;   Alexander Jared Kimp2 Author Orcid Image ;   Penny Kate Campbell2 Author Orcid Image ;   Rana Shane Hinman2 Author Orcid Image
Article Authors Cited by (9) Tweetations (22) Metrics

    Original Paper

    1Center for Biological and Health Sciences, Department of Physiotherapy, Federal University of São Carlos, São Paulo, Brazil

    2Centre for Health, Exercise and Sports Medicine, Department of Physiotherapy, School of Health Sciences, The University of Melbourne, Melbourne, Australia

    Corresponding Author:

    Rana Shane Hinman, PhD

    Centre for Health, Exercise and Sports Medicine

    Department of Physiotherapy, School of Health Sciences

    The University of Melbourne

    Level 7, Alan Gilbert Building

    The University of Melbourne

    Melbourne, 3010

    Australia

    Phone: 61 3 8344 3223

    Email: ranash@unimelb.edu.au


    Background: The COVID-19 pandemic necessitated clinicians to transition to telehealth, often with little preparation or training. The Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis (PEAK) e-learning modules were developed to upskill physiotherapists in management of knee osteoarthritis (OA) via telehealth and in-person. In the research setting, the e-learning modules are perceived by physiotherapists as effective when they are part of a comprehensive training program for a clinical trial. However, the effectiveness of the modules on their own in a real-world setting is unknown.

    Objective: This study aims to evaluate the reach, effectiveness, adoption, and implementation of PEAK e-learning modules.

    Methods: This longitudinal study was informed by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. Participants were clinicians, researchers, educators, and health care students who registered for access to the modules between April 1 and November 30, 2020. Reach was evaluated by outcomes (countries, referral sources, and attrition) extracted from registration data and embedded within precourse surveys in the Learning Management System (LMS). Effectiveness was evaluated by outcomes (confidence with videoconferencing; likelihood of using education, strengthening exercise, and physical activity in a treatment plan for knee OA; usefulness of modules) measured using a 10-point numeric rating scale (NRS; score range from 1=not confident or likely or useful at all to 10=extremely confident or likely or useful) in pre- and postcourse (on completion) surveys in the LMS. Adoption and implementation were evaluated by demographic and professional characteristics and outcomes related to the use of learning and usefulness of program elements (measured via a 4-point Likert scale, from not at all useful to extremely useful) in a survey administered 4 months after module completion.

    Results: Broad reach was achieved, with 6720 people from 97 countries registering for access. Among registrants, there were high levels of attrition, with 36.65% (2463/6720) commencing the program and precourse survey and 19.61% (1318/6720) completing all modules and the postcourse survey. The program was effective. Learners who completed the modules demonstrated increased confidence with videoconferencing (mean change 3.1, 95% CI 3.0-3.3 NRS units) and increased likelihood of using education, strengthening and physical activity in a knee OA treatment plan, compared to precourse. Adoption and implementation of learning (n=149 respondents) occurred at 4 months. More than half of the respondents used their learning to structure in-person consultations with patients (80/142, 56.3%) and patient information booklets in their clinical practice (75/142, 52.8%).

    Conclusions: Findings provide evidence of the reach and effectiveness of an asynchronous self-directed e-learning program in a real-world setting among physiotherapists. The e-learning modules offer clinicians an accessible educational course to learn about best-practice knee OA management, including telehealth delivery via videoconferencing. Attrition across the e-learning program highlights the challenges of keeping learners engaged in self-directed web-based learning.

    JMIR Med Educ 2021;7(4):e30378

    doi:10.2196/30378

    Keywords

    osteoarthritis; knee; physiotherapy; exercise; e-learning; telehealth; pain; education; implementation; evaluation; professional development; rehabilitation 


    Background

    The prevalence of knee osteoarthritis (OA) is rapidly increasing as the population ages and the obesity epidemic rises worldwide [1]. As a debilitating chronic musculoskeletal condition, OA is ranked as the 12th leading cause of disability worldwide [2]. Education and exercise are highly recommended as core interventions [3-6], and these are often delivered via in-person consultations by physiotherapists [7,8]. In 2020, the COVID-19 pandemic imposed a substantial change in the delivery of in-person health care services worldwide [9,10]. Social distancing requirements and lockdowns necessitated many physiotherapists to rapidly transition to telehealth service delivery using freely available real-time videoconferencing software (eg, Zoom, Zoom Video Communications; Microsoft Teams, Microsoft; WhatsApp, Facebook Inc; and FaceTime, Apple Inc) [11,12], often with little preparation or training. A global survey of allied health practitioners between April and June 2020 showed that 68% of respondents used telehealth to manage people with OA during the pandemic and that exercise and education were used by 96% of respondents as part of telehealth consultations [11].

    Telehealth is a term under the digital practice umbrella, which encompasses health care services, support, and information provided remotely via telecommunication technology [13]. Telehealth has been shown to be an effective model of physiotherapy service delivery for a range of musculoskeletal conditions [14], including OA [15]. A randomized controlled trial has shown the effectiveness [16] and patient acceptability [17] of the remote delivery of a physiotherapist-prescribed exercise program via video consultations for people with knee OA; however, before the COVID-19 pandemic, a minority of allied health clinicians provided telehealth services [11]. Before the pandemic, barriers to telehealth were multifactorial and included costs associated with implementing telehealth infrastructure, lack of third-party payer funding for telehealth consultations, and clinician resistance to practice change [15,18]. Significantly, many physiotherapists lack specific training in delivery of care via telehealth [13,19], and subsequently may have little knowledge [18], experience [20], and confidence [21] in telehealth delivery. Research shows that clinician education about and exposure to telehealth increases both acceptance of, and confidence with, delivery of care via telehealth [20,21]. Furthermore, physiotherapists and other health care professionals have highlighted limitations in their knowledge, skills, and confidence in managing people with knee OA [22]. Thus, providing education and knowledge resources to clinicians in telehealth skills to deliver evidence-based OA management is paramount.

    e-learning is an accessible and scalable method for delivering health professional education and training. e-learning broadly relates to the delivery of educational material through information and communication technology, often using the internet to wholly or partially replace the need for a human instructor [23]. The Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis (PEAK) e-learning modules [24] were developed by researchers, in the context of a clinical trial [25], to upskill physiotherapists in evidence-based management of knee OA through telehealth and in-person consultations. The self-directed modules aim to educate learners on how to implement evidence-based physiotherapy care using a structured program of education, strengthening exercises, and individualized physical activity over 5 individual consultations, delivered via videoconferencing or in-person. In the research setting, the PEAK e-learning modules were perceived by physiotherapists as effective when delivered as part of a comprehensive training program for a clinical trial [20]. However, the effectiveness of the PEAK e-learning modules on their own in a real-world setting is unknown.

    Objectives

    In light of the unfolding COVID-19 pandemic, the PEAK e-learning modules were released globally, free of charge, on April 1, 2020, to assist physiotherapists and other clinicians in providing care to patients with knee OA through telehealth. There is currently limited research evaluating professional development initiatives (ie, beyond entry-to-practice training) for physiotherapists regarding the management of OA. Guided by the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework [26], this study aims to evaluate the reach, effectiveness, adoption, and implementation of the PEAK e-learning modules during the COVID-19 pandemic. Specifically, we aimed to (1) evaluate the reach of, and attrition across the PEAK e-learning program; (2) evaluate the effectiveness of the PEAK e-learning program in building confidence with telehealth and intention to use core recommended OA treatments among learners who completed the program; and (3) evaluate how learners who completed the PEAK e-learning program implemented what they learned into practice 4 months after completion, including the settings it was adopted into.


    Design

    A longitudinal study with pre- and postcourse evaluations was conducted via an electronic survey. The RE-AIM framework [26] has informed the study.

    Participants

    Participants in this study were those who registered for access to the PEAK e-learning modules between April 1, 2020 (launch date), and November 30, 2020. There were no specific eligibility criteria for registration (learners self-select whether to register based on the English language description of the e-learning modules on the registration page) and no costs were charged for the modules; thus, participants in this study included health care clinicians, researchers, educators, and health care students from anywhere in the world. After activating access to the Learning Management System (LMS), learners provided consent for researchers to use their program data for research purposes. Four months after the completion of the e-learning modules, learners were sent an electronic survey about how they had implemented their learnings into practice. Completion of this survey was voluntary, and implied consent to participate. This study was approved by the University of Melbourne Human Research Ethics Committee (#2056938).

    The PEAK e-Learning Modules

    The PEAK e-learning modules [24] teach clinicians how to deliver an evidence-based management program (PEAK program) for people with knee OA, either via telehealth (videoconferencing) or in-person. The PEAK program and the e-learning modules were devised by researchers at the University of Melbourne specifically for use in a National Health and Medical Research Council–funded clinical trial [25] comparing videoconferencing with face-to-face care by physiotherapists for people with knee OA (trial ongoing). Thus, the program was designed for delivery by physiotherapists in Australia but is relevant to physiotherapists and other health care clinicians globally. The program focuses on education, strengthening exercise, and physical activity, delivered during 5 consultations over 3 months and can be individualized to patient needs. The e-learning modules were made publicly available across the globe on April 1, 2020, and they were promoted by researchers via social media and directly to physiotherapy professional organizations worldwide.

    The asynchronous e-learning modules were delivered on the University of Melbourne LMS (Canvas LMS by Instructure, 2019), covering (1) evidence-based best-practice knee OA management, (2) telehealth (the delivery of care via Zoom videoconferencing), and (3) the PEAK program (a structured physiotherapy treatment protocol). Each module included a quiz at the end to help reinforce learning. The modules were sequentially released in the order listed, with access to subsequent modules unlocked as the preceding module was completed (defined as all pages viewed within the module and quiz questions submitted). Quizzes required >80% correct answers for advancement to the next module, and learners were allowed unlimited, multiple attempts at quiz questions. Learners were instructed to allow approximately 3 to 4 hours to complete all modules. The e-learning modules also provided learners with access to a website of videos of the exercises contained within the PEAK program. For learners who completed all e-learning modules, a suite of resources was unlocked on completion for printing and downloading. This included educational booklets that clinicians can provide to their patients (Preparing for your Consultations, Osteoarthritis Information, Exercise Booklet, and Knee Plan and Log Book) as well as clinician resources (Zoom Troubleshoot Guide, Initiating and using Zoom for video consultations, Accessing the website of exercise videos, Pre-consultation survey, Consultation Outline, and Readiness Checklist). On completion of the e-learning modules, learners could request a certificate of completion via email.

    Users registered for access via a web-based form (Qualtrics International) housed at the University website [24], where they provided their names and email addresses. It took up to 24 to 48 hours after registration for the research staff to create a log-in. External users (non–University staff or students) were required to activate their account before they could log into the LMS to commence learning.

    Data Collection

    Registration Data

    The number of registrants was captured using the Qualtrics registration form. The geographic location of registrants was estimated from the approximate longitude and latitude obtained from their deidentified IP addresses. Researchers used the location data to correlate this with known country and continent information.

    LMS Data

    The PEAK e-learning modules contained pre- and postcourse survey questions (Multimedia Appendix 1) embedded within the LMS. After activating their LMS account, learners were provided with an introduction to the e-learning modules and completed a mandatory precourse survey before the first module was accessible. Learners who completed all the modules were invited to complete a postcourse survey. As an incentive, the printable and downloadable resources and instructions for obtaining a certificate of completion were made available to those who completed the postcourse survey. No fee was charged to obtain a certificate of completion.

    The precourse survey comprised brief descriptive questions regarding learners’ professional characteristics and their usual clinical practice. Precourse levels of confidence with videoconferencing and likelihood to use education and strengthening exercise and physical activity in a treatment plan for patients with knee OA were ascertained by a series of questions each rated via a 10-point numeric rating scale (NRS), ranging from 1=not at all confident or likely to 10=extremely confident or likely. The postcourse survey immediately reassessed these questions, with additional questions about how long it took to complete the modules (via dropdown lists for hours and minutes) and how useful was the course (NRS 1=not useful at all to 10=extremely useful).

    Adoption and Implementation Data

    Learners who completed the postcourse survey within the LMS were invited to complete another survey 4 months later to ascertain the adoption and implementation of learnings into practice. This electronic survey (Multimedia Appendix 2) link was sent by email and accessed via a secure web-based survey tool (REDCap, Research Electronic Data Capture, Vanderbilt University). The survey presented slightly different questions depending on whether the learner was a health professional, student, an educator, or a researcher. The first section ascertained demographic information and professional characteristics (data regarding adoption). The second section ascertained what the learner had implemented into practice from the PEAK e-learning modules, including how they used the downloadable resources. Questions evaluated the usefulness of the PEAK e-learning modules and their resources, as well as the extent to which learnings from the modules had changed learners’ practice and how. The barriers to implementation were also evaluated. Learners rated their agreement with relevant statements using a 4-point Likert scale ranging from not at all to extremely useful or to a large extent.

    Data Analysis

    Data were downloaded from the LMS and REDCap software and processed in Excel (Microsoft Corporation). Descriptive analysis of the data was performed using means, SD, and proportions with Excel, where appropriate. For the second aim, only data from individuals who had answered both the precourse and postcourse surveys were analyzed for paired outcomes. For paired outcomes, individual change scores (for each learner and each outcome) were calculated by subtracting precourse scores from postcourse scores. The mean change (95% CI) was then calculated for each outcome.


    Aim 1: Reach of, and Attrition Across, the PEAK e-Learning Modules

    From launch (April 1) to November 30, 2020, 6720 people registered for access to the PEAK e-learning modules. There was a broad international reach. Registrants came from 97 countries, with the top 10 most common countries being Australia (2077/6720, 30.91%), Canada (870/6720, 12.95%), United Kingdom (636/6720, 9.46%), United States (632/6720, 9.40%), South Africa (253/6720, 3.76%), Ireland (251/6720, 3.74%), Romania (152/6720, 2.26%), India (148/6720, 2.20%), Brazil (121/6720, 1.80%), and New Zealand (119/6720, 1.77%). The 5 most common nationalities have English as their native language.

    There was attrition of learners across the pipeline from registration to the 4-month survey completion (Figure 1). The greatest attrition occurred between requesting registration and activating the LMS account, with 39.48% (2653/6720) of registrants failing to activate their LMS account. Of the 4067 users who activated their account within the LMS, 60.56% (2463/4067) completed the precourse survey, 44.18% (1797/4067) completed the first module, and 32.41% (1318/4067) completed all modules as well as the postcourse survey. Of the 2463 learners who completed the precourse survey, 53.51% (1318/2463) completed all modules and the postcourse survey, but only 6.05% (149/2463) completed the 4-month survey.

    Of the participants who activated their LMS account and completed the precourse survey, most found out about the PEAK e-learning modules through a work colleague (795/2463, 32.28%). Other referral sources included Twitter (450/2463, 18.27%), professional organizations (368/2463, 14.94%), Facebook (281/2463, 11.41%), the internet (132/2463, 5.36%), a physiotherapy course instructor (128/2463, 5.20%), a friend/family (114/2463, 4.63%), other health professional course instructor (31/2463, 1.26%), LinkedIn (23/2463, 0.93%), and other sources (134/2463, 5.44%).

    ‎
    Figure 1. Users since program launch. Pipeline depicting the number of people who registered for access, activated their Learning Management System (LMS) account, and completed the various modules of the e-learning program. Percentages are calculated as a proportion of those who requested registration. OA: osteoarthritis.
    View this figure

    The characteristics of learners who completed the e-learning program (ie, those who completed both the pre- and postcourse surveys) and noncompleters (ie, those who completed the precourse survey but not the postcourse survey), along with those who completed the 4-month survey (n=149) are summarized in Tables 1 and 2. The characteristics were generally similar across completers and noncompleters, except that more Australians tended to complete the program, whereas other geographic locations had similar proportions of completers and noncompleters. Similarly, there was a higher proportion of completers than noncompleters among chiropractors compared with other professions. The small sample of respondents to the 4-month survey was largely physiotherapists, predominantly from Australia.

    Table 1. Demographic characteristics of learners who completed (completers) and did not complete (noncompleters) the e-learning program.
    VariablesCompleters (n=1318)aNoncompleters (n=1145)aCompleted 4-month survey (n=149)a
    Gender, n (%)

    		Female—b—93 (62)

    		Male——54 (36)

    		Do not wish to disclose——2 (1)
    Age (years), mean (SD)

    		≤30——29 (19)

    		51-60——41 (28)

    		≥60——33 (22)

    		31-40——16 (11)

    		41-50——4 (3)
    Main work role, n (%)c

    		Physiotherapist delivering clinical care to patients869 (66)796 (70)121 (81)

    		Physiotherapy student159 (12)100 (9)4 (3)

    		Other health professional student118 (9)43 (4)3 (2)

    		Other health professional delivering clinical care to patients74 (6)61 (5)14 (9)

    		Other27 (2)54 (5)0 (0)

    		Education of physiotherapy students23 (2)51 (4)2 (1)

    		Physiotherapy researcher20 (2)28 (2)2 (1)

    		Education of other health professional students24 (2)8 (1)0 (0)

    		Other health professional researcher3 (0)3 (0)3 (2)
    Location, n (%)d

    		Africa33 (3)45 (4)7 (5)

    		Asia53 (4)96 (8)12 (8)

    		Australia549 (42)373 (33)61 (41)

    		Europe261 (20)265 (23)27 (18)

    		North America372 (28)302 (26)30 (20)

    		Pacific islands36 (3)24 (2)8 (5)

    		South America13 (1)37 (3)4 (3)

    aIndividual characteristics may not add to totals due to missing data.

    b—: not recorded.

    cPercentages calculated for completers and noncompleters based on n=1317 and 1144, respectively.

    dPercentages calculated for completers and noncompleters based on n=1317 and 1142, respectively.

    Table 2. Clinical practice characteristics of learners who completed (completers) and did not complete (noncompleters) the e-learning program.

    		Completers (n=1318)aNoncompleters (n=1145)aCompleted 4-month survey (n=149)a
    Profession, n (%)b

    		Physiotherapist/physical therapist782 (78)692 (84)126 (89)

    		Chiropractor147 (15)54 (7)5 (3)

    		Rheumatologist3 (0)1 (0)0 (0)

    		General practitioner or family physician0 (0)2 (0)0 (0)

    		Sport and exercise medicine physician0 (0)3 (0)0 (0)

    		Orthopedic surgeon0 (0)2 (0)1 (1)

    		Dietitian1 (0)0 (0)0 (0)

    		Podiatrist1 (0)0 (0)0 (0)

    		Other67 (7)70 (8)10 (7)
    Average patients with knee OAc treated per month, mean (SD)12 (17)12 (38)8 (5)
    Clinical practice experience (years), mean (SD)9.4 (10.2)10.0 (9.9)—d
    Strategies usually used to manage people with knee OA, n (%)

    		Education1218 (92)1051 (92)—

    		Exercise1242 (94)1092 (95)—

    		Physical activity advice1164 (88)1003 (88)—

    		Weight loss advice887 (67)746 (65)—

    		Manual therapy610 (46)550 (48)—

    		Acupuncture110 (8)91 (8)—

    		Bracing147 (11)145 (13)—

    		Shoe orthotics130 (10)134 (12)—

    		Other174 (13)137 (12)—
    Currently offer videoconferencing consultations to patients with knee OA, n (%)e

    		Yes366 (28)317 (28)72 (51)

    		No939 (72)817 (72)68 (49)
    Experience with teleconsultations using videoconferencingf, mean (SD)2.4 (1.9)2.3 (1.9)—

    aIndividual characteristics may not add to totals due to missing data.

    bPercentages calculated for completers, noncompleters, and 4-month completion based on n=1001, 824, and 142, respectively.

    cOA: osteoarthritis.

    d—: not recorded.

    ePercentages calculated for completers, noncompleters, and 4-month completion based on n=1305, 1134, and 140, respectively.

    fScored on a 10-point numerical rating scale (1=no experience at all to 10=extremely experienced).

    Aim 2: Effectiveness of the PEAK e-Learning Modules

    On average, learners who completed all modules and the postcourse survey spent a mean of 4.1 (SD 2.7) hours using the e-learning program (n=1318 respondents). Regarding the usefulness of the PEAK e-learning modules (n=1317 respondents), learners scored on average, 8.7 (SD 1.4) on a 10-point NRS, indicating high levels of perceived usefulness. The learners who completed the course reported increased confidence in videoconferencing (Table 3) relative to precourse. This is visualized in Figure 2 and Figure 3, where regarding confidence in using videoconferencing consultations with patients with knee OA, 71.98% (935/1299) of completers scored at least 8 out of 10 on the NRS postcourse compared with just 13.00% (166/1277) before. The likelihood of using education, strengthening exercise, and physical activity in a treatment plan for people with knee OA also increased postcourse (Table 3). Figures 4-6 graphically display the distribution of scores, with the greatest shifts in NRS scores postcourse occurring with the likelihood of using physical activity in a treatment plan. Precourse, 82.99% (1078/1299) of learners were likely to use physical activity in a treatment plan, compared with 96.01% (1252/1304) postcourse.

    Table 3. Immediate changes in confidence with videoconferencing and likelihood to use education, strengthening exercise, and physical activity in a treatment plan for patients with knee osteoarthritis in learners (n=1299) who answered both the pre- and postcourse surveys.

    		Precoursea, mean (SD)Postcoursea, mean (SD)Mean change (95% CI)b
    Confidence with videoconferencing4.8 (2.4)7.9 (1.5)3.1 (3.0-3.3)
    Confidence with videoconferencing for people with knee OAc4.7 (2.4)8.2 (1.4)3.5 (3.4-3.6)
    Likelihood to use education9.3 (1.5)9.7 (0.9)0.4 (0.3-0.5)
    Likelihood to use strengthening exercise9.4 (1.3)9.8 (0.7)0.4 (0.3-0.5)
    Likelihood to use physical activity8.9 (1.6)9.6 (0.9)0.7 (0.6-0.8)

    aScored on a 10-point numeric rating scale (1=not at all confident/likely to 10=extremely confident/likely).

    bCalculated as postcourse score minus precourse score.

    cOA: osteoarthritis.

    ‎
    Figure 2. Distribution of confidence scores with videoconferencing consultations for participants (n=1299) who answered this question pre- and postcourse (where scores of 1=not confident at all and 10=extremely confident).
    View this figure
    ‎
    Figure 3. Distribution of confidence scores with videoconferencing consultations specifically for management of knee osteoarthritis for participants (n=1299) who answered this question pre- and postcourse (where scores of 1=not confident at all and 10=extremely confident).
    View this figure
    ‎
    Figure 4. Distribution of scores (n=1299) regarding likelihood to use education in a treatment plan for patients with knee osteoarthritis (where scores of 1=not at all likely and 10=extremely likely).
    View this figure
    ‎
    Figure 5. Distribution of scores (n=1299) regarding likelihood to use strengthening exercise in a treatment plan for patients with knee osteoarthritis (where scores of 1=not at all likely and 10=extremely likely).
    View this figure
    ‎
    Figure 6. Distribution of scores (n=1299) regarding likelihood to use physical activity in a treatment plan for patients with knee osteoarthritis (where scores of 1=not at all likely and 10=extremely likely).
    View this figure

    Aim 3: Adoption and Implementation of Learnings From the PEAK e-Learning Modules at 4 Months

    Demographic and clinical characteristics of the few learners who completed the 4-month adoption and implementation survey (n=149) are shown in Tables 1 and 2. Regarding health care setting, 48.3% (72/149) respondents were from private practice, 21.5% (32/149) from acute care hospitals, 10.7% (16/149) from rehabilitation hospitals, 16.1% (24/149) from community health center/settings, 1.3% (2/149) from Veterans Affairs settings, and 12.1% (18/149) from other settings. Musculoskeletal health care was the predominant area of clinical practice for respondents (106/149, 71.1%), followed by gerontology with 13.4% (20/149) of respondents.

    Figure 7 shows learner perceptions about the usefulness of the e-learning modules 4 months after completion of the course. More than half described the e-learning program (85/144, 59.0%) and its downloadable resources (88/149, 59.1%) as extremely useful. About 90.1% (128/142) of learners had recommended the PEAK e-learning modules and/or downloadable resources to others.

    Almost all learners indicated that the e-learning modules had changed/informed their usual practice in some way, with 29.9% (44/147) of learners indicating their usual practice was changed to a minor extent, 55.1% (81/147) to a moderate extent, and 14.3% (21/147) to a large extent. Learnings from the e-learning modules were incorporated into clinical practice in a wide variety of ways (Figure 8). More than half of all respondents used their learnings to structure/inform in-person consultations with patients (80/142, 56.3%) and used the patient information booklets in their practice (75/142, 52.8%). Five learners had translated the resources into other languages (Chinese/Mandarin, Greek, Hungarian, Portuguese, and Spanish). Only 4.2% (6/142) of respondents indicated that they had not incorporated any learnings into their usual practice.

    ‎
    Figure 7. Perceptions of 4-month survey respondents about usefulness of learnings.
    View this figure
    ‎
    Figure 8. Strategies used by respondents (n=142) at 4 months to incorporate learnings into usual practice.
    View this figure

    Principal Findings

    This study evaluated the reach, effectiveness, adoption, and implementation of learnings from the PEAK e-learning modules in a real-world context when the e-learning program is provided free of charge to users globally. Our findings showed a broad reach of the program from clinicians (mainly physiotherapists) across the world, predominantly driven through word of mouth, professional organizations, and social media. However, attrition was considerable from registration to the completion of the program. The learners who completed the course reported greater confidence in videoconferencing and increased likelihood of using education, strengthening exercises, and physical activity to manage knee OA. Four months after completion, learnings had been adopted in a range of musculoskeletal health care settings, predominantly by physiotherapists. Most respondents (85/144, 59.0%) to the follow-up survey described the e-learning program as extremely useful, and almost all (146/147, 99.3%) indicated that learnings had changed or informed their usual practice in some way. Collectively, these findings suggest that a self-directed e-learning approach may be an effective and scalable method for educating clinicians in a real-world context when resourcing for human instructors is not possible. Asynchronous e-learning programs also have additional advantages, including flexibility in pacing learning and overcoming the time and costs associated with travel by learners to attend educational courses [27].

    The findings of this real-world evaluation are aligned with those of our previous qualitative evaluation of the training program in a research context [20]. The PEAK e-learning modules were originally developed by our team at the University of Melbourne to train 15 physiotherapists to deliver care in the context of a clinical trial [25] comparing videoconferencing with face-to-face care by physiotherapists for people with knee OA (ongoing). Training for the trial included not only the e-learning modules (as evaluated in this study) but also practical synchronous components, whereby physiotherapists participated in a mock initial consultation via videoconferencing with a physiotherapist researcher (AJK) followed by 4 practice video consultations with 2 pilot patients with chronic (>3 months) knee pain (recruited by research staff). Although there was 100% completion of the e-learning modules by the 15 physiotherapists in the PEAK trial, module completion was mandatory to deliver care in the trial, and physiotherapists were financially compensated for their time spent in training. Our qualitative study [20] exploring the trial physiotherapists’ experiences with the training program showed that physiotherapists valued the self-directed and self-paced nature of e-learning, even though it was unfamiliar to them. Similar to this study, trial physiotherapists reported increased confidence and ability to deliver care through telehealth. They valued the combined package of e-learning modules and the practical components. In contrast, this real-world evaluation focused only on the asynchronous e-learning modules. Despite the lack of structured practical learning components, learners who completed the modules reported a 74% increase in confidence with videoconferencing for people with knee OA, showing that the modules can be effective at scale without practical components of training. These findings are consistent with those of other research showing preliminary evidence of the effectiveness, acceptability, and feasibility of an e-learning program to educate physiotherapists to deliver a group-based self-management complex intervention for low back pain and OA [28].

    There are 37,113 registered physiotherapists in Australia [29]. Thus, our 2077 registrants from Australia represent 6% of registered physiotherapists, suggesting a broad reach to this population. Several factors likely contributed to the broad reach of the PEAK e-learning modules. The COVID-19 pandemic has accelerated a rapid shift to telehealth service delivery for physiotherapy services [30,31]. We released the e-learning modules on April 1, 2020, as part of global efforts to facilitate adoption and implementation of physiotherapy via telehealth [10]. The content of the program is relevant to a wide number of clinical professions, as evident by learner enrollment data, particularly for professions that use exercise and physical activity to manage chronic diseases. Moreover, the telehealth module is relevant to any health professional. Although we did not collect data on why learners registered for the PEAK e-learning modules, it is highly likely that its focus on telehealth was of interest to many, particularly given that allied health clinicians worldwide have described inadequate training, resources, and confidence in telehealth delivery as a barrier to implementing telehealth services during the pandemic [11,32]. Educational courses are considered by many clinicians as a practice-changing phenomenon [27]. Our e-learning modules were free of charge to the learner, which probably also contributed to the large number of people registering for the learning program, spanning 97 countries. Financial constraints are known to be a barrier to participation in professional development among health professionals [33], particularly among those from resource-limited countries [34,35]. A recent systematic review showed that physiotherapists value time as well as accessible and trustworthy resources when undertaking learning and professional development activities [36]. Our e-learning modules, developed by expert researchers at a respected university, delivered at no cost over the internet and able to be undertaken in a self-paced manner aligns with these values. Indeed, a recent survey of 464 health care workers from Sub-Saharan Africa showed that web-based professional development opportunities were accepted and that self-paced internet or computer-based learning is a preferred learning modality [35].

    Consistent with the literature showing greater dropout rates with web-based learning compared with traditional classes [37], we observed high levels of attrition across our e-learning program. The greatest attrition occurred between registration and account activation. Although 6720 people registered for access, only 60.52% (4067/6720) activated their account within the LMS. The reasons for attrition at this point are unclear but may be related to the unwieldy activation (several technical steps are required to create an account as an external user and potential security features such as firewalls/spam that blocked emails from the LMS platform) and navigation processes of the LMS, which were highlighted as barriers in our qualitative evaluation [20]. These factors are consistent with a systematic review of enablers and barriers affecting health sciences e-learning, in which one of the major barriers to e-learning is the lack of user-friendly technology [38]. Of the 4067 users who activated their account within the LMS, 61.00% (2481/4067) completed the precourse survey, 43.99% (1789/4067) completed the first module, 31.99% (1301/4067) completed all modules and the postcourse survey, showing relatively little attrition once learners engaged with the first module. The requirement to complete the precourse survey before commencing the first module may have been a barrier to progressing through the course. Only 9.00% (366/4067) of the learners with activated accounts dropped out from module 1 to module 2. Given that health professionals and students perceive that limitations in their knowledge and skills about OA are barriers to implementing OA care [22], and that clinicians have inadequate training, resources, and confidence in telehealth delivery [11,32], these factors may explain the high retention across the 3 modules. Overall, our findings appear consistent with the literature, where it is reported that 40% to 80% of students drop out of web-based classes [37]. Furthermore, physiotherapists express a preference for face-to-face workshops to address learning needs regarding the management of patients with persistent knee pain [39], regarding web-based learning formats as convenient, but not as effective as face-to-face learning. Given that the learners in our study came from 97 countries, many of which do not have English as the native language, it is likely that many experienced difficulties with the English language of the PEAK modules, probably contributing to the attrition we observed.

    Our data show that the PEAK e-learning modules led to improved confidence in videoconferencing among learners, as well as an increased likelihood of using education, strengthening exercises, and physical activity to manage knee OA. The mean scores for the likelihood of using these interventions were already quite high precourse, with most learners (at least 88%) already using these strategies at the time they enrolled in the modules. This likely explains why the mean change in these scores was quite small. In contrast, confidence with videoconferencing was quite low among our sample precourse, and most (939/1305, 71.95%) were not offering videoconferencing to their patients with knee OA, leading to large improvements (64%-74%) in confidence outcomes postcourse. Interestingly, our adoption and implementation data (n=149) showed that 55.7% (83/149) of respondents used their learnings to structure or inform in-person consultations versus 30.2% (45/149) who used learnings to structure or inform telehealth consultations. It is not clear why there was a greater implementation of learnings in in-person consultations. However, this may be due to physiotherapists experiencing system-level and technological barriers to adopting and implementing telehealth during the pandemic [11,12,32] or related to the preferences of patients and health care professionals for in-person consultations [11,12].

    From an implementation perspective, the proportion of respondents who reported using patient information booklets (79/149, 53.0%), clinician resources (63/149, 42.3%), and video library (43/149, 28.9%) from the PEAK program is noteworthy despite the small sample size (n=149). Furthermore, some of these resources had been translated into 5 languages by users. Twenty-four percent of respondents indicated that they created their own personal resources from the materials provided. These findings are relevant to other developers of e-learning programs for health professionals, highlighting the importance of embedding an implementation tool kit, which contains clinically relevant resources that can be used in patient interactions to facilitate care delivery.

    Strengths and Limitations

    The strengths of our study include the evaluation of an e-learning program in a real-world context. This resulted in a broad reach across the globe, including low- and middle-income countries, as well as those where English is not the native language. Although the program was targeted to physiotherapists, our study participants included not only physiotherapists but also a range of health professionals as well as students, educators, and researchers. Limitations include our relatively short 4-month follow-up of adoption and implementation of learnings, and the low response rate (11% of those invited) to the 4-month email survey, leading to a small sample size (n=149). In addition, owing to the high attrition rate between pre- and postcourse surveys, our findings may overestimate the effectiveness of the PEAK e-learning modules, given that people who found the course useful may be more likely to have completed all modules and the postcourse survey than learners who did not find the modules useful. We also used custom-designed self-reported questions to determine changes in confidence and likelihood of using education, strengthening exercise, and physical activity in an OA treatment plan. As such, it is not clear if the improvements we observed pre- to postcourse are of clinical relevance. Future research should consider collecting patient-level data to determine whether improvements in clinician knowledge and confidence with e-learning translate into better clinical outcomes for patients with the health condition of interest. Although we did not develop the e-learning modules and embedded resources through formal co-design methods, we did refine the e-learning modules based on qualitative feedback from the physiotherapists who used the e-learning modules as mandatory training for the PEAK trial. The embedded patient and clinician resources were developed iteratively over many years by our research team, and consumers with OA provided feedback on resource content during this time. However, we may have seen less attrition with the e-learning modules had we co-designed these modules with physiotherapists at the outset.

    Conclusions

    In conclusion, this study provides evidence of reach and effectiveness of an asynchronous e-learning program provided globally free of charge in a real-world setting among physiotherapists. The PEAK e-learning modules offer clinicians an accessible educational course to learn about best-practice knee OA management, including telehealth delivery via videoconferencing. Attrition across the e-learning program highlights the challenges of keeping learners engaged in self-directed web-based learning.

    Acknowledgments

    This study was funded by the National Health and Medical Research Council (Project grant number 1157977). RSH was supported by a National Health & Medical Research Council Senior Research Fellowship (1154217). KLB was supported by a National Health & Medical Research Council Investigator grant (1174431). AESJ was supported by a São Paulo Research Foundation scholarship (2019/24473-1). The funders have no role in the conduct, analysis, or reporting of this study.

    Conflicts of Interest

    KLB receives consulting fees from Wolters Kluwer for UptoDate knee osteoarthritis clinical guidelines. The Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis (PEAK) e-learning modules are now hosted for free on the FutureLearn (The Open University and SEEK Ltd) platform; however, the University of Melbourne will receive a share of any revenue arising from sales of course upgrades.

    ‎
    Multimedia Appendix 1

    Survey questions embedded within the Learning Management System.

    DOCX File , 27 KB
    ‎
    Multimedia Appendix 2

    Four-month implementation survey questions emailed to learners who completed all modules and the post-training survey within the Learning Management System.

    DOCX File , 38 KB
    1. Safiri S, Kolahi A, Smith E, Hill C, Bettampadi D, Mansournia MA, et al. Global, regional and national burden of osteoarthritis 1990-2017: a systematic analysis of the Global Burden of Disease Study 2017. Ann Rheum Dis 2020 May 12;79(6):819-828. [CrossRef] [Medline]
    2. GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018 Nov 10;392(10159):1789-1858 [FREE Full text] [CrossRef] [Medline]
    3. Bannuru RR, Osani MC, Vaysbrot EE, Arden NK, Bennell K, Bierma-Zeinstra SM, et al. OARSI guidelines for the non-surgical management of knee, hip, and polyarticular osteoarthritis. Osteoarthritis Cartilage 2019 Nov;27(11):1578-1589 [FREE Full text] [CrossRef] [Medline]
    4. Fernandes L, Hagen KB, Bijlsma JW, Andreassen O, Christensen P, Conaghan PG, European League Against Rheumatism (EULAR). EULAR recommendations for the non-pharmacological core management of hip and knee osteoarthritis. Ann Rheum Dis 2013 Jul;72(7):1125-1135 [FREE Full text] [CrossRef] [Medline]
    5. National Clinical Guideline Centre. Osteoarthritis: Care and Management in Adults. 2014.   URL: https://www.ncbi.nlm.nih.gov/books/NBK248069/pdf/Bookshelf_NBK248069.pdf [accessed 2021-11-29]
    6. Kolasinski SL, Neogi T, Hochberg MC, Oatis C, Guyatt G, Block J, et al. 2019 American College of Rheumatology/Arthritis Foundation Guideline for the management of osteoarthritis of the hand, hip, and knee. Arthritis Care Res (Hoboken) 2020 Feb;72(2):149-162. [CrossRef] [Medline]
    7. Teo PL, Bennell KL, Lawford B, Egerton T, Dziedzic K, Hinman RS. Patient experiences with physiotherapy for knee osteoarthritis in Australia-a qualitative study. BMJ Open 2021 Mar 08;11(3):e043689 [FREE Full text] [CrossRef] [Medline]
    8. Teo PL, Bennell KL, Lawford BJ, Egerton T, Dziedzic KS, Hinman RS. Physiotherapists may improve management of knee osteoarthritis through greater psychosocial focus, being proactive with advice, and offering longer-term reviews: a qualitative study. J Physiother 2020 Oct;66(4):256-265 [FREE Full text] [CrossRef] [Medline]
    9. Signal N, Martin T, Leys A, Maloney R, Bright F. Implementation of telerehabilitation in response to COVID-19: lessons learnt from neurorehabilitation clinical practice and education. New Zealand J Physiother 2020 Dec 04;48(3):117-126. [CrossRef]
    10. World Physiotherapy - Impact of the COVID-19 pandemic on physiotherapy services globally. 2021.   URL: https://world.physio/sites/default/files/2021-03/Covid-Report_March2021_FINAL.pdf [accessed 2021-10-06]
    11. Malliaras P, Merolli M, Williams C, Caneiro J, Haines T, Barton C. 'It's not hands-on therapy, so it's very limited': telehealth use and views among allied health clinicians during the coronavirus pandemic. Musculoskelet Sci Pract 2021 Apr;52:102340 [FREE Full text] [CrossRef] [Medline]
    12. Bennell KL, Lawford BJ, Metcalf B, Mackenzie D, Russell T, van den Berg M, et al. Physiotherapists and patients report positive experiences overall with telehealth during the COVID-19 pandemic: a mixed-methods study. J Physiother 2021 Jul;67(3):201-209 [FREE Full text] [CrossRef] [Medline]
    13. World Confederation for Physical Therapy, International Network of Physiotherapy Regulatory Authorities. Report of the WCPT/INPTRA Digital Physical Therapy Practice Task Force. 2020.   URL: https:/​/world.​physio/​sites/​default/​files/​2020-06/​WCPT-INPTRA-Digital-Physical-Therapy-Practice-Task-force-March2020.​pdf [accessed 2021-10-06]
    14. Cottrell MA, Galea OA, O'Leary SP, Hill AJ, Russell TG. Real-time telerehabilitation for the treatment of musculoskeletal conditions is effective and comparable to standard practice: a systematic review and meta-analysis. Clin Rehabil 2017 May;31(5):625-638. [CrossRef] [Medline]
    15. Hinman RS, Lawford BJ, Bennell KL. Harnessing technology to deliver care by physical therapists for people with persistent joint pain: telephone and video‐conferencing service models. J Appl Behav Res 2018 Oct 30;24(2):e12150. [CrossRef]
    16. Bennell KL, Nelligan R, Dobson F, Rini C, Keefe F, Kasza J, et al. Effectiveness of an internet-delivered exercise and pain-coping skills training intervention for persons with chronic knee pain: a randomized trial. Ann Intern Med 2017 Apr 04;166(7):453-462. [CrossRef] [Medline]
    17. Hinman RS, Nelligan RK, Bennell KL, Delany C. "Sounds a Bit Crazy, But It Was Almost More Personal:" a qualitative study of patient and clinician experiences of physical therapist-prescribed exercise for knee osteoarthritis Via Skype. Arthritis Care Res (Hoboken) 2017 Dec;69(12):1834-1844. [CrossRef] [Medline]
    18. Cottrell MA, Hill AJ, O'Leary SP, Raymer ME, Russell TG. Service provider perceptions of telerehabilitation as an additional service delivery option within an Australian neurosurgical and orthopaedic physiotherapy screening clinic: a qualitative study. Musculoskelet Sci Pract 2017 Dec;32:7-16. [CrossRef] [Medline]
    19. Lee AC, Davenport TE, Randall K. Telehealth physical therapy in musculoskeletal practice. J Orthop Sports Phys Ther 2018 Oct;48(10):736-739. [CrossRef] [Medline]
    20. Jones SE, Campbell PK, Kimp AJ, Bennell K, Foster NE, Russell T, et al. Evaluation of a novel e-learning program for physiotherapists to manage knee osteoarthritis via telehealth: qualitative study nested in the PEAK (physiotherapy exercise and physical activity for knee osteoarthritis) randomized controlled trial. J Med Internet Res 2021 Apr 30;23(4):e25872. [CrossRef] [Medline]
    21. Cottrell MA, Hill AJ, O'Leary SP, Raymer ME, Russell TG. Clinicians' perspectives of a novel home-based multidisciplinary telehealth service for patients with chronic spinal pain. Int J Telerehabil 2018 Dec 11;10(2):81-88 [FREE Full text] [CrossRef] [Medline]
    22. Briggs A, Houlding E, Hinman R, Desmond L, Bennell K, Darlow B, et al. Health professionals and students encounter multi-level barriers to implementing high-value osteoarthritis care: a multi-national study. Osteoarthritis Cartilage 2019 May;27(5):788-804. [CrossRef] [Medline]
    23. Lawn S, Zhi X, Morello A. An integrative review of e-learning in the delivery of self-management support training for health professionals. BMC Med Educ 2017 Oct 10;17(1):183 [FREE Full text] [CrossRef] [Medline]
    24. PEAK Training. The University of Melbourne, Melbourne School of Health Sciences.   URL: https:/​/healthsciences.​unimelb.edu.au/​departments/​physiotherapy/​chesm/​clinician-resources/​peak-training [accessed 2021-05-10]
    25. Hinman RS, Kimp AJ, Campbell PK, Russell T, Foster NE, Kasza J, et al. Technology versus tradition: a non-inferiority trial comparing video to face-to-face consultations with a physiotherapist for people with knee osteoarthritis. Protocol for the PEAK randomised controlled trial. BMC Musculoskelet Disord 2020 Aug 07;21(1):522 [FREE Full text] [CrossRef] [Medline]
    26. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999 Sep;89(9):1322-1327. [CrossRef] [Medline]
    27. Whiteley R, Napier C, van Dyk N, Barton CJ, Mitchell T, Beales D, et al. Clinicians use courses and conversations to change practice, not journal articles: is it time for journals to peer-review courses to stay relevant? Br J Sports Med 2021 Jun;55(12):651-652. [CrossRef] [Medline]
    28. Hurley DA, Keogh A, Mc Ardle D, Hall AM, Richmond H, Guerin S, et al. Evaluation of an e-learning training program to support implementation of a group-based, theory-driven, self-management intervention for osteoarthritis and low-back pain: pre-post study. J Med Internet Res 2019 Mar 07;21(3):e11123 [FREE Full text] [CrossRef] [Medline]
    29. AHPRA homepage. AHPRA & National Boards.   URL: https://www.ahpra.gov.au/Publications/Annual-reports/Annual-Report-2020.aspx [accessed 2021-08-02]
    30. Duckett S. What should primary care look like after the COVID-19 pandemic? Aust J Prim Health 2020 Jun;26(3):207-211. [CrossRef] [Medline]
    31. Stanhope J, Weinstein P. Learning from COVID-19 to improve access to physiotherapy. Aust J Prim Health 2020 Aug;26(4):271-272. [CrossRef] [Medline]
    32. World Physiotherapy Response to COVID-19. Briefing Paper 6: Physiotherapy digital practice experiences and insights during COVID-19. 2021.   URL: https:/​/world.​physio/​sites/​default/​files/​2021-06/​COVID19-Briefing-Paper-6-digital-practice-FINAL-2021.​pdf [accessed 2021-10-14]
    33. Haywood H, Pain H, Ryan S, Adams J. Continuing professional development: issues raised by nurses and allied health professionals working in musculoskeletal settings. Musculoskeletal Care 2013 Sep;11(3):136-144. [CrossRef] [Medline]
    34. Feldacker C, Pintye J, Jacob S, Chung MH, Middleton L, Iliffe J, et al. Continuing professional development for medical, nursing, and midwifery cadres in Malawi, Tanzania and South Africa: a qualitative evaluation. PLoS One 2017 Oct 17;12(10):e0186074 [FREE Full text] [CrossRef] [Medline]
    35. Bwanga O. Barriers to Continuing Professional Development (CPD) in radiography: a review of literature from Africa. Health Professions Educ 2020 Dec;6(4):472-480. [CrossRef]
    36. Leahy E, Chipchase L, Calo M, Blackstock F. Which learning activities enhance physical therapist practice? Part 2: systematic review of qualitative studies and thematic synthesis. Phys Ther 2020 Aug 31;100(9):1484-1501. [CrossRef] [Medline]
    37. Bawa P. Retention in online courses: exploring issues and solutions—a literature review. SAGE Open 2016 Jan 05;6(1). [CrossRef]
    38. Regmi K, Jones L. A systematic review of the factors - enablers and barriers - affecting e-learning in health sciences education. BMC Med Educ 2020 Mar 30;20(1):91 [FREE Full text] [CrossRef] [Medline]
    39. Barton CJ, Ezzat AM, Bell EC, Rathleff MS, Kemp JL, Crossley KM. Knowledge, confidence and learning needs of physiotherapists treating persistent knee pain in Australia and Canada: a mixed-methods study. Physiother Theory Pract 2021 Apr 19:1-13 (forthcoming). [CrossRef] [Medline]

    ‎
    LMS: Learning Management System
    NRS: numeric rating scale
    OA: osteoarthritis
    RE-AIM: Reach, Effectiveness, Adoption, Implementation, and Maintenance

    Edited by G Eysenbach; submitted 12.05.21; peer-reviewed by H Slater, D Hurley, C Barton; comments to author 30.07.21; revised version received 13.08.21; accepted 29.09.21; published 01.12.21

    Copyright

    ©Ana Elisa Serafim Jorge, Kim Louise Bennell, Alexander Jared Kimp, Penny Kate Campbell, Rana Shane Hinman. Originally published in JMIR Medical Education (https://mededu.jmir.org), 01.12.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on https://mededu.jmir.org/, as well as this copyright and license information must be included.