Defining a Role for Webinars in Surgical Training Beyond the COVID-19 Pandemic in the United Kingdom: Trainee Consensus Qualitative Study

Introduction

The COVID-19 pandemic has significantly impacted many aspects of surgical training in the United Kingdom and the Republic of Ireland and has challenged the delivery of surgical training with the cancellation of examinations, training courses, and teaching programs, and a significant reduction in exposure to operative cases [1]. A study comparing surgical trainees’ operative logbook numbers in 2019 and 2020 reported an overall incident rate ratio (IRR) of 0.62, with exposure to elective surgery more affected (IRR 0.53) than emergency surgery (IRR 0.85) [2]. Subsequently, surgical training had to adapt, resulting in an accelerated shift in the use of digital learning environments by both trainees and trainers.

A webinar is defined as “a seminar conducted online” [3]. A seminar is a teaching method based on the Socratic dialogue of asking and answering questions with the word originating from the Latin seminarium, which means “seed plot” [4,5]. Webinars have evolved as a result of technological advancements leading to faster, more reliable internet connections and the use of video calls as a standard method of communication in today's society. As a result, digital learning was slowly being integrated into pedagogical learning methods to create new blended learning methods [6]. However, the COVID-19 pandemic presented a new opportunity for the use of webinars to provide remote learning for surgical trainees, and, as a result, they have become a popular and increasingly prevalent training tool [4]. One review of the use of webinars for training in plastic surgery indicated an increase of 12,017% in the number of webinars relevant to this specialty post lockdown in the United Kingdom on the March 23, 2020 [4].

While the pandemic forced training to transition to the internet, the question remains if and how webinars should be integrated within surgical training going forward as the National Health Service’s services start to recover and surgical training adapts to the new norm. The Joint Committee on Surgical Training quality indicators for Higher Surgical Training states that trainees should have a minimum of 2 hours of facilitated formal teaching every week [7].

This study, led by the Association of Surgeons in Training (ASiT), aimed to identify the role of webinars in surgical training, both during and after the COVID-19 era, and consider what, in a best-case scenario, their potential integration into surgical training would look like in the future. To achieve this, the following objectives were set: (1) to investigate the strengths and weaknesses of webinar-based training for surgical specialties, (2) to establish the limitations of webinars and the barriers faced when integrating them into surgical training, and (3) to propose recommendations for its integration within surgical training.

Methods

Overview

This study was led by the ASiT via a steering committee of surgical trainees. The ASiT is an independent professional organization (registered charity number 1196477), with a membership of over 3500 surgical trainees, that promotes excellence in surgical training. The organization represents and supports trainees across all surgical specialties and training grades on a regional and national level in the United Kingdom and the Republic of Ireland and is the largest representative body for surgical trainees. The ASiT was originally founded in 1976 and is independent of, but works with, the National Health Service, General Medical Council, Surgical Royal Colleges, the Joint Committee on Surgical Training, and the Trainee Surgical Specialty Associations.

This study used mixed qualitative methods in accordance with COREQ (consolidated criteria for reporting qualitative research) guidelines [8] to consolidate the arguments for and against webinars and the drivers and barriers to their delivery in surgical training. This involved 3 phases: (1) a web-based survey, (2) focus group interviews, and (3) a consensus session at the annual ASiT 2021 international conference.

Phase 1: Web-Based Survey

A survey was developed to gather wider trainee perspectives on webinars in surgical training. It was built using Qualtrics (Qualtrics XM) in accordance with the published CHERRIES (Checklist for Reporting Results of Internet E-Surveys) guidelines on conducting web-based surveys with the aim of establishing broad opinion and key themes concerning the use of webinars before and during the COVID-19 pandemic [9-11]. The survey contained binomial, multiple-choice, Likert-scale questions and those with free-text responses (Multimedia Appendix 1) and was developed on the basis of consensus from the steering group. The survey was peer reviewed and piloted by the ASiT council prior to dissemination, where 1 question was omitted and 2 questions reworded, resulting in a final set of 26 questions. The survey was sent out to all ASiT members via the ASiT mailing list (MailChimp) and advertised via ASiT social media channels (Facebook and Twitter). Survey completion was voluntary and open to current and future surgical trainees of all grades and specialties. All responses were collected with informed consent provided by responders at the time of completion, and anonymized over a 6-week period (December 3, 2020, to January 14, 2021) without sampling. Thereafter, the survey was removed from the ASiT website. Descriptive statistics were used to aggregate data from predefined questions. Responses to a final free-text question—“Please feel free to write any other comments regarding your experience of teaching/training via webinars.”—were subjected to inductive thematic analysis by 2 authors (AW and JH) independently.

Phase 2: Focus Group Interviews

Trainees interested in participating in the interview stage were contacted via email with further study information and available interview slots between Monday, February 8, and Sunday, February 14, 2021. The interviews were arranged as 30-minute focus group sessions with up to 4 participants in each group and one member of the study group present to facilitate the session, prompt discussions, and answer queries. These sessions were conducted as web-based video calls using Zoom (Zoom Video Communications). Each session was recorded and transcribed with participants consent. Interviews were conducted using structured discussion. Interview transcripts and recordings were analyzed using thematic analysis, which produced 4 themes including strengths, limitations, drivers, and barriers, with focused questions generated through steering group consensus from the initial survey responses. All focus group participants were offered collaborative authorship [12].

Phase 3A: Formation of Consensus Statements

This project used a transparent consensus process [13]. Based on the findings of the survey and focus group interviews, a preliminary list of consensus statements (Multimedia Appendix 2) was formed by the steering committee (EB, WZ, JH, AW, BD, and JB), and refined with input from the ASiT council, into a final list for presentation during the consensus meeting (Multimedia Appendix 3). These were designed to capture the perspectives gathered so far and structured around the following core areas: the role, timing, conduct, content, opportunities, and knowledge gaps for webinar use.

Phase 3B: Consensus Meeting

The consensus meeting was held at the 2021 ASiT annual international conference, which was the first ASiT conference to be held remotely. It was advertised as free for any conference delegate via the conference platform (MedAll) and program, the ASiT website [14], and social media platforms. A Google Form (Google LLC) was shared among attendees and used to record the name, gender, specialty, grade, and deanery of all participants who attended the meeting. The session was held using Zoom, and discussion during the meeting took place both verbally and using the question-and-answer function. Slido (sli.do s. r. o.) was used during the consensus session to present each consensus statement and permit attendees to vote. Votes were binary (yes/no or agree/disagree). For a statement to be ratified, 70% or more of the consensus participants had to vote in agreement (yes/agree). The cutoff of 70% was based on the Grading of Recommendations Assessment, Development and Evaluation approach and agreed on by the study group prior to the session [15,16]. The Slido software automatically stored the results and facilitated their export for data analysis.

Ethics Approval

According to the Health Research Authority Guidance, ethics approval was not sought for this study.

Results

Phase 1: Web-Based Survey

Demographics

The survey was sent to 2790 ASiT members. The total number of survey responses was 281, with a response rate of 9.9%. In total, 278 complete responses were included in the final analysis, as incomplete responses were excluded.

All training regions in the United Kingdom and the Republic of Ireland were represented. In total, 56.7% of respondents were female. The majority of respondents were trainees in general surgery (43.2%), followed by orthopedics (21.6%), with responses from trainees in all other surgical specialties. There were responses from core surgical trainees (41.0%), specialist registrars (35.6%), foundation trainees and medical students (17.9%), specialty doctors or associate specialists (3.8%), and post–certificate of completion of training fellows 1.7%; Multimedia Appendix 4).

Key Findings

Respondents reported attending more training webinars in 2020 than in 2019 (Figure 1) and 96.5% of respondents agreed that this change was due to the COVID-19 pandemic. Overall, 98.9% of respondents agreed that webinars have become a standardized format for surgical teaching to replace face-to-face teaching during the COVID-19 pandemic. However, 80.9% agreed that webinars cannot fulfil the practical aspects of surgical training.

Specific reasons for attending more webinars included webinars being the only resource available (28.9%), more webinars being available (23.4%), to meet specific training requirements (15.8%), more awareness of webinars (15.6%), and webinars being an effective way to use time (14.3%). Only 10 respondents did not attend webinars, and the reasons for this included that there are too many webinars, so it is difficult to choose which ones to attend, and that they seem less effective for surgical training. The main factors that are considered when deciding to attend a webinar include the topic (27.6%), training requirements (20.6%), the speaker (15.4%), and cost (15.6%).

Regarding the delivery of webinars (Multimedia Appendix 5), the preferred duration of a webinar for 69.1% of respondents was 30 to 60 minutes. The most popular time of webinars was weekdays out of working hours. The majority of respondents had 1-2 hours available to attend webinars each week.

Respondents were asked to rate aspects about the effectiveness of webinars on Likert scales. Overall, the mean score for the webinar as a format for surgical training during the COVID-19 pandemic was 6.8 (SD 1.8) out of 10. The organization and structure of webinars had a mean score of 7 (SD 1.86) out of 10. The interaction between speakers and participants had a mean score of 5.2 (SD 2.22) out of 10, and technical aspects scored 6.4 (SD 2.29) out of 10. Meeting the demand of practical education scored poorly at 4.3 (SD 2.75) out of 10. Most people (44.5%) felt that the topic of the webinar determined whether it would be as effective as face-to-face teaching. Overall, 91.2% of respondents felt that they were more likely to ask questions during face-to-face teaching than during a webinar and that they were more likely to pay for face-to-face teaching.

Going forward, 90.2% of respondents were very likely, likely, or somewhat likely to attend training webinar after the COVID-19 pandemic. Respondents thought that webinars and web-based learning would be most useful for exam preparation, journal clubs, annual reviews of competency progression), and supervisor meetings (Multimedia Appendix 5).

In total, 106 respondents provided free-text answers when asked if they had any other comments about webinars. The themes from these responses were identified as demonstrated in Table 1.

‎View this figure

Phase 2: Focus Group Interviews

Demographics

In total, 18 trainees (outwith the steering committee facilitators) participated in 8 semistructured interviews conducted in February 2021. The interviewees were trainees in general surgery (n=7), trauma and orthopedics (n=5), otolaryngology (n=1), plastic surgery (n=1), urology (n=1), and vascular surgery (n=1). The foundation trainees (n=2) did not declare their intended surgical specialty. The training grades of the interviewees were specialist registrars (n=8), core surgical trainees (n=6), foundation trainees (n=3), and senior house officer–level staff grade (n=1; see Multimedia Appendix 4).

Key Findings

Interview responses generally supported survey findings with regards to strengths, limitations, drivers, and barriers. Recurrent themes of accessibility, timing, and technical aspects were explored in more detail.

Accessibility and Cost

Avoiding travelling to remote locations was reported as beneficial for time, financial, and environmental reasons. Accessibility to worldwide experts offered an opportunity to access international speakers and panels usually only available at major conferences, while avoiding the associated cost and study leave. On a regional level, webinars allowed deanery teaching programs to be combined to maintain high-quality speakers despite reduced speaker availability. There was considerable heterogeneity in individual experience, a minority of interviewees reported formal protected time and spaces for locally arranged webinars, this may not be practical for national or international events.

Timing of Webinars

Interviewee preferences for timing of webinars varied with their individual circumstances. Some identified midweek evenings as a time with less conflicting demands. Concerns were raised that a reliance on out-of-hours webinars disenfranchised those with family or other commitments and could contribute to burnout. One trainee highlighted the diversity of views, explaining that as she lived away from her family during the working week, she preferred to watch webinars during weekday evenings to keep her weekends and working days free. Some expressed a preference for watching webinars at home, others found it difficult to focus with multiple distractions. Access to study leave for webinars during working hours was reported as limited.

Recordings

The ability to view recordings later was welcomed as adding flexibility, with the reservation that recordings lose interactivity, particularly the opportunity to ask questions. Repeat recordings may not remain up to date with current evidence and guidelines and may need to be reviewed. Archiving of content is variable and could be improved.

Technical Aspects

Many interviewees described variation in technical fluency between presenters. Live polls, chat, and question-and-answer functions were highlighted as improving interactivity. Equipment and bandwidth problems were common; in one example, an anatomy session from a dissecting room had insufficient resolution to identify structures. This may represent the learning curve of adopting new teaching methods and could be addressed through targeted training on the use of web-based platforms, and appropriate technical support. When available, a session chair can have a valuable role to set expectations and manage features (including muting and unmuting, screen sharing, and reviewing audience chat) to allow the presenter to focus on presenting.

Future Opportunities

To counter the loss of social interaction, a suggestion was made that a hybrid approach with remote access to face-to-face teaching offers a solution for deanery teaching to reduce the need to travel. Small groups meeting locally to access remote teaching would allow some element of networking and social interaction.

Phase 3: Consensus Session

Demographics

The consensus session was attended by 33 delegates, of whom 32 completed the demographics questionnaire. In total, 20 (62.5%) participants were female and 12 (37.5%) were male. A range of training grades and surgical specialties were represented among the consensus session cohort with specialist registrars (ST3-8; 43.8%) the most represented grade and general surgery (37.5%) the most represented specialty (see Multimedia Appendix 4).

Voting Results

Of the 25 Statements, 23 obtained at least 70% approval from the consensus meeting participants (Table 2) and are included in the recommendations of this paper.

Discussion

Principal Findings

This initiative has produced the first trainee-led, consensus-based recommendations on the use of webinars in surgical training with unanimous agreement that webinars have a role in future surgical training. Both practical and communication training was considered poorly suited, although it was acknowledged that this may change as methods of remote training evolve. These 23 consensus recommendations should support their continued and effective implementation and iteration.

A role for webinars in surgical training was anticipated, in line with the positive experiences that have been increasingly published across health care disciplines globally [17]. Blythe and Thompson [18] report the successful implementation of a novel web-based surgical teaching program in Northern Ireland with support from the Royal College of Surgeons Edinburgh. The program was designed to meet the needs of core surgical trainees during the COVID-19 pandemic owing to cancellation to face-to-face teaching and used both videoconferencing software and web-based webinars. As a result of the positive feedback received, the course has been further developed into the primary surgical teaching method for the core surgical training cohort of 2020-2021 in Northern Ireland. This shows the sustainability and value of remote education going forward [18].

Webinars allow the remote delivery of expert teaching to a large number of trainees simultaneously, and recording these sessions offers further opportunity to increase the audience to people unable to attend live. This also provides trainees the option of revisiting the webinar to consolidate their learning or revise it at a later date. As observed during the development of our ASiT recommendations, accessibility and convenience are the commonly cited reasons for their use. The Virtual ACCESS conference (a core-trainee–led web-based conference to enhance surgical succession) surveyed delegates and found that the major factors that attracted attendees to their web-based conference were that the conference was free (91/130, 70%), allowed an opportunity to present (81/130, 62.3%), and did not require travel (78/130, 60%) [19].

Limitations

However, as highlighted in our process, webinars elsewhere also have their limitations. This seems to be largely around variation in technical delivery [20]. This was a large focus of the feedback received during our initiative and subsequently recognized within the recommendations; for example, around timing, duration, technical support, and archiving. An expectation of (unpaid) availability out of hours has implications for the costs of surgical training [21]. Although not specifically for webinars, technical challenges are a commonly cited barrier in telemedicine and are more frequently studied [20]. The preference for live delivery was also noteworthy, given that many web-based education initiatives use recorded videos [22,23]. As the role of guidelines is to improve experience by standardizing practice with the best available evidence [24], it is hoped that these recommendations can therefore have an immediate and positive impact.

However, there are some important limitations to acknowledge in our process. First, this process only captured the perspective of trainees. Therefore, it did not incorporate evidence from medical education, such as the comparative effectiveness of webinars, nor did it incorporate the perspective of surgical trainers specifically [25]. We also acknowledge a low response rate. The collaborative coauthorship model used and the outlined study design were chosen to incentivize participation. While these must be considered, it is worth recognizing that response rates parallel those of similar surveys [26-28] and perspectives across disciplines and training stages were still represented. Further, trainees are typically involved in education themselves and will have a trainer’s perspective.

Second, while most recommendations reached near unanimous consensus, there were inconsistencies. For example, while the initial survey suggested that trainees did not feel that communication skills were well suited to webinars, this did not become a consensus recommendation. This discordance may reflect an evolving perspective on webinars and communication as the initial concerns centered around being unable to recognize nonverbal communication in a web-based environment. Given that clinical practice has also transitioned to many web-based consultations [29], it is likely that remote teaching will have a more relatable role. Similarly, while practical training was felt to be better suited to alternative teaching methods, this was not unanimous, with agreement that, particularly as training tools develop [30], the role of webinars in practical training be investigated further. Of note, Fehervari et al [31] reported the first web-based, large-scale, surgical skills course that compared different web-based teaching methods and was validated against the gold standard of face-to-face teaching. The published feedback showed the success of this web-based course as delegates felt that the event met the required standards of a high-quality surgical teaching course and did not feel that face-to-face teaching would have been more appropriate despite the course focusing on practical skills [31]. This differs from the results of our consensus session and shows that there is scope for further research into the utility of webinars and web-based teaching for practical training.

The inconsistencies likely reflect the rapidly changing landscape of webinars in surgical education [20], and the need to iterate these recommendations as evidence and experience changes [29]. Based on the uncertainties identified during this process, a number of key knowledge gaps for clarification were identified, which too should help to guide their optimization.

Therefore, while we consider these recommendations to have been formed through an inclusive and structured process, and of immediate value, as with any form of recommendation, they should be considered a starting point that will require update over time.

Conclusions

This is the first initiative to produce consensus-based recommendations on the role and use of webinars within surgical training. These recommendations were produced by trainees and should serve as an initial framework for ensuring that they add value to surgical training in the future.