Prolonged grief disorder (PGD) is a new diagnosis in the Diagnostic and Statistical Manual of Mental Disorders (DSM) fifth edition text revision [1], and the World Health Organization’s International Classification of Diseases 11th edition (ICD-11) in 2019 [2]. PGD is characterized by persistent pervasive intense grief that interferes with functioning for a period that exceeds expectations of the person’s social, cultural, or religious groups and is at least 6 months after the loss in the ICD-11 and at least 12 months in the DSM fifth edition. The disorder is characterized by persistent intense yearning, longing, or preoccupation with the person who died, accompanied by at least 3 of 8 associated symptoms, also persistent and pervasive to a clinically significant degree and occurring daily for at least the past month: the loss of a sense of identity, marked sense of disbelief about the death, avoidance of reminders that the person has died, intense emotional pain such as anger or sadness related to death, difficulty reengaging with others or with one’s own life, a feeling of emotional numbness, feeling that life is meaninglessness because of death, or intense loneliness as a result of death. Studies have found that PGD is associated with major impairment in social, occupational, and leisure activities [3,4]; increased risk for suicide [5] (rates higher than depression [6]); and negative health consequences, for example, cancer and cardiovascular disease [3].

The introduction of a new mental disorder begs the question of available treatments. In the case of PGD, PGD therapy (PGDT; formerly called Complicated Grief Therapy) was developed [7] and tested in 3 randomized controlled trials with a total of 641 participants [8-10] before the inclusion of PGD as an official diagnosis. This treatment research initiative paralleled and contributed to the research that validated the criteria for new diagnosis [11,12]. Each of the 3 studies, sponsored by the National Institute of Mental Health, compared PGDT to a proven efficacious treatment for depression, either interpersonal psychotherapy in 2 of the studies or antidepressant medication in the third. PGDT produced an average response rate of 71%, compared with 33% for depression treatment. Importantly, depression is most often confused with PGD [8,9,13].

PGDT is a short-term (16 sessions) integrated psychotherapy targeting adaptation to loss. PGDT is based on research-informed principles and evidence-based methods from cognitive behavioral therapy, interpersonal psychotherapy, motivational interviewing, positive psychology, and psychodynamic psychotherapy. Attachment theory is used to understand bereavement and grief and to define the treatment goal of facilitating adaptation to loss. Following the dual-process model of coping with bereavement [14], adaptation is conceptualized as entailing the acceptance of the reality of the loss and restoration of the capacity for well-being. The foundational premises of PGDT are that grief is a stress response and a form of love that emerges naturally and finds a place in our life. Although everyone grieves and adapts in their own way, there are commonalities. Adapting to loss progresses naturally if it does not become derailed. Derailers are naturally present during early grief and can get in the way of adapting if they persist over time and gain too much prominence in mental functioning [15]. PGD therapists use active listening and personalized interventions, as they work through a planned sequence of sessions and a series of well-specified psychological exercises. These exercises provide experiential learning opportunities for each of the 7 themes that operationalize the process of adapting to loss, understanding and accepting grief, managing grief-related emotions, seeing a future with promise, strengthening relationships, narrating a coherent story of death, living with reminders, and feeling connected with memories (Figure 1). The first 2 themes help patients understand and manage grief; the next 2 focus on restoring the capacity for well-being using the self-determination theory goals of autonomy, competence, and relatedness [16]. The last 3 themes help patients to accept the reality of the loss and establish a sense of connection with the person who died. For more detailed information, refer to the study by Shear [16].

The critical shortage of clinicians trained in evidence-based treatment has been amplified by the pandemic. The rate of psychiatric disorders has increased [17], with more patients seeking help [18]. This has already put a strain on clinicians who are unlikely to be knowledgeable about PGD or PGDT. This means that a large number of therapists will need to be trained in a short amount of time. One way to facilitate access to training is through the use of digital technologies.

Internet-based training, both synchronous and asynchronous, can be available to any clinician with internet access. Asynchronous training is unconstrained by enrollment limitations, trainer availability, and time limitations, as busy clinicians can work at their own pace [19]. Using the principles of instructional design, such as high interactivity and multimodal learning, enhances the quality of training and increases knowledge retention [20]. Web-based technologies have been used to successfully train clinicians in several evidence-based treatments such as interpersonal psychotherapy for depression [21], cognitive therapy for adolescent depression [22], anxiety disorders [23], and drug abuse [24]. Web-based training has also been used to help train non–mental health clinicians to deal more effectively with mental health issues, such as emotional trauma [25] and adolescent mental health [26]. While no web-based therapist training for treating grief has been reported, several web-based self-help interventions for grief have been published [27-30].

In a review of the use of technology to train clinicians in evidence-based treatments, Singh and Reyes-Portillo [31] found that technology-based training can be just as effective as traditional training and has the potential to facilitate the adoption of evidence-based practice. Fairburn and Wilson [32] suggested that internet-enhanced technologies might provide the only scalable solution to the challenge of disseminating clinician training on evidence-based treatments as well as supporting the actual use of skills in clinical settings after completing training. Internet-based training can also provide opportunities for ongoing updates, specific steps to prevent drift, retesting, and monitoring. In addition, we have had positive experiences combining synchronous and asynchronous training methods.

A major issue in training clinicians on evidence-based treatments is not only how to facilitate the uptake of conceptual knowledge but also how to teach and assess effective clinical implementation of the treatment. Several studies have found that web-based training is more effective when followed up by clinical consultation [33-35]. However, live supervision is costly, time-consuming, and not easily scalable. To help make this aspect of training scalable, German et al [36] used a train the trainer approach, where a cohort of clinicians within a setting were trained by experts, using in-person workshops followed by live supervision. Once trained, these clinicians used their expertise to train new clinicians within the group [36] using a combination of web-based training followed by live supervision. They found that the combination of web and live training was as effective in producing clinical competency as training performed entirely by clinicians. Murphy et al [37] used web-based video playback technology (mPath) to help trainees reflect upon their interpersonal counseling skills in specific therapy sessions. In our previous studies, we used live remote observation via videoconference for training on applied skills using standardized patients [38] or trainers playing the parts of the patient while providing live feedback [21,23,39,40]. These studies found that live remote training improved not only didactic knowledge but also applied clinical skills and led to successful treatment outcomes [22,23]. However, this approach is still costly and does not solve the scalability problem.

In the tutorial, we included a small pilot component to address clinical implementation strategies using web-based multimedia patient scenarios with animated vignettes or previously produced actors with scripted videos. This approach allows for repetitive practice and immediate feedback in a safe and structured environment. To pilot this effort, users observed an actor-therapist interacting with an actor-patient (or an animated therapist interacting with an animated patient). This scenario portrayed a challenging clinical situation that clinicians might encounter when performing PGDT. Users indicated one of several possibilities for how they would respond to a situation. Through practice and feedback, clinicians can learn ways to engage in similar conversations with actual patients. The use of similar actor-patient scenarios to train clinicians has been reported for suicide risk assessment [41] and the treatment of major depression [42], alcohol [43], and substance use disorders [44].

The purpose of this study was to report on trainees’ experiences and their learning outcomes after completing our recently released web-based therapist training tutorial on PGD and PGDT. Specifically, we measured (1) trainee satisfaction with the tutorial, (2) improvement in trainee knowledge of the principles and procedures used in PGDT, and (3) improvement in trainee choice of clinical implementation strategies in delivering PGDT.

Participants were therapists with a mental health–related degree or graduate students in a mental health program, recruited between October 2021 and May 2022. Study participants were recruited from electronic mailing lists of licensed psychologists in New York State, announcements to persons on e-newsletter lists from the Columbia Center for Prolonged Grief, announcements to individuals who had previously taken a course at the Center for Telepsychology, announcements to graduate students at the Columbia School of Social Work, and by word of mouth. Interested individuals reviewed the web-based information sheet consent form and indicated that they freely agreed to participate in the study. Those agreeing to participate were provided a username and password and linked to a brief demographic survey, a 55-item multiple-choice pretest on the principles and concepts of PGD and PGDT, and a 4-item web-based pretest to assess PGD clinical implementation strategies. Once the demographic survey and pretests were completed, the link to course content was activated.

The web-based PGDT tutorial contains didactic information, web-based exercises, simulated patient scenarios, animated graphics, ongoing web-based self-tests, and video examples of patient role-plays in a multimodal, multimedia, and web-based learning approach that research has found to enhance learning efficacy [20] (Multimedia Appendix 1).

The tutorial contains 11 modules covering the following topics: the nature of grief, an overview of PGD and PGDT, pretreatment assessment, a module for each of the 7 PGDT themes, and a final module that provides a summary of the treatment progress and addresses treatment termination. As PGDT is a measurement-based approach (ie, an intervention that includes regular structured assessment with simple validated instruments) [45], the tutorial reviews the assessment tools used in PGDT and how to integrate them into treatment. Each module is approximately 20 to 40 minutes long.

Trainees work through the tutorial at their own pace. However, they are encouraged to space out the time they work on it rather than take it in a few long sessions, as spaced learning increases knowledge retention [46]. A posttest is given immediately after completing each module, as testing performed closer to when the material was learned improved retention [47]. Consistent with continuing education guidelines, successful completion requires an overall score of 80% on the posttests. Users can retake the module until a passing score is obtained. The participants in this study were given 8 weeks to complete the training.

This study used a pre- and poststudy design. Paired t tests (2-tailed) were used to measure the pre- to posttest changes in PGDT knowledge and clinical implementation strategies. The effect sizes (Cohen d) were calculated to examine the magnitude of the changes. The McNemar chi-square test was used to test the differences in percentages. Descriptive statistics were calculated for user satisfaction and learning objectives data.

The study was reviewed and approved by the Columbia University Institutional Review Board on October 13, 2021 (registration number: AAAT7389). The approval process also obliged us to register this study at ClinicalTrials.gov. All participants provided written informed consent before being given access to the tutorial.

The demographic characteristics of the participants who completed the tutorial are presented in Table 1. The sample was primarily female (178/196, 90.8%) and White (159/196, 81.1%), with a mean age of 48.9 (SD 13.7; range 22-79) years. The sample predominantly included social workers and psychologists with a smattering of other mental health professions. Therapists had a mean of 2.7 (SD 11.0; range 0-44) years’ experience doing therapy. Most (115/196, 58.7%) reported having had prior grief training, and 85.7% (168/196) reported having personally experienced grief.

As shown in Table 2, trainee scores on PGDT concepts and procedures ratings showed statistically significant improvement from pretraining to the postmodule assessment, with the total number of correct answers increasing from 29 (SD 5.5; 53% correct) to 36.7 (SD 5.2; 67% correct; t₁₉₅=18.93; P<.001; Figure 2; effect size: Cohen d=1.44; 95% CI 1.23-1.65). The largest increases in scores were found in modules 2 (pretreatment assessment) and 8 (living with reminders), and the smallest increase was found in module 10 (putting the treatment together and managing treatment ending). While those with prior grief training scored slightly higher on the pretest than those without prior grief training, 29.9 (53%) versus 27.6 (50%; t₁₉₄=2.93, P=.004), the mean change was 7.7 for both groups. Not surprisingly, there was a difference between the mean change for graduate students (n=16; 1=5.8; SD 5.9) and licensed clinicians (n=179; 1=7.9, SD 5.7; t₁₉₄=1.41, P=.19).

The mean change in trainees’ scores on the pilot clinical implementation assessment increased substantially, from 2.6 (SD 0.7; 65%) correct out of 4 to 3.1 (SD 0.4; 78%) out of 4 (t₁₈₈=7.03; P<.001). The standardized effect size was d=1.05 (95% CI 0.84-1.29). The changes in each response are presented in Table 3. There was no major difference in this measure between those with and without prior grief training (t₁₈₆=0.98; P=.32) or between clinicians and graduate students (t₁₉₃=1.40; P=.16).

The principal findings of this study support the growing body of literature supporting the helpfulness of asynchronous web-based technologies for training clinicians in evidence-based treatments [31,54,55]. Scores on trainee ratings on our PGDT assessment measures increased substantially, as, to a lesser extent, did our trainee clinical implementation strategy assessment. In addition, the high response rate to our recruitment notices, as well as the unusually large percentage who completed the tutorial, suggests a high level of interest in this tutorial. Higher user satisfaction was further supported by the User Satisfaction Questionnaire scores and the percentage of trainees reporting that the learning objectives were met. The high completion rate we obtained was notable and may be related to the appeal of this program, including appealing interactivity, novelty, animated and video clinical examples, and the novelty of a new DSM diagnosis that already has a well-validated treatment approach. User satisfaction is especially important in asynchronous web-based training, where there is a high risk of discontinuing training.

Given the recent introduction of PGD in ICD-11 and DSM fifth edition text revision, as well as a marked increase in persons with prolonged grief owing to the pandemic, rapid dissemination of training on effective PGD treatment is needed. Over 6 million people died worldwide during the pandemic. A total of 1.1 million people died of COVID-19 in the United States [56], adding substantially to the 2.8 million deaths per year. A recent US demographic study estimated that each death leaves about 9 bereaved relatives [57], which suggests that the bereaved outnumber the deceased by nearly 10-fold [58]. Overall, the rates of PGD among all bereaved individuals have been estimated to be approximately 10% [59], with higher rates for more difficult deaths. Circumstances of pandemic deaths have been especially challenging and thus qualify as particularly difficult deaths that are likely to be associated with elevated rates of PGD.

Notably, 85.7% (168/196) of the trainees reported having experienced a major loss. Many therapists become interested in grief therapy after experiencing their own grief. Life experience is a good way to understand the experience of a patient, so this may be a benefit. Alternatively, a personal loss may sensitize a therapist in a way that might make therapy more challenging. However, we found no difference in responses to this tutorial among those who did or did not report an important grief experience.

Therapists with prior grief training scored substantially higher on our pretest than those without this training, but the difference was only a few points. The scores of both groups showed a reliable increase on our assessment questionnaire after taking the tutorial. This suggests that even experienced therapists and those with prior personal grief experience or prior grief training can benefit from this training on PGD and its treatment. However, a mean score of 67% indicates that more learning is required to achieve optimal training results.

We included 4 pilot questions related to the use of PGDT clinical implementation skills, which mostly showed improvements in these assessments, suggesting that this is a promising extension of web-based training that might be further developed. Live role-plays with immediate feedback and clinical supervision may still be the best way to support the development of clinician skills [33-35]. However, given that PGD is a new disorder, there are not yet sufficient numbers of PGDT trainers to meet the needs of a large number of clinicians needing this training, and the observation of the trainee’s role-plays (either live or remote) is not yet scalable. Web-based technology can offer a useful tool to augment didactic training. Web-based patient scenarios provide an alternative in which trainees may gain comfort in implementing and experimenting with new skills. This may be a more comfortable way to receive feedback on their performance, especially in the early stages of training. Future work should continue to build methods for the effective asynchronous practice of clinical skills.

This study has several limitations. First, we recruited individuals through internet announcements to a wide range of professionals, but those who signed up were mostly more experienced therapists, and the majority had already had grief training. It is important to know how therapists who are less comfortable or knowledgeable about grief would respond to this tutorial. Second, perhaps the most important limitation of this study is that there are no patient outcomes and no measures of therapist adherence when providing therapy. Thus, it is unclear whether training improves treatment efficacy. Third, although we showed large pre-post effect sizes in our measured outcomes, our conclusions are limited by the lack of information about the reliability and validity of the test items. In addition, posttests were conducted immediately after the material was presented. Whether the knowledge gained was retained is unknown. Fourth, the therapists in our sample were primarily White (159/196, 81.1%). Most concerning, only 6.1% (21/196) self-identified as Black compared with the population prevalence of 13% in the United States and the elevated rates of both yearly death rates and PGD among people of color. This low rate may be due to failure to reach Black professionals; failure of Black therapists to be interested in the tutorial; or perhaps most concerning, a low level of Black professionals trained as therapists. A secondary analysis of our most recent intervention efficacy study showed no difference in response rates between individuals who self-identified as White or Black. Clearly, more work is needed to ensure that efficacious evidence-based treatment is available to people of different cultures [60].

This preliminary study provides support for the effectiveness of web-based training in teaching clinicians to recognize PGD and administer PGDT. The inclusion of a web-based multimedia tutorial for didactic training and simulated patient scenarios to develop PGDT-related clinical skills holds promise for increasing the effectiveness of web-based training. The model and components used in this tutorial model may also be helpful in the dissemination of training for other evidence-based treatments. Web-based training may help facilitate training in evidence-based treatments by overcoming barriers owing to limited trainer capacity, time, and scheduling constraints [61,62]. Further studies are warranted to determine the reliability and validity of the tutorial outcome measures and explore the optimal use of web-based training in the context of other approaches to PGDT training. Such studies would provide data that would enable a cost-benefit analysis of the best ways to integrate each approach into the training of grief therapists.