PCPs (ie, physicians, nurse practitioners, or other trainees) from the VA, other community health care systems, and university affiliates across the United States were recruited via email. PCPs with at least five military service veterans on their panels were eligible; PCPs lacking adequate computer or internet speeds to support VW technology were excluded. Following baseline self-report assessments and SP interviews, consenting PCPs were randomized to either the VW training (intervention) or the web-based training (control).

Ethical Considerations

The study was approved by the Institutional Review Board of the University of California, San Francisco, and the Research Protection Program of the San Francisco VA Health Care System (IRB number 14-15004). All participants provided consent for their participation in this study. Participants received a gift card valued up to US $50 and the opportunity to earn up to 5.75 CME credits for participation (the renumeration amount varied depending on whether the participant opted to receive CME credit). Participants also received US $20 if they participated in a qualitative interview at the end of the study. All study data have been deidentified in this manuscript.

The overall goal of this pilot randomized controlled trial was to assess the feasibility, acceptability, usability, and preliminary efficacy of a VW training format, compared to an asynchronous web-based training in improving PTSD-related assessment and management as well as MI skills among PCPs. PTSD assessment skills were measured by participant self-report measures. PCP skills were assessed via standardized behavioral coding and participant self-report measures. The feasibility and acceptability of the training formats (ie, VW and web-based training platforms) were evaluated in qualitative interviews with participants.

PCP participants were emailed a link to complete the web-based baseline and posttraining self-report measures 1 week and 90 days after training completion. Outcome assessment domains included sociodemographic or clinical practice characteristics (baseline only), PTSD knowledge (8 items) [14], PTSD clinical skills self-confidence, System Usability Scale (SUS) [19-21], and participant feedback on the training.

SPs were trained actors portraying 1of 6 randomly assigned cases of veterans with PTSD symptoms. Participants completed a telephone interview with an SP at baseline, posttraining, and 90-days follow-up. Interviews were coded to evaluate PCPs’ communication and PTSD-related clinical skills in the following domains: (1) shared decision-making and patient engagement, (2) PTSD symptom assessment and symptom management, and (3) MI skills. Each domain included subcategories that were coded on a 5-point global rating scale. The MI domain measured PCP partnership and empathy from the Motivational Interviewing Treatment Integrity behavioral coding system [22]. Other coding domains were developed specifically for this project (Multimedia Appendix 1).

All coders received training and attended weekly coding meetings. SP interviews were deidentified, and coders were blinded to session order (ie, baseline, posttraining, and 90 days). Roughly 20% of interviews were randomly selected for double coding to calculate interrater reliability (IRR).

Qualitative semistructured interviews were conducted with a subset of PCPs to learn more about their experiences with either training platform. Each interview was analyzed by 2 trained analysts using rapid qualitative analysis. Analysts listened to each interview and created a summary of interview content to identify themes with exemplary quotations.

Using paired (2-tailed) t tests, within-treatment group change in the proportion of correct PTSD knowledge responses were compared from baseline to posttraining and follow-up. A difference-in-differences analysis compared the mean change over time between treatment groups for knowledge or self-confidence and standardized patient coding scores. Coded items were grouped into concepts that measured the same underlying psychometric construct and standard t tests compared constructs within study arms. IRR analyses evaluated consistency between coders [23]. Participant feedback was evaluated by comparing mean responses in SUS between groups using standard t tests.

Recruitment and enrollment of PCP participants in the trial is shown in Figure 5. Of 200 eligible PCPs, 99 were randomized to the VW (intervention) training and 101 to the web-based PTSD training (control). In the VW condition, a total of 51 participants received training, and 48 did not receive training for various reasons. Specifically, 23 PCPs dropped out or had no contact before the training, 14 dropped out due to a lack of time, 4 had IT barriers, and 7 cited other reasons; Figure 5). In the web-based training, 51 participants received training, and 50 participants did not receive training. Among those who did not receive training, 43 dropped out or had no contact before the training, 2 had no contact after the videos were sent, and 5 cited other reasons for not participating. In sum, a total of 102 PCPs (51 in each arm) completed training. Characteristics of study participants are shown in Table 1.

There was a significant increase in PTSD knowledge post training among participants in both the web-based training condition (Cohen d=0.7) and the VW training condition (Cohen d=0.6). This increase was maintained at the 90-day follow-up (web-based condition Cohen d=0.3 and VW condition Cohen d=0.7). Similarly, there was a significant increase in self-confidence in PTSD clinical skills in both the web-based condition (Cohen d=1.4) and the VW condition (Cohen d=1.2) at posttraining and at the 90-day follow-up (web-based condition Cohen d=1.3 and VW condition Cohen d=1.5), with no statistically significant difference between groups at either time point (Table 2).

Participants in the web-based training reported significantly greater usability (mean 86.9, SD 17.5; Multimedia Appendix 2), compared to participants in the VW group (mean 56.8, SD 21.7), yielding a large effect size (Cohen d=1.5).

The IRR between coders was moderate for most items (range 0.4-0.7; Table 3). There was a significant within-group increase in the VW group in discussions of pharmacotherapy and psychotherapies from baseline (mean 2.9, SD 0.9) to posttraining (mean 3.4, SD 0.8; Cohen d=0.6; Table 4) and follow-up (mean 3.3, SD 1.0; Cohen d=0.4; Table 5). There were also significant within-group increases in the domains of partnership and empathy for the VW group from baseline (mean 2.9, SD 0.9) to posttraining (mean 3.3, SD=0.7; Cohen d=0.5) and follow-up (mean 3.2, SD 0.8; Cohen d=0.4), compared to the web-based group, which remained the same at baseline, posttraining, and follow-up. Between-group differences in partnership and empathy for the VW group, compared to the web-based group, approached significance (P=.08; Cohen d=0.4) from baseline to posttraining.

PCPs reported mixed perspectives on the value of the VW platform and whether this mode of delivery was worth the time required to install or set up the program and navigate using their avatars. Several participants described the VW as “clumsy” or “inefficient.” Nevertheless, participants overwhelmingly found the content of the VW training memorable and valuable. The interactive and applied components of the training distinguished it from other trainings. Because the VW training incorporated a variety of immersive audio-visual experiences in different virtual settings, participants felt the VW modality was especially strong in accommodating different learning styles. Perspectives were mixed on whether they would choose the VW format again. Some liked the interactive aspects of the VW format, while others liked the greater flexibility and reduced time commitment of the more traditional web-based option. Some participants agreed that any provider who interacts with patients with a history of trauma could benefit from this training, from PCPs to emergency department providers and specialists. Some noted that even though mental health providers are likely to have had training in PTSD, they might not have had as much training in MI and could still benefit from the interactive MI training.

Results from this pilot randomized controlled trial of a synchronous VW versus an asynchronous web-based training indicate participants in the VW condition achieved greater gains in some dimensions of MI (ie, partnership and empathy) and in their discussions of pharmacotherapy and psychotherapy treatment options with individuals with PTSD. The positive findings regarding the impact of the VW training on MI skills in this study is consistent with other trials of VW training formats to improve PCPs’ MI skills and suggest that this training platform warrants further study [24]. In this study, the web-based training was viewed as more usable compared to the VW format. Nonetheless, both methods of training were successful in increasing PCPs’ knowledge of PTSD assessment and management. These results occurred even though most PCP participants had more than 10 years of professional experience, including experience in caring for patients with trauma. This highlights the importance of ongoing and accessible training in PTSD assessment and management for PCPs.

Data from the SP interviews showed increases in each of the PTSD- and MI-related learning domains measured, many of which were sustained over time. These increases were larger among PCPs who participated in the VW compared to those in the web-based training, but the difference between groups was not significant. Notably, PCPs participating in the VW training achieved significantly higher and sustained MI scores of partnership and empathy. Thus, the VW immersive and interactive activities and practice as well as the real-time feedback may have yielded significantly higher skill levels in engaging patients through empathy and partnership and in discussions about PTSD symptom management using medication and psychotherapy.

Despite the significant increase in PCPs’ pharmacotherapy or psychotherapy discussions and MI skills in the VW training group, participants viewed the synchronized virtual platform as less usable compared to those in the web-based training condition. This gap between the efficacy of the VW platform and its ease of use highlights the need for greater efforts in improving usability, particularly in terms of navigating avatars through virtual spaces using a standard personal computer and keyboard or mouse, as opposed to video game systems designed specifically for virtual world navigation.

This study’s findings are limited due to the attrition rate (approximately 50% in each group) from randomization to training as well as the small sample size. Nevertheless, the rates of dropout among PCP participants were similar across the 2 conditions. Moreover, high rates of attrition are common in studies of frontline clinical providers [25]. This suggests that future trials should provide greater incentives, include less burdensome study assessments, and plan for potential attrition in sample size calculations.

The asynchronous web-based PTSD training was not as effective as the VW immersive experience in teaching PCPs to use MI skills in assessing and managing patients with PTSD but was viewed as more usable among PCPs. Participants in both platforms demonstrated increased PTSD-related knowledge and self-confidence in assessing and treating PTSD.

Improving PCP knowledge of trauma and PTSD and their MI communication skills is important to better engage patients in treatment and can serve as a vital gateway to specialty mental health treatment, given the high utilization rate of health care among individuals with PTSD [26]. Nonetheless, prior research indicates that PCPs require training in MI and PTSD assessment to increase PCP competence and comfort. VW training in PTSD and MI shows promise but requires moderate facilitation and technical support. As computer technology improves, immersive and interactive VW educational interventions may become more feasible and useful, particularly in teaching clinical and communication skills.

Acknowledgments

This work was supported by Department of Defense (award W81XWH-15-C-0088). We thank Adam Batten, BA, for his assistance with data analysis and the research participants for their involvement in this project.