The COVID-19 pandemic has prompted many health care providers to transition to remote or virtual simulations to comply with physical distancing guidelines. Many instructors opted to use commercial simulation software such as vSim for Nursing [1], Shadow Health [2], and Lippincott Clinical Experiences [3]. These software products have been valuable resources for health care instructors during the pandemic [4] and were appreciated by students [5]; however, flexibility is limited as these products are typically predeveloped and offer few options for customization. This creates challenges for instructors when the predeveloped scenarios do not meet the institution’s learning objectives. Some companies offer to modify their existing content or develop new scenarios according to requested specifications; however, these services often come at a high cost and are time-consuming. Many instructors who sought a more flexible and cost-effective modality used tele-simulation [6]. In tele-simulation, the instructor uses a videoconferencing platform to demonstrate a mannequin-based simulation to students remotely [7]. Tele-simulation has been shown to be beneficial for learning and well received by students [8], but the logistics of conducting a tele-simulation are difficult to orchestrate, especially during the pandemic [9]. Instructors often host the tele-simulation from a simulation facility and may need to assemble additional computer equipment to connect with students via the videoconferencing platform. As an alternative to tele-simulation, many instructors have found that simply recording their simulation videos was more feasible and cost-effective during the pandemic [10,11]. Similar to tele-simulation, simulation videos may require instructors to access simulation facilities; however, the recorded videos can be used to conduct numerous simulations without returning to the facility. A disadvantage of simulation videos is that interactivity is reduced compared with modalities such as tele-simulation. While watching videos, students’ engagement is passive, and they have minimal opportunity to collaborate or play an active role in the scenario.

In response to the need for a virtual simulation technology that is flexible, cost-effective, and interactive, we developed a software that transforms multimedia content (eg, video, images, and text) into an interactive simulation that can be conducted remotely or locally. In a previous study, we found that our interactive video simulation (IVS) software promoted higher-order learning and authenticity to a greater extent than noninteractive simulation videos when used remotely over a videoconferencing application [12]. The IVS software can also be used in the classroom as a modality that reduces physical contact and engages students in an interactive and team-oriented experience. As a continuation of our prior work, this study investigated the role of interactivity in local simulations. We asked two research questions and two exploratory questions as follows:

The participants of this study were 32 third-semester undergraduate nursing students at the UCF College of Nursing. Participants were recruited through a course required in the nursing curriculum. Student participation in the simulation scenarios was mandatory as part of the course, but completion of the surveys for the study was voluntary. The incomplete data of 9% (3/32) of participants were excluded, resulting in the inclusion of data from 91% (29/32) of participants. Of the 29 participants, 24 (83%) participants were identified as female and 5 (17%) as male. Racially and ethnically, 38% (11/29) of participants were identified as Hispanic, 34% (10/29) as White, 24% (7/29) as Asian, and 3% (1/29) as West Indian. All (100%) the participants reported previous experience with simulation: 24 (83%) participants had experience with mannequins, and 26 (90%) participants had experience with virtual simulation.

The measures evaluated in this study were technology acceptance and perceived authenticity of the simulations. Technology acceptance refers to the students’ willingness to use and adapt to a simulation technology, and authenticity refers to the extent to which a real-life encounter is accurately represented in a simulation. The survey used in this study included questionnaires derived from the Technology Acceptance Model (TAM) [16] and Virtual Patient Evaluation (VPE) [17] to measure technology acceptance and authenticity, respectively. The original TAM and VPE questionnaires are validated [16,17]; however, to make the questionnaires more suitable for this study, we modified or excluded some items. The TAM and VPE questionnaires were not validated after our modifications. The TAM questionnaire included 10 items scored on a Likert scale from lower level (1) to higher level (10). The TAM scores ranged from 10 to 100. The TAM questionnaire is presented in Textbox 1. The VPE questionnaire included 6 items scored on a Likert scale ranging from strongly disagree (1) to strongly agree (5). The VPE scores ranged from 6 to 30. Textbox 2 presents the VPE questionnaire. After the TAM and VPE questionnaires were completed, the survey included 3 open-response items to collect student feedback. The first item asked the students, “Which simulation technology did you prefer (video vs interactive video) and why?” The last 2 items asked the students, “Any comments about the simulation technology you just used?” and “Any other comments?” These data were used to quantify students’ preferences of the INT or VID simulations and to understand the factors that contributed to their preferences. All items of the survey were marked as required, except for the last 2 items, which permitted students to leave additional comments. The TAM questionnaire, VPE questionnaire, and student feedback questions were presented on different pages of the same survey. The survey was administered to participants via a QR code on Google Forms (Google LLC) [18]. The usability and technical functionality of the survey were tested before the study was conducted.

Ethics approval was granted by the UCF Institutional Review Board (ID: STUDY00002297). This study was approved with an exemption determination because it involved no or minimal risk to participants. Informed consent was obtained before students’ participation in the study. Students were informed that their deidentified survey data would be stored on a protected computer.

The students’ TAM scores ranged from 50 to 100 for the INT simulation and from 35 to 100 for the VID simulation. The mean TAM scores were 89.72 (SD 11.76) for the INT simulation and 83.38 (SD 14.89) for the VID simulation. The results were statistically significant for TAM scores of the INT and VID simulations (P<.001). The results for the TAM scores are shown in Figure 3A and Table 1. Comparisons between students’ INT and VID scores of individual TAM questionnaire items revealed statistical significance for TAM1 (P=.002), TAM3 (P=.002), and TAM9 (P=.004); these items pertained to learning, time, and clinical practice, respectively. Students’ mean TAM scores were higher for the INT simulation than for the VID simulation for all statistically significant TAM questionnaire items. The results for individual items of the TAM questionnaire are shown in Table 2.

The students’ VPE scores ranged from 19 to 30 for the INT simulation and from 8 to 30 for the VID simulation. The mean VPE scores were 27.97 (SD 3.04) for the INT simulation and 23.07 (SD 6.18) for the VID simulation. The results were statistically significant for the VPE scores of the INT and VID simulations (P<.001). The results for the VPE scores are shown in Figure 3B and Table 1. Comparisons between students’ INT and VID scores for individual VPE questionnaire items revealed statistical significance for VPE1 (P=.001), VPE2 (P<.001), VPE3 (P<.001), VPE4 (P=.003), and VPE6 (P=.005); these items pertained to decision-making, nursing care, gathering information, revising the image of the patient’s problem, and defining nursing priorities, respectively. The students’ mean VPE scores were higher for the INT simulation than for the VID simulation for all statistically significant VPE questionnaire items. The results for individual items of the VPE questionnaire are shown in Table 3.

Of the 29 students who participated in this study, 28 (97%) preferred INT simulation and 1 (3%) preferred VID simulation. The student who preferred the VID simulation did not specify a reason but mentioned that although they preferred the VID simulation, they felt that they learned more in the INT simulation. Some of the students’ comments were given in the Textbox 3.

Students’ feedback indicated that they preferred the INT simulation over the VID simulation, primarily for reasons pertaining to critical thinking, knowledge retention, engagement, and enjoyment.

The exploratory analysis evaluated students’ TAM and VPE scores of the INT simulation from the first and second studies. Study 1 refers to our previous study [12] and study 2 refers to this paper. Students’ TAM scores ranged from 15 to 100 in study 1 and from 50 to 100 in study 2. The mean TAM scores were 76.06 (SD 23.60) for study 1 and 89.72 (SD 11.76) for study 2. The results were statistically significant for the TAM scores from studies 1 and 2 (P=.007). The TAM scores from the studies are shown in Figure 4A and Table 4. Students’ VPE scores ranged from 6 to 30 in study 1 and from 19 to 30 in study 2. The mean VPE scores were 25.43 (SD 5.51) for study 1 and 27.97 (SD 3.04) for study 2. The results were statistically significant for the VPE scores from studies 1 and 2 (P=.027). The results of the VPE scores from the studies are shown in Figure 4B and Table 4.

Our results indicate that interactivity in health care simulation promotes students’ technology acceptance and perceived authenticity. Students also exhibited a strong preference for interactive simulation over noninteractive simulation.

The exploratory analysis evaluated whether remote and local simulation modalities could impact students’ technology acceptance and perceptions of authenticity. The results indicate that local simulation may increase technology acceptance and authenticity compared with remote simulation. In our analysis, data for remote simulation were collected in our previous study, which was conducted over the internet via a videoconferencing application [12], and data for local simulation were collected in this paper. In both the studies, we measured technology acceptance and authenticity using TAM and VPE questionnaires. Our previous study was limited by poor internet connection, which caused the videos to lag over the videoconferencing application. The technology acceptance results in that study were not statistically significant and therefore not reported in our previous publication [12]. Students had mentioned that the lagging videos negatively impacted their simulation experiences [12], and we suspected that the poor internet connection contributed to the insignificant results. We decided to conduct this study to re-evaluate students’ perceptions of the simulations and eliminate any factors caused by poor internet connection. This allowed us to focus on the effects of interactivity on technology acceptance and authenticity more exclusively, without the results being obscured by uncontrolled variables. Our first study supported that interactive remote simulation promotes higher-order learning and increases authenticity compared with noninteractive remote simulation. This study demonstrated that interactive local simulation may further increase technology acceptance and authenticity compared with interactive remote simulation. While remote simulation has advantages, internet connection may introduce limitations that inhibit students’ experiences, in which case local simulation conducted without internet or with a more stable internet connection may be more advisable. These results answer our exploratory research questions.

These results are reported as exploratory and not definitive because there were minor discrepancies in the study procedures. In the first study, students in the INT simulation did not watch the exemplar video and students in the VID simulation did not view the video clips. Therefore, the students’ exposure to the multimedia content was unequal between the simulations. In this study, the students’ exposure to the content was equal in the INT and VID simulations. Our motivation for modifying the procedure was to improve the experimental design; however, this modification may have influenced the results of our analysis. A separate study focusing on evaluating remote and local simulations is required to provide definitive results. Nonetheless, this exploratory analysis provides further insight into remote and local simulation technologies.

Our study was limited by 2 factors. First, interactivity in multimedia education is formally defined as direct learner-computer interaction [22]; however, participants’ interaction with the IVS software in the INT simulation was indirect. In our study, students determined the system input (selection of the button representative of the patient care step) and were the recipients of the output (display of simulation content). Ultimately, it was the facilitator that directly interacted with the system by pressing buttons on the dashboard to prompt the display of content. The facilitator acted as a mediator between the students and the simulation system, resulting in indirect learner-computer interaction. However, despite the students’ indirect interaction, the INT simulation promoted a level of interactivity far greater than the VID simulation did. In the VID simulation, the students only passively watched the simulation video and provided no input. As a result, we believe that our comparison between interactive and noninteractive simulations remains valid. Direct learner-computer interaction in the INT simulation may have strengthened our results, but many of our comparisons between the INT and VID simulations remained statistically significant despite this limitation. We are currently developing the IVS software to permit direct learner-computer interaction, and we plan to conduct future studies to further investigate the role of interactivity in health care simulation. Second, our exploratory analysis compared the results for the INT simulation from this study and our first study published in [12], although the procedures of the studies were not the same. We modified the procedure of this study to equalize students’ exposure to the multimedia content between the INT and VID simulations because it was not equal in the first study. In the first study, the INT simulation did not include the exemplar video, whereas in this study, it did. Consequently, students had greater exposure to the content in this study than in the first study. Students’ higher TAM and VPE scores in this study may have been attributed not only to the local facilitation but also to the greater exposure to content. We included the analysis in this paper because it still has value, but we call it an “exploratory” analysis owing to this limitation. To confirm the validity of these results, we would need to conduct a future study in which the local and remote simulations incorporate the same procedure.

The role of interactivity in health care simulations has been addressed in previous studies. Medical education research often differentiates between passive learning and active learning. Passive learning implies a direct transfer of knowledge from the educator to the learner with minimal involvement from the learner, whereas active learning emphasizes engagement, observation, and reflection, and knowledge is constructed by the learner rather than transferred to them in active learning [23]. The advantages of active learning in students’ cognition have been supported by ample literature [24]. One meta-analysis of 225 studies found that active learning resulted in a 6% increase in students’ exam scores and failure rates were 55% higher in traditional lectures than in active learning classes [25]. Active learning has also been shown to promote long-term knowledge retention [26] and cultivate engagement [27]. After the onset of the COVID-19 pandemic, educators have used web-based infrastructure that can further facilitate active learning, promote knowledge acquisition, and improve learner satisfaction [28].

Despite the overwhelming endorsement of active learning, some educators are reluctant to implement these methods without more evidence-based research [29]. The results of active learning studies are often generalized without thorough evaluation of significant variables, such as the intensity of active learning, teacher and student characteristics, and outcome measures [30]. One study found that active participation did not improve students’ performance in simulation compared with passive observation and suggested that the debriefing structure may be the more influential factor [31]. Despite limited knowledge of the variables affecting active learning outcomes, the multiplicity of studies advocating active learning suggests that there must be some value in these methods. Active learning research is continuing to develop, and more critical analyses will enhance our understanding of active learning and its contribution to students’ experiences.

As we increased interactivity in our study, we observed a shift toward a nontraditional simulation structure. In traditional simulation, the debriefing is conducted after the simulation. Postsimulation debriefing involves providing minimal feedback during the simulation and discussing students’ performance after the simulation has been completed. In an alternative approach called Rapid Cycle Deliberate Practice (RCDP), the debriefing is a continuous process that occurs throughout the course of the simulation. The RCDP simulation is paused at various points to allow students to reflect on their decisions, discuss their subsequent tasks, and receive feedback from the facilitator. These reflective pauses are commonly referred to as microdebriefs. Previous research has demonstrated that microdebriefing reduces the cognitive load of the simulation by breaking it into segments that are more manageable for students to comprehend [32]. Learners have also reported that reflective pauses add greater value to their simulation experience than postsimulation debriefing [33]. In this study, the INT simulation incorporated a debriefing method resembling RCDP, whereas the VID simulation incorporated the traditional postsimulation debriefing. The use of segmented and itemized multimedia content in the INT simulation permitted the students to pause, reflect, and discuss at each step of the scenario. During these pauses, students collaborated among their groups to decide their next action, and the facilitator was present to guide their discussion. However, in a typical RCDP simulation, the facilitator immediately acknowledges students’ mistakes and allows them to rethink their actions. The INT simulation differed from the RCDP simulation in that the facilitator did not provide immediate corrections unless students described actions that were inappropriate for the scenario (eg, administering contraindicated medications). In these cases, the facilitator would address the mistake and let the students reconsider their decisions. However, if students missed or incorrectly ordered some steps, the facilitator proceeded with the simulation and discussed these mistakes after completion of the simulation. Productive failure pedagogy recognizes that there is value in allowing students to commit mistakes in simulations [34]. In this pedagogy, explicit instruction is avoided to allow students to execute their mistakes in a safe environment. Students’ mistakes are then discussed between the students and facilitators in a postsimulation debriefing. Productive failure has been shown to benefit students’ learning to a greater extent than explicit instruction [34]. The IVS software generates a simulation that combines the elements of both RCDP and productive failure. Reflective discussion is guided by the facilitator after each step of the scenario; however, students are not prevented from committing and learning from their mistakes.

As the use of remote and virtual simulation technologies becomes more prevalent, the role of interactivity in students’ simulation experiences should be considered. This study demonstrated that interactivity in simulations may have advantages in terms of technology acceptance and authenticity. The interactive simulation in this study was met with greater technology acceptance and was perceived to be more authentic than the noninteractive simulation. Our exploratory analysis revealed that interactive simulation conducted locally without an internet connection may promote greater technology acceptance and perceptions of authenticity compared with remote delivery over an internet connection. Students also indicated a strong preference for interactive simulation over noninteractive simulation.