This paper is in the following e-collection/theme issue:

Virtual Reality and Augmented Reality in Medical Education (76) Training and Education in XR and Spatial Computing (77) UX and Human Factors in XR and Spatial Computing (48) e-Learning and Medical Education (691) Simulation (102) Games for Medical Education and Training (95) Extended Reality, Virtual Reality and Virtual Worlds (560) Serious Games for Education (149) Nursing Education and Training (129) Usability of Games and Gamification (70) Usability of Virtual Reality Simulations (41) Student/Learners Perceptions and Experiences with Educational Technology (73) Training Tools for Medical Students (130) New Methods and Approaches in Medical Education (232) Qualitative Methods (70) Learning and Education (76) Evaluation of Medical Education (78) Undergraduate Education for Future Healthcare Professionals (32)

Published on in Vol 10 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/56844, first published .
Virtual Reality Simulation in Undergraduate Health Care Education Programs: Usability Study

Virtual Reality Simulation in Undergraduate Health Care Education Programs: Usability Study

Virtual Reality Simulation in Undergraduate Health Care Education Programs: Usability Study

Authors of this article:

Gry Mørk1 Author Orcid Image ;   Tore Bonsaksen1, 2 Author Orcid Image ;   Ole Sønnik Larsen1 Author Orcid Image ;   Hans Martin Kunnikoff1 Author Orcid Image ;   Silje Stangeland Lie1 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Original Paper

    1Department of Health, Faculty of Health Sciences, VID Specialized University, Stavanger, Norway

    2Department of Health and Nursing Sciences, Faculty of Social and Health Sciences, Inland Norway University of Applied Sciences, Elverum, Norway

    Corresponding Author:

    Gry Mørk, MSc

    Department of Health

    Faculty of Health Sciences

    VID Specialized University

    Misjonsmarka 12

    Stavanger, 4024

    Norway

    Phone: 47 47234324

    Email: gry.mork@vid.no


    Background: Virtual reality (VR) is increasingly being used in higher education for clinical skills training and role-playing among health care students. Using 360° videos in VR headsets, followed by peer debrief and group discussions, may strengthen students’ social and emotional learning.

    Objective: This study aimed to explore student-perceived usability of VR simulation in three health care education programs in Norway.

    Methods: Students from one university participated in a VR simulation program. Of these, students in social education (n=74), nursing (n=45), and occupational therapy (n=27) completed a questionnaire asking about their perceptions of the usability of the VR simulation and the related learning activities. Differences between groups of students were examined with Pearson chi-square tests and with 1-way ANOVA. Qualitative content analysis was used to analyze data from open-ended questions.

    Results: The nursing students were most satisfied with the usability of the VR simulation, while the occupational therapy students were least satisfied. The nursing students had more often prior experience from using VR technology (60%), while occupational therapy students less often had prior experience (37%). Nevertheless, high mean scores indicated that the students experienced the VR simulation and the related learning activities as very useful. The results also showed that by using realistic scenarios in VR simulation, health care students can be prepared for complex clinical situations in a safe environment. Also, group debriefing sessions are a vital part of the learning process that enhance active involvement with peers.

    Conclusions: VR simulation has promise and potential as a pedagogical tool in health care education, especially for training soft skills relevant for clinical practice, such as communication, decision-making, time management, and critical thinking.

    JMIR Med Educ 2024;10:e56844

    doi:10.2196/56844

    Keywords

    360° videos; health professions education; virtual reality; usability study; undergraduates; university; students; simulation 


    Background

    Virtual reality (VR)–based training has a generally high acceptance amongst trainees, regardless of the technology limitations, usability challenges, and cybersickness [1]. A systematic review of the effectiveness of VR-based simulation training from the past 30 years showed evidence for VR as useful for training cognitive skills, such as spatial memory, learning and remembering procedures, and psychomotor skills. VR was also found to be a good alternative where regular job training was either impossible or unsafe to implement [1].

    Targeted training of health care students is paramount to ensuring the provision of patient-centered care and effective communication with patients and their families. Using 360° videos in VR headsets as a starting point for VR simulation has immense potential for the systematic design of learning experiences and for fostering social and emotional learning through collaborative interactions with students [2-4]. Social and emotional learning concerns the development of emotional intelligence skills, including self-awareness, self-management, social awareness, relationship skills, and responsible decision-making [5]. Further, VR simulation offers an active and engaging approach to learning, which can positively impact the motivation of both teachers and students while enhancing learning opportunities and creativity in the learning process [4,6]. Videos of 360° are prerecorded using an omnidirectional camera that films in every direction. This means that the viewer can look around freely as in other VR experiences, but movement and interaction are limited because the camera only records from one position at a time [4].

    To ensure active learning, the vicarious VR learning experiences should be provided with an instructional component of debriefing and peer discussion [7]. According to Biggs et al [8], education is about conceptual change, not just the acquisition of information. Such conceptual change takes place when students work collaboratively and in dialogue with others, both peers and teachers. Good dialogue is an essential element of activities that shape, elaborate, and deepen understanding [8].

    VR simulation has the great advantage of facilitating a rich, detailed learning environment through different scenarios that are difficult to replicate in reality (eg, frightening situations) [9]. Considering that most health professions are underpinned by a client-centered philosophy of practice, the use of VR will be more relevant when educators create content that is particularly aligned with each specific education program [10]. Consideration of design issues and student experiences of usability are central for developing VR technologies in medical education programs [11]. According to Nielsen [12], usability is defined by five quality components: (1) how easy it is to accomplish the task, (2) efficiency (when learned), (3) memorability, (4) number of user errors, and (5) satisfaction related to the design. Usability has also been extended to comprise user experience that has a more compelling emotional and motivational effect [13]. We were interested in exploring the usability of a VR simulation program designed for health care students, in particular with a view to perceived learning outcomes and possible enhancement of soft skills such as communication skills and clinical decision making.

    Even though recent studies indicate that VR is increasingly being used in higher education for clinical skills training and role-playing among health care students [6,14], there is a research gap concerning the use of VR to facilitate the nontechnical skills inherent in the social and emotional competences [5], such as communication, decision-making, and ethical reflection [15-18]. There is a need for more research on VR simulation in higher education programs within health care [4,19,20]. Further research needs to be directed toward the application of immersive 360° videos and how it is experienced by students in higher education [4,19,20] and studies that explore the usability of VR simulation [2,19].

    A recent review conducted on implementation of VR argued that future studies should explore viable approaches to incorporate VR into health professions education [21]. During the last 2 years, there have been published a few qualitative [22-24] as well as quantitative studies on the usability of VR in nursing education [4]. Literature searches in Google Scholar using the search words *usability AND *virtual reality AND *occupational therapy/social education, published after 2022, revealed 1 mixed methods study on the usability in physical therapy education [25]. This search did not identify any existing publications on usability of VR simulation in occupational therapy or social education programs.

    The recommendations from earlier research as well as the scarcity of usability studies concerning VR in health education programs suggest that VR implementation processes and the usability of VR simulation programs should be further explored and evaluated.

    Study Aim

    This study aimed to explore student-perceived usability of VR simulation in 3 health care education programs in Norway. The research question was: What are the students’ perceptions of the usability of the employed VR simulation and related learning activities, and are there differences in perceptions between students in different study programs?


    Design and Study Context

    This study is part of a larger interdisciplinary project in a Norwegian university, the Solstien 3 project, with the objective to create a VR simulation that portrays situations future health and social workers might encounter in their professional practice. The purpose was to offer students a safe and controlled environment to practice handling challenging and unexpected scenarios without the risk of compromising the well-being of clients, patients, or themselves. The VR simulation, developed and used for soft skills training, was intended to supplement teaching and field placements [26].

    The project has developed a “virtual learning center” accessible on a webpage, featuring portrayals of service recipients (patients or clients) through text and images, along with 360° videos illustrating various scenarios [27]. The design of the VR simulation included scenarios and related discussion tasks based on shared learning outcomes for higher education in Norway [28]. The VR simulation in all education programs commenced with a briefing, followed by students watching the scenario using VR headsets, without any possibility to interact with the actors or the environment. The students subsequently participated in facilitated debrief discussions in groups or conducted planned assignments in groups for peer learning. The debrief was concerned with the ethical issues related to the scenario, as well as communication and interpersonal skills. The VR simulation was conducted as part of the regular curriculum for undergraduate students enrolled in the involved education programs; occupational therapy, social education, and nursing. After graduation, those who have undergone these education programs with satisfactory results are qualified to work as health care personnel in Norway, provided authorization from The Norwegian Directorate of Health.

    In the early development phase of the Solstien 3 project, both faculty members’ and students’ experience with the prototype 360° video was investigated. The findings showed the importance of VR being contextualized directly in educational programs to create a safe environment for learning [26]. Further, in a pilot study, the students’ experience of the VR simulation was explored. Findings showed that students experienced VR simulation as valuable as a space for authentic, engaging, and reflective practice, and therefore the students felt prepared for professional practice [29]. The use of 360° videos in combination with group discussions activates social and emotional learning and appears to be promising for enhancing professional learning [30]. The process and content of the Solstien 3 project are described in detail and are accessible through the aforementioned webpage [27].

    Participants

    Students from 3 undergraduate health care education programs at the same university were invited to participate in this study. These students, from occupational therapy, social education, and nursing, had VR simulation scenarios that were directly associated with their professional practice (developed by the Solstien 3 project). The VR simulation was therefore carried out in different subject-specific courses in different semesters for these 3 education programs. The data were collected during the academic year of 2022-2023, specifically between October 2022 and May 2023. An overview of the VR simulation and the related learning activities in the 3 education programs are displayed in Table 1, and a description of the scenarios and the students’ tasks is displayed in Multimedia Appendix 1. The social education students were in their second semester in their first year of study, while the occupational therapy students were in their fourth semester in their second year of study. The nursing students were in their fifth and sixth semesters in their third year of study.

    Table 1. Overview of virtual reality (VR) simulation learning activities in 3 different education programs.

    		Occupational therapySocial educationNursing
    Semester and name of subjectSecond year, fourth semester
    Participation and belonging
    (15 ECTSa)    		First year, second semester
    Social education work processes
    (10 ECTS)    		Third year, fifth and sixth semesters
    Home nursing care
    (15 ECTS)
    Students in class38 students95 students280 students
    IntroductionThe simulation started with some initial information in plenum. The students were given a short instruction about the task and shown how to use and adjust the VR headsetsThe simulation started with some initial information in plenum. The students were given a short instruction about the task and shown how to use and adjust the VR headsetsThe simulation started with some initial information in plenum. The students were given a short instruction about the task and shown how to use and adjust the VR headsets
    Group sizesVR simulation: 12-15 students
    Discussion groups: 6-7 students    		VR simulation: 14-15 students
    Discussion groups: 6-8 students    		VR simulation: 8-9 students
    Discussion groups: 8-9 students
    Organization of the learning activityThe class was divided into 3 VR simulation groups and 6 peer discussion groups. The students spent approximately 3 hours on the whole session.The class was structured into 3 cohorts for teaching on different days. Each cohort was divided into 2 VR simulation groups and 5 peer discussion groups. The students spent between 2 and 4 hours on the whole session.The class was structured into 3 cohorts for teaching on different days. The cohort was divided into approximately 11 VR simulation and peer discussion groups. The students spent 1 hour on the whole session (as a part of a full-day health care simulation program).
    Peer debriefAfter watching the 360° videos, students were organized into peer discussion groups of 6-7 students. Each group had 1 facilitator.After watching the 360° videos, students were organized into peer discussion groups of 6-8 students. Only 1 facilitator circulated between the groups.After watching the 360° videos, students stayed in the same peer groups for discussion and debrief. Each group had 1 facilitator.
    Learning outcomes The debrief is concerned with the ethical issues that arise in the situation, as well as communication and interpersonal skills and a discussion on fall prevention.The students collaborated to discuss and create a written assignment based on their experiences in the situation. The task involved the identification of possible conflicts of values and target behaviors and a discussion on the choice of appropriate mapping tools.The debrief is concerned with the ethical issues that arise in the situation and the relationship and communication between the nurse and the patient as well as the family.

    aECTS: European Credit Transfer and Accumulation System.

    Measurement

    To find the most suitable questionnaire for this study, research work with similar objectives was examined. We found 2 relevant studies related to the usability of VR simulation in nursing education [31,32]. Verkuyl et al [31] developed a questionnaire based on earlier usability testing in the same study context. This questionnaire of 18 items had 2 sections: perceived easeof use and perceived usefulness. The questionnaire was relevant to our study as items considered whether the students found the VR simulation easy to use, as well as whether the learning activity would enhance soft skills such as communication skills and clinical decision-making [31]. The questionnaire of Lee et al [32] was a further refinement of the work of Verkuyl et al [31] and consisted of 17 items to be rated and 7 open-ended questions. Based on these 2 studies, a questionnaire was developed by the research group, containing 17 items to be rated and 2 open-ended questions. The questionnaire also included 6 items assessing student characteristics: study program affiliation (social education, occupational therapy, and nursing), gender (male vs female), age (years), current working status in health care or social services (yes vs no), prior experience using VR technology (yes vs no), and attitude toward VR technology in education (sceptical or indifferent vs positive).

    In total, 17 statements related to the students’ perceptions of the usability of VR simulation and the related learning activities were used. Of the total items, 9 concerned perceived ease of use and 8 items concerned perceived usefulness. The participants were instructed to rate their level of agreement (1=disagree, 2=disagree somewhat, 3=unsure, 4=agree somewhat, and 5=agree). Toward the end of the questionnaire, the participants were also asked to evaluate the VR simulation activity on a 1-5 scale (1=poorest evaluation and 5=best evaluation).

    The questionnaire also had 2 open-ended questions: What specifically did you learn from the learning activity (360° video, group discussions, etc.) and Do you have other comments about the 360° videosand the related learning activities? (Improvements? Other ideas?)

    The translation process was first undertaken by one of the research group members before the resulting translations were discussed and adjusted by the team. The students were invited to respond to a digital questionnaire in their native language (Norwegian) shortly after completing the learning activity.

    Data Analysis

    Differences between students enrolled in different education programs were examined with Pearson chi-square tests for categorical variables, and with 1-way ANOVA for continuous variables. To adjust for inflating type I error rates in the multiple comparisons, Tukey honestly significant difference (HSD) correction was applied to the analysis of variance. To analyze the data from the open-ended questions, we used qualitative content analysis [33]. Initially, the responses were thoroughly read to gain a broad understanding, followed by a systematic categorization into distinct codes. This process helped in identifying similarities and differences in the responses. Subsequently, these codes were grouped into 2 relevant categories, which facilitated the identification of recurring patterns focusing on students' experiences of the usability aspect of the VR simulation.

    Ethical Considerations

    This study is registered with the Norwegian Centre for Research Data (protocol code 423788). Research ethics were strictly adhered to throughout the study. The learning activity was conducted as a mandatory activity as part of the curriculum for all students in the relevant study programs (n=413). After having conducted the learning activity, the students were invited to complete the questionnaire, which was provided by a QR code in the classroom or a link sent by email. On the first page of the questionnaire, the students received information about the study's purpose and procedures, and that participation was voluntary. The students were also informed that they consented to participate in the study by pressing “go to questionnaire”. In this way, informed consent was obtained from all participants before their involvement. The questionnaire was anonymous, and data cannot be traced back to individual informants. Moreover, the questionnaire did not ask for any sensitive information.


    Student Characteristics

    Overall, 146 students responded to the questionnaire out of a total of 413 invited students (35.4% response rate), representing students of social education (n=74, response rate 77.9%), occupational therapy (n=27, response rate 71.1%), and nursing (n=45, response rate 16.1%). The social education students comprised 50.7% of the sample, while the nursing students comprised 30.8% and the occupational therapy students 18.5% of the sample. Most of the students were women (n=117, 80.1%), while 29 (19.9%) were men, with a larger proportion of younger students in the social education program and a larger proportion of older students in the nursing education program. The sample is described in Table 2.

    At the time of the data collection, 105 (71.9%) students worked in health care or social services, and half of the students (n=73) had prior experience using VR technology. Statistically significant differences between the groups were found in the “attitude toward VR in education” and the “evaluation of the VR simulation learning activity.” A larger proportion of nursing students were positive toward VR technology in their education program (n=42, 93.3%), compared with occupational therapy students (n=21, 77.8%) and social education students (n=56, 75.7%). Among the nursing students, 26 (72.2%) gave the highest possible rating (5) when evaluating the VR simulation learning activity. In contrast, this rating was given by 7 (33.3%) of the occupational therapy students and 20 (37%) of the social education students.

    Table 2. Characteristics of students by study program (N=146)a.
    CharacteristicsAll, NOccupational therapy, n (%)Social education, n (%)Nursing, n (%)P valueb
    Age (years).009

    		18-20 172 (7.4)15 (20.3)0 (0)

    		21-25 7816 (59.3)40 (54.1)22 (48.9)

    		26-30 327 (25.9)11 (14.9)14 (31.1)

    		31+ 192 (7.4)8 (10.8)9 (20)
    Gender.75

    		Male294 (14.8)16 (21.6)9 (20)

    		Female11723 (85.2)58 (78.4)36 (80)
    Currently working in health care or social services.07

    		Yes10517 (63)50 (67.6)38 (84.4)

    		No4110 (37)24 (32.4)7 (15.6)
    Prior experience from using VRctechnology.16

    		yes7310 (37)36 (48.6)27 (60)

    		No7317 (63)38 (51.4)18 (40)
    Attitude toward VR technology in education.048

    		Positive11921 (77.8)56 (75.7)42 (93.3)

    		Skeptical or indifferent276 (22.2)18 (24.3)3 (6.7)
    Evaluation of the VR simulation learning activityd.002

    		1-253 (14.3)2 (3.7)0 (0)

    		3-45311 (52.4)32 (59.3)10 (27.8)

    		5537 (33.3)20 (37)26 (72.2)

    aN=146 unless otherwise stated.

    bP-values are based on Pearson chi-square test.

    cVR: virtual reality.

    dn=111, and higher ratings indicate more positive attitudes.

    Perceived Usability of the VR Simulation and Related Learning Activities

    In the pairwise comparisons between study programs, using the occupational therapy program as the reference group, the nursing students had the highest mean scores, while the occupational therapy students, for the most part, had the lowest mean scores.

    Students’ Perceptions of Ease of Use

    In the pairwise comparisons, statistically significant differences were found on all items related to the perceived ease of use. The highest mean scores for all students were on the items “the pace and the narrative in the 360° video were good” (mean 4.71, SD 0.64) and “the presented situation was realistic” (mean 4.68, SD 0.64). The lowest mean score was found on the item “the audio quality of the 360° video was good” (mean 3.86, SD 1.41), which was also the lowest mean score for both the occupational therapy students (mean 2.81, SD 1.57) as well as the social education students (mean 3.72, SD 1.41) throughout the whole questionnaire.

    Students’ Perceptions of Usefulness

    In the pairwise comparisons, statistically significant differences were found on all items related to the perceived usefulness. The highest mean scores for all students were on the item “the 360° video and the related learning activities were useful as a part of my education program” (mean 4.46, SD 0.95). The lowest mean score was found on the item “what I experienced in the 360° video improved my professional competence” (mean 3.93, SD 1.19). The difference between the study programs was rather large, with the occupational therapy students scoring this item on average 1.2 points lower than the nursing students and 0.8 points lower than the social education students. Table 3 displays the total mean scores and the pairwise comparisons of mean scores between study programs.

    Table 3. Usability of VRa simulation and learning activities, N=140-146.
    StatementsTotal, mean (SD)Occupational therapy, mean (SD)Social education, mean (SD)Pb valueNursing, mean (SD)Pb value
    Ease of use

    		The VRa headset and the showtime app were easy to use4.45 (0.97)3.67 (1.36)4.57 (0.83)<.0014.73 (0.62)<.001

    		I received sufficient information and instructions before using the equipment4.65 (0.81)4.07 (1.24)4.77 (0.66)<.0014.82 (0.50)<.001

    		It was easy to know what to do4.43 (0.90)3.89 (1.01)4.49 (0.89).0074.68 (0.71)<.001

    		I did not have any technical problems4.01 (1.38)3.07 (1.47)4.25 (1.31)<.0014.18 (1.21).002

    		The presented situation was realistic4.68 (0.64)3.89 (0.89)4.86 (0.42)<.0014.87 (0.34)<.001

    		The pace and the narrative in the 360° video were good
    		4.00 (1.00)4.86 (0.39)<.0014.89 (0.32)<.001

    		The visual quality of the 360° video was good4.50 (0.96)3.85 (1.57)4.64 (0.72)<.0014.67 (0.67).001

    		The audio quality of the 360° video was good3.86 (1.41)2.81 (1.57)3.72 (1.41).0044.71 (0.59)<.001

    		I would like to see more scenarios together with related learning activities4.55 (0.97)4.19 (1.27)4.51 (1.00).284.82 (0.58).02
    Usefulness

    		What I experienced in the 360° video was useful for my future professional practice4.43 (0.99)4.00 (1.18)4.45 (0.97).114.67 (0.82).02

    		What I experienced in the 360° video was useful in improving my communication skills3.95 (1.34)3.50 (1.53)3.80 (1.29).574.45 (1.15).01

    		What I experienced in the 360° video improved my professional competence3.93 (1.19)3.15 (1.43)3.94 (1.10).0064.39 (0.92)<.001

    		This 360° video was useful as part of my education program4.38 (1.03)3.70 (1.38)4.40 (0.93).0054.75 (0.69)<.001

    		The learning activities were useful for my future professional practice4.37 (1.01)3.56 (1.34)4.42 (0.91)<.0014.78 (0.60)<.001

    		The learning activities were useful for improving my communication skills4.17 (1.12)3.62 (1.33)4.08 (1.13).144.62 (0.75)<.001

    		The learning activities improved my professional competence4.15 (1.08)3.13 (1.26)4.26 (1.00)<.0014.53 (0.67)<.001

    		The 360° video and the related learning activities were useful as a part of my education program4.46 (0.95)3.85 (1.29)4.51 (0.88).0044.75 (0.62)<.001

    aVR: virtual reality.

    bP-values are based on pairwise comparisons between study programs, with 1-way analyses of variance using the Tukey HSD correction. The occupational therapy program is used as reference group.

    Open-Ended Questions

    Out of 146 participants, 110 answered one or both open questions. There were a total of 135 qualitative responses, which were analyzed. Two categories were created based on the responses, focusing on “perceived learning” and “difficulties and recommendations.”

    Perceived Learning

    The participants described learning outcomes related to soft skills such as conflict management, reflection, emotion regulation, and non-verbal and verbal skills. Furthermore, they also emphasized professional conduct such as empathy, client-centered practice, and a focus on family involvement. The participants found the following debrief and group discussion particularly useful in their learning experience. One participant expressed it as follows:

    With this type of learning, you are better prepared for clinical practice and the learning curve is steep when we reflect together. It was realistic and touched a lot of emotions. Solutions were discussed together. I think this type of learning is extremely beneficial.

    The fact that this learning took place in a safe environment was also highlighted as positive by several of the participants. Several of the participants answered the open-ended questions with few words in a general, but positive, manner. For example, the learning activity was cool, exciting, fun, realistic, and relevant for clinical practice. One participant expressed it as follows:

    Being a fly on the wall in such a situation was very enlightening.

    Some participants also answered that this learning activity was a new type of learning and that the website was particularly useful for getting to know the material more thoroughly.

    Difficulties and Recommendations

    The participants' experience of difficulties related to the learning activity mostly concerned either technical or organizational problems. Particularly poor sound quality and the need for noise-reducing headsets were highlighted as technical issues. On the other hand, organizational problems revolved around too many students in the groups and that the preparations could have been more informative, especially for the students with no experience with VR simulation. Two participants reported discomfort related to a rather unpleasant scenario that they found them “in the middle of,” and one of them wrote:

    …for me, who is a little light sensitive etc., I had great difficulty having the screen so close to my face. The experience was simply quite unpleasant and triggered a headache.

    The participants recommended other types of cases and scenarios involving complex choices and interactive roles in the VR simulation, such as:

    …options where you can “decide” the actions further.

    It was also desirable that the 360° videos would be more available for self-training and that the students could access more VR simulations at their educational program.


    Overview

    This study aimed to explore student-perceived usability of VR simulation, including 360° videos in VR headsets and related learning activities, in 3 health care education programs in Norway. All in all, high mean scores indicate that the students experienced the VR simulation and the related learning activities as very useful. Nevertheless, our findings showed that the nursing students were most satisfied with the usability of the VR simulation, while the occupational therapy students were least satisfied. The qualitative data from the 2 open-ended questions confirmed the findings from the questionnaire, but also illuminated other factors regarding the usability of VR simulation and the related learning activities.

    Differences Between the Study Programs

    The nursing students in this study were third-year students with previous experience with other simulations, practical skill training, and clinical practice. They had considerably more experience than the first-year social education students. The second-year occupational therapy students had some prior experience with skill training and clinical practice, but not with simulation. Saab et al [22] emphasized in particular that in order to succeed, the key stakeholders, including students and educators, need to be trained in the use of VR prior to implementing VR simulation. The differences in VR experience among the students prior to this learning session may contribute to explaining why the nursing students scored the usability of the VR simulation significantly higher than students in the other 2 educational programs. They may be due to the novelty effect, where novice VR users tend to focus on managing the hand controllers and exploring the VR environment, rather than focusing on the proposed learning tasks [34].

    The key findings in the systematic review of Woon et al [3] illuminate that VR increases nursing students’ engagement in the learning experience when allowing them to participate in activities that are close to reality and when implementing an effective training regime consisting of short interval training (≤30 min each) for a number of sessions. In our study, the nursing students had this learning activity as part of a larger session with a multiple number of short interval trainings. The larger number of trainings for the nursing students and their embedding the VR simulation into a more comprehensive learning activity may contribute to explaining their higher scores on usability and relevance. Also, the nursing program had the smallest groups in the VR simulation (up to 9 students), while both the social education program and the occupational therapy program reported difficulties with instruction and initiation of VR simulation in somewhat larger student groups (up to 15 students). The occupational therapy program experienced technical problems, particularly with sound control, which may also be reflected in the low mean scores from the survey. Educators at the social education program experienced difficulties with one of the cohorts, due to students having little to no prior experience using VR technology. Noble et al [35] emphasized that one should not take students’ knowledge and skills in using VR equipment for granted. However, their study mirrors our results, with approximately half of the students having no experience with VR even though the education program facilitates the use of VR equipment free of charge.

    Group differences related to study progression, previous experience with VR simulation, as well as how the learning activity was structured, both according to group size and how many facilitators were available for the students, can partly explain why students scored the usability differently. A valuable recommendation, consistent with previous research [36], is that smaller student cohorts and having a sufficient number of staff available are important so that the students can receive sufficient help at the right time.

    Practicing Clinical Skills With Realistic Scenarios in a Safe Learning Environment

    Our findings in both qualitative and quantitative data showed that the students perceived the scenarios to be realistic and useful for clinical practice. Also, several students shared in the open-ended responses that they experienced the VR learning activity as a safe way to practice their clinical skills. The sober and realistic nature of the scenarios depicting real-world situations appears to be beneficial for students’ skill acquisitions with relevance for clinical practice [14,37-39]. In addition, the open-ended responses indicate that students were emotionally touched and had an increasing engagement with this learning activity. This is beneficial, as increased emotions and engagement in VR simulation can enhance the learning processes [9]. In the development of the prototype for the 360° videos, students experienced the VR simulation to be genuine and realistic due to actors using direct eye contact with the camera. Furthermore, the importance of having a safe learning environment was highlighted, especially when experiencing strong emotions [26]. Although the learning environment was described as safe by many students, a few of them pointed out that they found the VR simulation uncomfortable. Cybersickness can influence the learners’ attitude toward the technology negatively and has been correlated with poorer learning outcomes [40]. For these students in particular, it will be crucial to facilitate the VR simulation to avoid discomfort. To reduce the discomfort among individuals experiencing it, one suggestion is to adapt the VR simulation to a standard desktop [36]. Nevertheless, we have reason to believe that VR simulation with realistic scenarios is an appropriate learning activity to precede clinical practice. Furthermore, this type of teaching has a particular value as students can practice skills needed in a difficult situation but still feel safe in a peer learning process.

    Student Active Learning in Peer Debrief and Group Discussions

    Our findings based on both qualitative and quantitative data showed that the students perceived the related learning activities useful for their future professional practice. Many students highlighted the group discussions in the debriefing sessions as particularly beneficial, which is consistent with previous research [7,17,26]. These methods are known to enhance learning by allowing students to actively process information, engage in critical thinking, and articulate their understanding or questions [41,42]. Debrief sessions, in particular, offer a reflective space where students can consolidate their learning, address misconceptions, and gain insights from peers and instructors. This discussion contributes to the larger conversation about active learning in education. Active learning, characterized by student participation and engagement, has been increasingly recognized for its effectiveness in promoting deeper understanding and retention of knowledge [43]. However, some of the students remarked that they were passive observers of the 360 video. They suggested including interactivity in the VR simulation for a better learning experience. This aligns with a Canadian study underscoring the benefits of active participation in VR simulation for occupational therapy students [10]. The learning activity in our study still required active involvement in the debrief discussion where students reflected and engaged with the teaching material together with their peers. A suitable alternative could have been active participation in the VR environment, where students would have the opportunity to interact with the virtual environment, make decisions, and influence the outcome of the scenarios. This could lead to a more engaging learning experience, probably more appropriate for training soft skills such as decision-making, time management, and critical thinking.

    Our findings have important implications for educational practices. They suggest that incorporating structured group discussions and debriefing sessions can significantly enhance the VR learning experience by allowing students to engage with and process the learning material related to situations they might encounter in a real-world setting. In addition, VR simulations with the student in an active role will probably increase students' motivation and promote learning.

    Study Limitations and Suggestions for Future Research

    The study is based on self-report data only. The response rates among both occupational therapy and social education students were high with 71% (n=27) and 78% (n=74). In contrast, the response rate among the nursing students was only 16% (n=45), and, therefore, the findings must be interpreted with caution. Further to this, there may be a response bias, especially among the nursing students, a group possibly consisting of particularly motivated and high-achieving respondents. This may be one important reason why students in this group were more satisfied with the usability of this learning activity. Differences between the student groups may also have been caused by varying levels of VR experience and competence among the faculty facilitating the relevant sessions. While we have the impression that the faculty in the nursing program had more experience with VR compared to their counterparts in the other education programs, we did not collect data from faculty that could support or contradict this view. Thus, we are unable to assess the possible impact of faculty’s VR experience and competence on the students’ experience, and we suggest that future studies include this information.

    Other methodological issues concern that the VR simulation and learning activity was carried out at different year levels among the 3 health care education programs and the presence of age differences between students in the different study programs. Higher age and maturity among the participating nursing students may have resulted in higher levels of satisfaction, while these results may not be representative of differences in the study population. We also note that there might be different “group cultures” related to the evaluation of teaching in different groups of students, essentially meaning that similar experiences are systematically given dissimilar ratings across groups. Thus, there is a possibility that the students’ expressed levels of satisfaction differed more than their experienced levels of satisfaction.

    Conclusions

    In summary, the students in all 3 educational programs perceived the high usability of the VR simulation and the related learning activities. Nevertheless, the nursing students were most satisfied, while the occupational therapy students were somewhat less satisfied. By using realistic scenarios, health care students can be prepared for complex clinical situations in a safe environment. Group discussions in the debriefing sessions are a valuable part of the learning process and enhance active involvement with peers. Particular attention should be given to the organization of the VR simulation with small student groups and a sufficient number of involved educators so that students receive guidance when problems occur. These findings contribute to the overall growing evidence showing that VR simulation has promise and potential as a pedagogical tool in health care education, perhaps particularly for training soft skills such as communication, decision-making, time management, and critical thinking—skills that are unquestionably relevant for clinical practice. However, future studies need to expand beyond the pilot phase to study if VR simulation exceeds the value of traditional teaching methods.

    Acknowledgments

    The authors thank the students who volunteered to take part in this study. In addition, we thank our colleagues for conducting the VR simulation and their contribution to the data collection for this study. The larger project was funded by the Norwegian Directorate for Higher Education and Skills and VID Specialized University (project AKTIV-2019/10162).

    Data Availability

    The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

    Conflicts of Interest

    None declared.

    Authors' Contributions

    GM, TB, and SSL developed the study design. All authors contributed to collecting the data. GM, TB, and SSL conducted the data analyses, but all the authors interpreted the data in light of the learning activities. GM wrote the first manuscript draft. All authors contributed to the editing of the manuscript. Lastly, all authors read and approved the final manuscript.

    Multimedia Appendix 1

    Descriptions of scenarios and tasks.

    PDF File (Adobe PDF File), 159 KB
    1. Renganayagalu SK, Mallam SC, Nazir S. Effectiveness of VR head mounted displays in professional training: a systematic review. Tech Know Learn. Jan 01, 2021;26(4):999-1041. [CrossRef]
    2. Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I. A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comput Educ. Apr 2020;147:103778. [CrossRef]
    3. Woon APN, Mok WQ, Chieng YJS, Zhang HM, Ramos P, Mustadi HB, et al. Effectiveness of virtual reality training in improving knowledge among nursing students: a systematic review, meta-analysis and meta-regression. Nurse Educ Today. Mar 2021;98:104655. [CrossRef] [Medline]
    4. Haugan S, Kværnø E, Sandaker J, Hustad JL, Thordarson GO. Playful learning with VR-SIMI model: the use of 360-video as a learning tool for nursing students in a psychiatric simulation setting. In: Akselbo I, Aune I, editors. How Can We Use Simulation to Improve Competencies in Nursing? Switzerland. Springer, Cham; 2023:103-116.
    5. Conley CS. SEL in higher education. In: Durlak JAD, Weissberg CE, Gullotta RP, Thomas P, editors. Handbook of Social and Emotional Learning: Research and Practice. New York, USA. Guilford Publications; 2015:197-212.
    6. Choi J, Thompson CE, Choi J, Waddill CB, Choi S. Effectiveness of immersive virtual reality in nursing education: systematic review. Nurse Educ. 2022;47(3):E57-E61. [CrossRef] [Medline]
    7. Luo H, Yang T, Kwon S, Li G, Zuo M, Choi I. Performing versus observing: investigating the effectiveness of group debriefing in a VR-based safety education program. Comput Educ. Dec 2021;175:104316. [CrossRef]
    8. Biggs J, Tang C, Kennedy G. Teaching for Quality Learning at University. 5th ed. Maidenhead. Open University Press; 2022.
    9. Allcoat D, von Mühlenen A. Learning in virtual reality: effects on performance, emotion and engagement. Res Learn Technol. 2018;26:2140. [CrossRef]
    10. Kim J, Nowrouzi-Kia B, Ho ES, Thomson H, Duncan A. Appraising occupational therapy students' perceptions of virtual reality as a pedagogical innovation. Comput Educ: X Real. 2023;3:100039. [CrossRef]
    11. Atas G, Topalli D, Cagiltay N. A systematic review of virtual reality and user experience in medicine. 2023. Presented at: 15th International Conference on Education and New Learning Technologies; July 3-5, 2023; Palma, Spain.
    12. Nielsen J. Usability 101: introduction to usability. 2012. URL: https://www.nngroup.com/articles/usability-101-introduction -to-usability/ [accessed 2024-07-04]
    13. Lewis JR. Usability: lessons learned … and yet to be learned. Int J Hum-Comput Interact. 2014;30(9):663-684. [CrossRef]
    14. Beverly E, Rigot B, Love C, Love M. Perspectives of 360-degree cinematic virtual reality: interview study among health care professionals. JMIR Med Educ. 2022;8(2):e32657. [FREE Full text] [CrossRef] [Medline]
    15. Plotzky C, Lindwedel U, Sorber M, Loessl B, König P, Kunze C, et al. Virtual reality simulations in nurse education: a systematic mapping review. Nurse Educ Today. 2021;101:104868. [CrossRef] [Medline]
    16. Bracq MS, Michinov E, Jannin P. Virtual reality simulation in nontechnical skills training for healthcare professionals: a systematic review. Simul Healthc. 2019;14(3):188-194. [FREE Full text] [CrossRef] [Medline]
    17. Coyne E, Calleja P, Forster E, Lin F. A review of virtual-simulation for assessing healthcare students' clinical competency. Nurse Educ Today. 2021;96:104623. [CrossRef] [Medline]
    18. Jiang H, Vimalesvaran S, Wang JK, Lim KB, Mogali SR, Car LT. Virtual reality in medical students' education: scoping review. JMIR Med Educ. 2022;8(1):e34860. [FREE Full text] [CrossRef] [Medline]
    19. Blair C, Walsh C, Best P. Immersive 360° videos in health and social care education: a scoping review. BMC Med Educ. 2021;21(1):590. [FREE Full text] [CrossRef] [Medline]
    20. Cypress BS, Caboral-Stevens M. "Sense of presence" in immersive virtual reality environment: an evolutionary concept analysis. Dimens Crit Care Nurs. 2022;41(5):235-245. [CrossRef] [Medline]
    21. Lie SS, Helle N, Sletteland NV, Vikman MD, Bonsaksen T. Implementation of virtual reality in health professions education: scoping review. JMIR Med Educ. 2023;9:e41589. [FREE Full text] [CrossRef] [Medline]
    22. Saab MM, McCarthy M, O'Mahony B, Cooke E, Hegarty J, Murphy D, et al. Virtual reality simulation in nursing and midwifery education: a usability study. Comput Inform Nurs. 2023;41(10):815-824. [FREE Full text] [CrossRef] [Medline]
    23. Andreasen EM, Høigaard R, Berg H, Steinsbekk A, Haraldstad K. Usability evaluation of the preoperative ISBAR (identification, situation, background, assessment, and recommendation) desktop virtual reality application: qualitative observational study. JMIR Hum Factors. 2022;9(4):e40400. [FREE Full text] [CrossRef] [Medline]
    24. Mäkinen H, Haavisto E, Havola S, Koivisto JM. Graduating nursing students' user experiences of the immersive virtual reality simulation in learning - a qualitative descriptive study. Nurs Open. 2023;10(5):3210-3219. [FREE Full text] [CrossRef] [Medline]
    25. Hartstein AJ, Verkuyl M, Zimney K, Yockey J, Berg-Poppe P. Virtual reality instructional design in orthopedic physical therapy education: a mixed-methods usability test. Simul Gaming. 2022;53(2):111-134. [CrossRef]
    26. Lie SS, Røykenes K, Sæheim A, Groven KS. Developing a virtual reality educational tool to stimulate emotions for learning: focus group study. JMIR Form Res. 2023;7:e41829. [FREE Full text] [CrossRef] [Medline]
    27. Solstien 3. URL: https://www.solstien3.no/index_en.html [accessed 2024-09-28]
    28. National regulations relating to a common curriculum for health and social care education. URL: https:/​/www.​regjeringen.no/​contentassets/​389bf8229a3244f0bc1c7835f842ab60/​national-regulations-relating-to-a-common-curriculum-for-health-and -social-care-education.​pdf [accessed 2024-09-28]
    29. Lie SS, Alvestad RW, Helle N, Vikman MD, Dahl-Michelsen T. Exploring VR simulation in healthcare and social work education: students’ experiences with VR simulation as preparation for professional practice. Uniped. 2024;47(1):18-31.
    30. Helle N, Vikman MD, Dahl-Michelsen T, Lie SS. Health care and social work students' experiences with a virtual reality simulation learning activity: qualitative study. JMIR Med Educ. 2023;9:e49372. [FREE Full text] [CrossRef] [Medline]
    31. Verkuyl M, Romaniuk D, Mastrilli P. Virtual gaming simulation of a mental health assessment: a usability study. Nurse Educ Pract. 2018;31:83-87. [CrossRef] [Medline]
    32. Lee Y, Kim SK, Eom MR. Usability of mental illness simulation involving scenarios with patients with schizophrenia via immersive virtual reality: a mixed methods study. PLoS One. 2020;15(9):e0238437. [FREE Full text] [CrossRef] [Medline]
    33. Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105-112. [CrossRef] [Medline]
    34. Miguel-Alonso I, Checa D, Guillen-Sanz H, Bustillo A. Evaluation of the novelty effect in immersive virtual reality learning experiences. Virtual Real. 2024;28(1):27. [CrossRef]
    35. Noble SM, Saville JD, Foster LL. VR as a choice: what drives learners’ technology acceptance? Int J Educ Technol High Educ. 2022;19(1):6. [CrossRef]
    36. Saab MM, Hegarty J, Murphy D, Landers M. Incorporating virtual reality in nurse education: a qualitative study of nursing students' perspectives. Nurse Educ Today. 2021;105:105045. [FREE Full text] [CrossRef] [Medline]
    37. Coyne E, Frommolt V, Rands H, Kain V, Mitchell M. Simulation videos presented in a blended learning platform to improve Australian nursing students' knowledge of family assessment. Nurse Educ Today. 2018;66:96-102. [FREE Full text] [CrossRef] [Medline]
    38. Wang W, Liang Z, Blazeck A, Greene B. Improving Chinese nursing students' communication skills by utilizing video-stimulated recall and role-play case scenarios to introduce them to the SBAR technique. Nurse Educ Today. 2015;35(7):881-887. [CrossRef] [Medline]
    39. Massey D, Byrne J, Higgins N, Weeks B, Shuker MA, Coyne E, et al. Enhancing OSCE preparedness with video exemplars in undergraduate nursing students. a mixed method study. Nurse Educ Today. 2017;54:56-61. [FREE Full text] [CrossRef] [Medline]
    40. Polcar J, Horejsi P. Knowledge acquisition and cyber sickness: a comparison of VR devices in virtual tours. MM Sci J. 2015;2015(02):613-616. [CrossRef]
    41. Luctkar-Flude M, Tyerman J, Verkuyl M, Goldsworthy S, Harder N, Wilson-Keates B, et al. Effectiveness of debriefing methods for virtual simulation: a systematic review. Clin Simul Nurs. 2021;57:18-30. [CrossRef]
    42. Gardner R. Introduction to debriefing. Semin Perinatol. 2013;37(3):166-174. [CrossRef] [Medline]
    43. Børte K, Nesje K, Lillejord S. Barriers to student active learning in higher education. Teach High Educ. 2020;28(3):597-615. [CrossRef]

    ‎
    HSD: honestly significant difference
    VR: virtual reality

    Edited by B Lesselroth; submitted 28.01.24; peer-reviewed by A Bustillo, A Carrillo, E Rincon; comments to author 20.06.24; revised version received 06.08.24; accepted 24.09.24; published 19.11.24.

    Copyright

    ©Gry Mørk, Tore Bonsaksen, Ole Sønnik Larsen, Hans Martin Kunnikoff, Silje Stangeland Lie. Originally published in JMIR Medical Education (https://mededu.jmir.org), 19.11.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on https://mededu.jmir.org/, as well as this copyright and license information must be included.