This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on https://mededu.jmir.org/, as well as this copyright and license information must be included.
Clinical workplace learning takes place in a dynamic and complex learning environment that is designated as a site for patient care and education. Challenges in clinical training can be overcome by implementing blended learning, as it offers flexible learning programs suitable for student-centered learning, web-based collaboration, and peer learning.
The aim of this study is to evaluate the Small Private Online Course (SPOC) by interns’ first impressions and satisfaction measures (N=20) on using the SPOC. This study describes the design process of a SPOC from a theoretical and practical perspective and how it has been integrated into a clinical internship in internal medicine.
The design of the SPOC was based on general theoretical principles that learning should be constructive, contextual, collaborative, and self-regulated, and the self-determination theory to stimulate intrinsic motivation. Interns’ impressions and level of satisfaction were evaluated with a web-based questionnaire and group interview.
Interns thought the web-based learning environment to be a useful and accessible alternative to improve knowledge and skills. Peer learning and web-based collaboration through peer interaction was perceived as less effective, as student feedback was felt inferior to teacher feedback. The interns would prefer more flexibility within the course, which could improve self-regulated learning and autonomy.
The evaluation shows that the SPOC is a useful and accessible addition to the clinical learning environment, providing an alternative opportunity to improve knowledge and skills. Further research is needed to improve web-based collaboration and interaction in our course.
Clinical workplace learning (WPL) mostly takes place during normal daily collaboration and patient care activities, or organized formal learning activities [
Blended learning can be used to remedy several of these problems. Blended learning refers to a deliberate blending of face-to-face and web-based learning, with the goal of stimulating and supporting learning [
The web-based component of blended learning permits flexible education at a time, place, and pace convenient for the learner [
A Small Private Online Course (SPOC) is one possible instrument to blend web-based learning with clinical WPL. The SPOC concept was first introduced in 2013, and it has been progressively implemented in higher education thereafter. This type of course is often used locally with on-campus students and has a limited number of students that can enroll in the course [
The development of a dedicated SPOC instead of using publicly available course materials has the advantage that several secondary conditions such as contents definition, availability to interns, the alignment of goals, desired teaching modes, and assessments can be addressed by design. This avoids many of the current challenges with using open web-based education from others as described by de Jong et al [
In 2017, we developed a SPOC for the Internal Medicine internship at Leiden University Medical Center (LUMC) in the Netherlands. The theoretical framework of the course is based on the self-determination theory (SDT) [
In the Netherlands, medicine students enter medical school at the bachelor level, which is followed by internships at the master level. At LUMC, each month approximately 20 interns start their clinical internships. In the first 4 weeks, the interns attend a joint program at the university. The so-called introductory internship (2 weeks) prepares them for the internships in general and is being followed by a specific 2-week theoretical course as preparation for the Internal Medicine internship. Thereafter, they start their clinical internship in Internal Medicine (12 weeks), in which they work in different affiliated hospitals in the region and cannot meet each other physically. A major aim during the internship is to obtain clinical reasoning skills for a broad range of clinical scenarios. However, the interns only have limited exposure to new patients who have not yet been diagnosed by other physicians, and the clinical scenarios that interns face in practice do not cover all the clinical scenarios that they need to know and understand. Collaboration between peers is limited because the group of interns is split up to have their internship in different hospitals.
To overcome several of the limitations mentioned, a SPOC has been developed. The course was designed using a design-based research approach in which practical and theoretical aspects are integrated in the educational design [
For the development of the course, a group of stakeholders has been identified including clinical interns, clinical teachers, educational experts, technical experts, and a graphic designer. During a dedicated learning experience design (LED) session the stakeholders set a framework for the SPOC, including the team and course’s goals (eg, improve patient expose), students’ needs (eg, track progression within the course), learning goals (eg, improve clinical reasoning skills), and the aimed look and feel (eg, authenticity of the cases).
The outcomes of the LED session were used to define the exact content and learning activities centered around authentic clinical problems that are typically encountered in internal medicine. NEO Learning Management System was used as a platform for the SPOC. Authentic cases in Dutch were developed to train clinical reasoning skills and medical knowledge. The SPOC has a modular design that involves preparing for the internships (2 weeks), internal medicine (2 weeks), and several inpatient, outpatient, and emergency room cases (12 weeks). Every week, the interns can study 1 new activity. A course overview of the 12 clinical weeks is shown in
Small Private Online Course (SPOC) layout. Assessments during the 12 weeks of the clinical internship are displayed only (lessons presented in preparting for the internships and internal medicine are not displayed). The lessons include several inpatient, outpatient, and emergency room cases and assignments related to those participants.
Exemplary screenshots of exercises in the Small Private Online Courses (SPOCs) as they are offered to the students.
On the basis of the SDT and the general learning principles that learning should be constructive, contextual, collaborative, and self-regulated, authentic patient cases that are centered around virtual reality patients with complaints of diabetes, electrolyte disorders, infectious diseases, oncological diseases, and tiredness in a setting of the inpatient clinic, outpatient clinic, or emergency room have been developed. A 3D virtual reality patient ward has been developed, giving the interns a realistic impression of the inpatient setting. It aims to introduce interns to the ward and ward rounds and train professional standards such as hygiene regulations. Constructive learning activities such as doing rounds on 3D virtual reality patients with increasing complexity are used to gather knowledge and to improve learning effectiveness in the workplace.
Relatedness is promoted in the SPOC, as the interns (who work in different hospitals and do not have contact) meet up on the web on discussion forums, during peer feedback sessions and group assignments. This stimulates collaboration and relatedness among interns who would normally not meet each other during the internship. Collaborative forum assignments are included for peer learning and direct feedback by both peers and dedicated clinical teachers. Peer feedback is either given through open peer discussion on a forum or using a rubric format. Peer discussion, in this case on the web, might help interns to develop their critical thinking and clinical reasoning skills. Interns are prepared on their role as assessors by a web-based peer feedback training.
Besides obligatory content, the SPOC contains several optional assignments and resources that interns can choose from. The roster during the internship is different for each intern. An intern can spend 2 weeks in the cardiology department, whereas another intern works in the emergency room. Therefore, the interns can choose several sequential assignments for more extensive learning, depending on the training they need at that specific moment. By stimulating autonomy and attention to precourse goal setting and tracking and ranking course activity, it is aimed that interns can control their own learning process.
Training and assessment of competencies occur through peer feedback and self-assessment. Interns that have self-assessed their competencies and know that they are on the right track feel more confident while carrying out their new skills on real patients. In this blended program, skills that are trained on the web can be directly transferred to clinical practice. Practical examples of the integration of theory in the SPOC are shown in
Examples of theoretical integration in the Small Private Online Course (SPOC). (A) Constructive learning in the clinical context. The figure illustrates how interns go through an authentic simulated patient case in the SPOC. The learning objectives are defined for each learning activity separately. For example, in one of the diabetes cases, the learning objectives are defined as follows. By the end of this lesson, the intern (1) is able to distinguish between type 1 and type 2 diabetes based on epidemiology, history, physical examination, and additional tests; (2) knows the complications of diabetes mellitus and knows the screening protocols; (3) knows the general treatment modalities for diabetes mellitus and cardiovascular risk management; and (4) knows the chain of care for patients with diabetes and knows the physician’s role within the chain. (B) Collaborative learning and relatedness. This figure displays a group assignment in the SPOC. A clinical scenario is described by one group of interns (group A), and another group of interns (group B) elaborates their clinical reasoning process that is completed by a diagnosis. Thereafter, feedback is provided by both groups on the quality of the case and the diagnosis, respectively. (C) Self-regulated learning or autonomy. The courses’ lessons contain required and facultative issues for further learning.
Participants were a first group of 20 interns who worked with the SPOC during a pilot period. They enrolled during their introductory internship and remained in the SPOC until completion of their Internal Medicine internship. The participants were in the first year of their clinical phase. In this phase, interns are aged approximately 21 to 25 years, and, on average, 60% to 70% are women. As the aim is to evaluate the impressions of the first group, no sample size calculation has been performed.
The evaluation existed of a web-based questionnaire (
As the study did not involve patients and no health intervention has been administered to participants, the study was not subjected to the Dutch WMO (Medical Research involving Human Subjects Act). The study has been conducted in compliance with the European Union General Data Protection Regulation 2016/679, and data have been anonymized and stored according to the
The students completed the weekly assignments and remained in the SPOC until they had completed their Internal Medicine internship. Subsequently, they were asked to fill out the web-based questionnaire. A group interview was led by an independent interviewer who had no intern-teaching relationship with the interns. The interview used the
Means and SDs were calculated for the Likert-scale questions in the web-based questionnaire. The answers to the 5 open-ended questions were summarized.
The first and third authors discussed the transcript coding template, which was based on the different items in the theoretical framework, until consensus was obtained. The outcome was a template consisting of six predefined, overarching themes: (1) contextual learning, (2) collaboration and relatedness, (3) constructive learning, (4) self-regulated learning or autonomy, (5) competence, and (6) other. For each category, the same authors agreed on a definition for each theme. The principal investigator (ECH) analyzed the interview data and clustered the answers in the template using a Microsoft Word.
The aim of this study is to investigate the perceptions of interns concerning the use of the SPOC using a questionnaire and interview. Of the 20 interns eligible to enroll in the SPOC, 19 (95%) actually enrolled in the course, 10 (50%) finished the whole course, and 1 (5%) never started. Ten interns filled out the web-based questionnaire. Only questionnaires that were fully completed were included in the analysis. All 20 interns attended the group interview.
The results are shown in
Outcomes of the web-based questionnaire (n=10).
Question | Value, mean (SD)a |
1. Time per week spent in the course (hours) | 1.8 (0.92)b |
2. Following the course is useful during the internship | 4.5 (1.96) |
3. The course was informative | 5 (1.33) |
4. The SPOCc was a motivation for learning | 3 (1.49) |
5. I would recommend the course to peer students | 3.9 (1.91) |
6. The knowledge obtained from the SPOC was fairly applicable to clinical practice | 4 (1.89) |
7. Through working in the SPOC, my patient exposure has been increased | 3.3 (1.64) |
8. The cases in the SPOC were authentic | 4.8 (1.23) |
9. Making the cases, I really felt like a physician making decisions | 3.6 (2.01) |
10. I felt more competent in clinical reasoning after finalization of the SPOC | 4.1 (1.79) |
11. The SPOC had a good construction of increasing difficulty | 4.7 (1.06) |
12. I could organize my own time well within the SPOC | 4.1 (1.79) |
13. In the course, I had good interaction with peers | 2.4 (1.71) |
14. The interaction with peers was useful | 1.8 (1.03) |
15. The assignments and tests were challenging | 4.3 (1.70) |
aThe 7-point scale ranges as follows: (1) totally disagree to (4) neither disagree nor agree to (7) totally agree.
bNote that the number of of hours spent per week is displayed in this row.
cSPOC: Small Private Online Course.
The interns highlighted the patient cases as a positive aspect of the course. They particularly valued their connection with clinical practice and the elaboration and variety of relevant cases. Furthermore, the interns appreciated the group assignment in which they
The interns were interviewed in their last week of the internship.
The interns experienced the SPOC to be accessible and adequately designed for them as a target group. The participants felt the patient cases and SPOC content were useful and informative. However, although authenticity was integrated in the design of the SPOC, the participants felt that the contents did not match the real world. It seemed unclear to them how the SPOC should complement clinical WPL. They would prefer assignments that matched the clinical problems they encountered at that time instead of fixed weekly assignments.
Development of critical thinking skills by peer discussion was also integrated in the SPOC’s design; however, the interns indicated that they preferred model answers over peer discussion. The participants also felt it was not really useful to receive feedback from peers instead of a teacher, because in their opinion peers know as much as they do themselves. In general, feedback was perceived to be very short.
The interns indicated they had enough time to finish the assignments during their normal day shifts, although not everyone agreed. They also appreciated the facultative character of the SPOC. They felt that more facultative assignments would be helpful, although they addressed the possibility that nobody would mind finishing them. They also indicated that they needed more flexibility and choice in the course, and the fixed order of the assignments did not work for everyone. Some participants felt they had “finally reached the clinical phase of their internship” and therefore did not appreciate completing web-based assignments in this stage of the curriculum. They also preferred a complete overview of a certain clinical presentation instead of looking up information themselves through links in the SPOC.
Group interview concerning SPOCa perceptionsb.
Theme | Positive | Negative |
Contextual learning |
SPOC was adequately targeting interns Useful patient cases Content was handy and informative |
Assignments did not match clinical problems encountered in WPLc Learning objectives were unclear Unclear how SPOC complements WPL Contents did not match the real world |
Collaboration, relatedness, and constructive learning | —d |
Peer feedback is less useful than feedback from teacher Interaction Interns wanted answer sheets instead of peer discussion |
Self-regulated learning or autonomy |
Sufficient time to finish assignments during daily shift Nice that SPOC is not obligatory |
Tight deadlines Time-consuming Insufficient time to finish assignments during daily shift Need more choice instead of fixed assignments Do not want theoretical assignments during the practical phase Less obligatory and more optional assignments Want a complete overview of a clinical presentation, instead of looking up information Wished the SPOC to be more motivating |
Competence |
Good patient cases, e-learnings, and quizzes Useful lessons, mainly virtual reality patients Useful for interns with less patient contacts and less moments for clinical reasoning Enjoyed videos observing others taking history History taking videos led to discussion among peers Those videos must be part of the training |
SPOC did not fill gaps encountered in clinical practice Watching videos seeing others taking history is not active learning Learned more from observation in workplace Need more specific physical exam tools instead of observation general physical examination Need more assignments that specifically enhance knowledge |
Other |
SPOC was accessible Good learning environment Videos were enjoyable |
Unclear deadlines Problems planning patient-related assignments (suitable patients dismissed) Technical shortcomings Insufficient support |
aSPOC: Small Private Online Course.
bThe positive and negative perceptions of the interns are displayed in the middle and right columns. Those were assembled in 6 predefined clusters (left column). Collaboration and constructive learning have been clustered because of overlapping interview data.
cWPL: workplace learning.
dNot available.
The interns particularly valued the competence training received through completion of the patient cases, e-learning, and quizzes. The virtual reality patient cases and videos were rated as useful and likable. The videos concerning observation of patient interviews were enjoyable and led to useful discussion between peers. However, others felt that observation in clinical practice is more informative than watching web-based videos. Although the virtual reality patient cases can be an alternative for practicing clinical scenarios that have not been encountered in live patient contact, the interns indicated that the SPOC did not completely fill the gaps owing to the limited number of virtual reality patient cases addressed in the SPOC.
In general, the interns appreciated the accessibility of the SPOC and its learning environment, and the videos were enjoyable. They were less content about the deadlines that sometimes appeared to be unclear and the planning of SPOC activities that were directly patient related, which was challenging because of fast patient turnover rates. Completion of some of the assignments was difficult because of technical shortcomings or insufficient support.
In this study, a SPOC has been developed and implemented into an internship curriculum of Internal Medicine at LUMC. Course development was based on general learning principles and SDT, aiming to promote learner motivation and optimize the course’s quality and integration in the clinical curriculum. The SPOC was evaluated by measuring the satisfaction of the interns with the course. Interns thought the SPOC to be a useful and accessible addition to the clinical learning environment. Peer learning and web-based collaboration through peer interaction were perceived as less effective, as student feedback was felt inferior to teacher feedback. The interns preferred more flexibility within the course, which could improve self-regulated learning and autonomy. Overall, the interns felt that the web-based learning environment provided an alternative opportunity to improve knowledge and skills.
Our SPOC was integrated into clinical practice by making it part of the internship program, instead of using it as a stand-alone web-based training. The advantage of this is that learners can directly apply their new knowledge in clinical practice and go back to the web-based resources for further exploration of a topic. In our opinion, using design sessions and a theoretical framework can facilitate integration of such a course into an existing curriculum and can be an effective solution for some of the complexities faced in clinical training.
From the results, we learned that the SPOC meets the expectations for some of the categories fairly well, although the overall student satisfaction seems to be modest. In general, completing the SPOC assignments during the internships seemed feasible but not for all interns. Concerning learning in the clinical context, interns appreciated the authenticity and usefulness of patient cases as an additional opportunity to improve their clinical knowledge and skills. Regarding the clinical skills development, the SPOC therefore meets our hypothesis; however, this is not the case for stimulation of the social cohesion of the interns through working in the SPOC. The quantitative data show a modest to very low outcome on several topics (score ≤3.3: motivation, interaction, and usefulness of interaction with peers and feeling like a physician making decisions).
It appeared that the interns were not satisfied with the collaboration and relatedness aspects of the SPOC, in particular the web-based interaction and peer feedback. Other studies confirm that students prefer teacher feedback over peer feedback [
Learning activities in a SPOC are on the web and asynchronous, with only written interaction and no visual interaction and body language [
Although flexibility was integrated in the design of the course based on previous literature [
Regarding the feelings of being a physician making decisions and contextual learning, it seemed that the SPOC somehow failed in this aspect. In the LED session it was predefined to make authentic simulated patient cases to enhance this feeling and the feeling of increased patient exposure; however, this was not experienced by the interns. Although studies have shown that virtual reality patients can improve clinical reasoning skills and knowledge, it should be recognized that patient simulations are not equivalent to real patients and cannot replace traditional clinical WPL [
Reflecting on our study, this may imply some improvements in our SPOC’s design. First of all, we should critically review the technical shortcomings, deadlines, and obligatory components of the SPOC and improve the flexibility of the learning environment. For instance, replacing the weekly assignments by a more variable set of assignments on a monthly basis could be considered. We are currently investigating the feasibility of such adaptations and the expansion of the number of optional lessons. In addition, the SPOC will soon be transferred to another internet platform, which is technically supported by our University’s Information Technology department.
Second, it may be helpful to incorporate training of self-regulated learning skills before using our hypermedia environment and more support and augmentation of learners’ self-regulated learning within the learning system [
Third, several other conditions may be met to increase student satisfaction. Student satisfaction in clinical internships is enhanced by supervisor support, perceived social value [
When critically looking at the quantitative data, it is apparent that the variance is modest and particularly low for the questions concerning the interaction with peers and increasing difficulty. An explanation might be that the interns know each other fairly well and their answers may not be independent or even biased by information, sharing leading to a group opinion. Limitations of the study were the small sample size and use of an unvalidated questionnaire. As the study was designed just to evaluate the first group of interns enrolled in the SPOC, no formal sample size calculation has been performed. However, we did include qualitative data to supplement these limited quantitative outcomes. We therefore think that the impressions still provide valuable insights in the SPOC’s strengths and weaknesses that we can use for further adaptation.
From our study, we have learned that interns perceive several learning opportunities after adding the SPOC to their clinical learning environment, mainly in skills and knowledge acquisition. However, particularly web-based collaboration and perception of relatedness among the interns within the course need further improvement. In the future, interviews with the interns may be beneficial for a deeper investigation of this issue and others, their context, and which additional adaptations might be needed to our SPOC. Future research is also needed to further investigate how learning principles can be optimally integrated in web-based courses.
Questionnaire.
learning experience design
Leiden University Medical Center
self-determination theory
Small Private Online Course
workplace learning
This research was funded by an educational grant of Leiden University (project reference 4020522012). The authors wish to thank Franka Luk and Manon Zuurmond for their contribution to this study and illustrations, and Mandy Segers for her assistance in finalizing the manuscript revisions.
All authors were involved in the design of the study and draft of the manuscript. ECH and FMvB analyzed and interpreted the results, and PGMDJ had major contributions to the main conclusions. All authors read and approved the final manuscript. The authors alone are responsible for the content and writing of the paper.
None declared.