Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 01.10.20 in Vol 6, No 2 (2020): Jul-Dec

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/17030, first published Nov 19, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    Understanding Medical Students’ Attitudes Toward Learning eHealth: Questionnaire Study

    1Maastricht University Medical Center, Maastricht, Netherlands

    2Skillslab, Maastricht University Medical Center, Maastricht, Netherlands

    3Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Center, Maastricht, Netherlands

    4Department of Educational Development and Research, Maastricht University Medical Center, Maastricht, Netherlands

    5Department of Neurosurgery, Maastricht University Medical Center, Maastricht, Netherlands

    Corresponding Author:

    Kjeld Vossen, MD

    Maastricht University Medical Center

    P. Debyelaan 25

    Maastricht

    Netherlands

    Phone: 31 628091727

    Email: kjeldvossen@hotmail.com


    ABSTRACT

    Background: Several publications on research into eHealth demonstrate promising results. Prior researchers indicated that the current generation of doctors is not trained to take advantage of eHealth in clinical practice. Therefore, training and education for everyone using eHealth are key factors to its successful implementation. We set out to review whether medical students feel prepared to take advantage of eHealth innovations in medicine.

    Objective: Our objective was to evaluate whether medical students desire a dedicated eHealth curriculum during their medical studies.

    Methods: A questionnaire assessing current education, the need for education about eHealth topics, and the didactical forms for teaching these topics was developed. Questionnaire items were scored on a scale from 1 (fully disagree with a topic) to 10 (fully agree with a topic). This questionnaire was distributed among 1468 medical students of Maastricht University in the Netherlands. R version 3.5.0 (The R Foundation) was used for all statistical procedures.

    Results: A total of 303 students out of 1468, representing a response rate of 20.64%, replied to our questionnaire. The aggregate statement “I feel prepared to take advantage of the technological developments within the medical field” was scored at a mean value of 4.8 out of 10. Mean scores regarding the need for education about eHealth topics ranged from 6.4 to 7.3. Medical students did not favor creating their own health apps or mobile apps; the mean score was 4.9 for this topic. The most popular didactical option, with a mean score 7.2, was to remotely follow a real-life patient under the supervision of a doctor.

    Conclusions: To the best of our knowledge, this is the largest evaluation of students’ opinions on eHealth training in a medical undergraduate curriculum. We found that medical students have positives attitudes toward incorporating eHealth into the medical curriculum.

    JMIR Med Educ 2020;6(2):e17030

    doi:10.2196/17030

    KEYWORDS



    Introduction

    According to the World Health Organization (WHO) definition, eHealth is the use of information and communications technology (ICT) to provide enhanced health services to communities [1]. eHealth services are defined as telehealth, electronic health records, mobile health, social media, and big data [1]. Several articles show promising results when eHealth is being used in medical fields [2-11]. For instance, it has been shown that eHealth interventions help to improve medication adherence [4], glycemic control in diabetes patients [6], and self-care among heart failure patients as well as improve the outcomes of cardiac rehabilitation among coronary heart disease patients [7,8] and improve mental health [9,10]. Teleconsultation by general practitioners has been proven as an alternative to face-to-face consultations in certain situations [11]. A review of 58 systematic reviews showed that overall eHealth provides beneficial results in a wide variety of medical applications [12]. These developments improve the quality of care or maintain the current standard and reduce health care costs [13-15]. These studies indicate that eHealth would allow for a change clinical practice for the better by using technology. However, this requires a workforce that is prepared to practice medicine in a way where eHealth is integrated into clinical practice.

    Literature about educating medical students in the field of eHealth is scarce [16]. Due to this scarcity, we concluded that the implementation of eHealth education into the medical curriculum is limited. A recent assessment of medical curricula in Sweden showed that only one university had concrete plans about implementing eHealth into their medical curriculum [17]. Another trial in Australia showed that none of the universities had established an eHealth program [18]. In addition, the European Health Parliament found that current health professionals do not feel adequately trained in eHealth and found that formal eHealth training is lacking from an early stage in the training of medical professionals [19]. Universities and their executives are aware of the lack of formal eHealth education, but the medical curriculum is already crowded and priorities are given to other subjects [18].

    Studies with composite student groups show that including eHealth courses in curricula increases knowledge and awareness about the topic [20,21]. There are several studies focusing solely on medical students where specific eHealth topics were tested; for example, app development or telehealth consultation skills [21-24]. These studies all show that a course, or even just one class, enhances the knowledge about specific topics and is appreciated by students. All this research assumes the top-down idea that eHealth education is important and necessary [16-27].

    It may seem logical to incorporate eHealth into the medical curriculum; however, we do not know students’ perceptions about this. In fact, we could not find any articles that attempted to find out where students stand with regard to eHealth education. It might be that students are unaware of the lack of eHealth training and, therefore, do not feel the need for additional education. If students feel that education about eHealth is unnecessary, a different approach to teaching them is needed, compared to when students feel like they need more education about eHealth. The goal of our study is to evaluate whether medical students feel prepared to take advantage of eHealth innovation in medicine.


    Methods

    Setting

    The following study was conducted at the medical school of Maastricht University in the Netherlands between February and May 2018. During the 6-year-long undergraduate medical curriculum—the duration of the bachelor and master programs are 3 years each—there is no formal education about eHealth. The most likely way students might encounter eHealth is through their medical rotations.

    Ethics Approval and Consent to Participate

    This study was not submitted nor approved by an institutional ethics committee because we did not deem this necessary in accordance with Dutch law. Dutch ethical law states that ethical approval is only necessary in the case of medical research including human test subjects, as can be read in the Medical Research Involving Human Subjects Act (Wet medisch-wetenschappelijk onderzoek met mensen, in Dutch), paragraph 2 [28]. Our research was aimed at the improvement of education and training without submitting the participants to any medical intervention. Therefore, this was not deemed medical research but educational research. Consent to participate was obtained at the beginning of the questionnaire administration. When the participants opened the questionnaire online, they were met with a statement stating that participants consented to participate and the data could be used for research purposes.

    Questionnaire

    Overview

    We were unable to find any pre-existing questionnaire that assessed students’ attitudes toward learning eHealth; therefore, we decided to create our own questionnaire. To assess students’ attitudes toward learning eHealth, we developed a Dutch questionnaire using Google Forms that was made accessible for the participants to fill out from February 2018 until May 2018. We chose this type of survey in order to reach as many students as possible and to increase the number of responses. Most of the students spend little time at the university and prefer to either work from home or spend their time learning at the hospital during their clinical rotations. The most effective way for us to reach these students was by using a format such as Google Docs.

    We based the statements on the WHO definition of eHealth. We incorporated a question about every aspect of this definition in our questionnaire. Furthermore, we specified between the bachelor and master curricula to see if there were any significant differences between the two subgroups.  Because there is currently no dedicated eHealth education course offered, we could not evaluate these topics; we could only assess whether or not our students would want access to education about the topics.

    Excluding the personal questions, the questionnaire contained 18 or 20 statements, depending on the participants’ study phase: bachelor or master program. The statements, which were translated into English for this paper, are listed in the Results section later in this article. The statements about current education and didactical options were piloted among 6 master students and 4 bachelor students, who found the statements clear and comprehensive. The master students’ questionnaire contained two additional statements about the medical rotations and in-class education during the master program. The rest of the questionnaire was identical for both master and bachelor program students.

    Characteristics

    The first section of the questionnaire gathered participant characteristics (see Table 1). The answer given for the question study level routed participants to the next section. Bachelor students were routed to a form evaluating solely the bachelor education. Master students were routed to a form evaluating both the bachelor and master education. Participants could indicate how long ago they finished their medical education, choosing between longer than 2 years ago or between 0 and 2 years ago.

    Table 1. Participant characteristics.
    View this table
    Current Education

    The second section reviewed current education. All questions about the current education and didactical options were answered using a 10-point Likert scale, ranging from 1 (fully disagree) to 10 (fully agree). We decided on a scale from 1 to 10 because our students are used to be graded using this scale, where a grade above 5.5 is considered satisfactory. Both the bachelor and master students’ forms contained the following statement: “I feel prepared to take advantage of the technological developments within the medical field.” This statement was included as an aggregate global judgment about the entire curriculum.

    eHealth Topics and Didactical Format

    After the section about the students’ current education, the questionnaire was the same for every participant. The third section evaluated how students felt about different eHealth subjects in the medical curriculum. Topics listed were chosen based on the WHO definition of eHealth, namely, mobile apps, telemonitoring, applying modern technology in practice, data science, and machine learning. The last seven statements evaluated which didactical format the students preferred.

    At the end of the survey, students had the option to give feedback or add explanations to their answers. It was not possible to skip questions or statements during the questionnaire; therefore, all questionnaires we received were complete.

    Participants

    A total of 316 medical students enroll at Maastricht University’s medical school every year, resulting in a total of about 1896 students. We promoted the questionnaire via social media groups that are only accessible by our university medical students. The local medical student association allowed us to use their newsletter to promote the questionnaire. In addition to this, we reached out to medical students through their social media accounts by sending them a personal message about the questionnaire with a link to the Google Form. There was no incentive for students to fill out the form and there were no negative consequences if students did not fill out the form. The inclusion criterion was as follows: medical student actively studying at the time of the survey. The reason for applying this criterion was that all participants will have studied the same curriculum.

    Statistical Analysis

    Descriptive statistics consisting of the mean, standard deviation, 95% confidence interval, and Cronbach α were calculated, and box and whisker plots were drawn to give a graphic, representation of the results. R version 3.5.0 (The R Foundation) was used for all statistical procedures.


    Results

    Demographics

    In total, 1468 invitations were sent to medical students to participate in the survey using social media, WhatsApp, and the platforms provided by the medical student association. There were 303 responses to the questionnaire, giving a response rate of 20.64% (303/1468). Characteristics of the participants are listed in Table 1. Most participants were female (215/303, 71.0%). The mean age was 22 years (range 20-25). This is comparable to the average age of medical students at our university. Master students were the largest subgroup, with a total of 183 participants out of 303 (60.4%). The other 120 participants were bachelor students (39.6%).

    Results From the Survey

    Table 2 shows how prepared the students feel to use eHealth in their future medical practice. The global aggregate statement “I feel prepared to take advantage of the technological developments within the medical field” scored a low value of 4.8 out of 10 (95% CI 4.6-5.0). Figure 1 shows the students’ attitudes toward different topics upon which we questioned them. Students assigned positive values (ie, a score of 6 or higher) to all topics, meaning that they would like to receive more education about a given topic. The least popular topic was that of machine learning, which had a mean score of 6.4 (SD 1.8, 95% CI 6.2-6.9). The most popular topic was that of applying modern electronic technologies in health care, which had a mean score of 7.3 (SD 1.6, 95% CI 7.1-7.4). When comparing the results from the statements about the current curriculum with results from statements about the eHealth topics, there was a difference between the two. There was a discrepancy between eHealth-related content in current medical education and the amount of eHealth training that is considered useful by medical students.

    Figure 1. Students' attitudes toward given topics in the medical curriculum (N=303). Scores range from 1 (fully disagree) to 10 (fully agree).
    View this figure
    Table 2. Responses to statements regarding how prepared medical students feel with regard to eHealth and the education they would like to receive.
    View this table

    Figure 2 lists the didactical format the students would prefer when learning about the given topics. The most popular format, with a mean score of 7.2 (SD 2.0), was to remotely follow a real patient under the supervision of a doctor. The least popular form of education was the development of a student’s own health app or mobile app. This only received a mean score of 4.9 (SD 2.7).

    Figure 2. Students' opinions about how they would like to receive education about eHealth topics (N=303). Scores range from 1 (fully disagree) to 10 (fully agree).
    View this figure

    Discussion

    Principal Findings

    To the best of our knowledge, this is the largest evaluation of students’ opinions on eHealth training in a medical undergraduate curriculum. We found that medical students have positive attitudes toward incorporating eHealth into the medical curriculum. This study showed that students do not feel well prepared to take advantage of eHealth in medical practice. The students scored an average of 4.8 out of a maximum of 10 when asked how prepared they felt to take advantage of eHealth in medical practice, while the need for education about the given topics scored a minimum of 6.4 and a maximum of 7.3. These results might provide a basis from which to continue a discussion regarding integrating eHealth into the medical curriculum.

    eHealth has been proven to be effective in clinical practice [1-13,29,30]; however, eHealth has not yet been implemented into the working standards of many doctors [31-33]. Prior research indicates that training and education for all those involved with implementation and the use of eHealth is a key factor for the successful incorporation of eHealth [13-16,30]. There are various obstacles to implementing eHealth into clinical practice. The two biggest barriers to the use of eHealth in clinical practice are as follows: technically challenged staff (11%) and resistance to change (8%) [32].

    Resistance to change is linked to several factors; lack of knowledge and skill obsolescence are major contributing factors [34]. Lack of eHealth skills and training is prevalent among the current medical workforce. This has been shown to be a major barrier to eHealth adoption [35-38]. This results from the fact that the workforce is not adequately trained to implement eHealth in medical practice [25-27,39]. Awareness and knowledge of what needs to change are essential in enabling change [40]. It has previously been shown that education can be used to overcome resistance to change [41,42]. Therefore, education about eHealth for those involved, in this case the undergraduate students, can lessen the barriers previously mentioned and help to create a workforce that is open and able to use eHealth in their daily practice. Many universities do not have dedicated eHealth training in the current curriculum, adding to the resistance to change [17,18]. Universities and their executives are aware that eHealth training is important, but due to the already overcrowded curricula with competing interests, implementation is lacking [18]. Our results suggest that we should dedicate more time to eHealth training, even though the curriculum is already overcrowded. This overcrowding can be overcome by changing education in the same way that clinical practice is being changed by eHealth. Case-based discussions that are based on a fictional patient can be replaced by cased-based discussions that are based on a real-life patient being remotely monitored in in a hospital, simulation patients can be replaced by teleconference simulation patients, and clinical rotations can incorporate remote care. eHealth education can be combined with the current subjects taught in the curriculum without taking up more time. By thinking of eHealth education the same way as eHealth implementation into clinical practice, it should be possible to incorporate this education into the medical curriculum. eHealth is not some added technology that takes up time; it should be an integrated aspect in clinical practice that improves the quality of care while reducing the workload. Showing students early on during their education that eHealth can be used in these ways might add to a certain digital mindset that is needed to use eHealth to its full extent as a clinician.

    Medical students are not receiving enough education to prepare them for eHealth competency. The current workforce is not adequately trained to implement new technologies in working practice [17,18,25,26,34-39,43]. The data we gathered support this because the participants indicated they do not feel prepared to benefit from the technological developments in health care. This holds true for both the bachelor and master students. The master students gave the amount of eHealth training in the bachelor program a lower score, which might indicate that during their medical rotations they were confronted by the fact that their digital skills were lacking. The students indicated that during their rotations, they were not learning enough about eHealth and the use of technology within health care. This might be due to the fact that there is no mention of eHealth skills in the national Dutch framework, which states what competencies a future doctor needs. During these medical rotations, opportunities to develop digital skills are lacking. Their teachers make up the current workforce of doctors. We previously established that this workforce is not adequately prepared to take advantage of the possibilities offered by eHealth. Therefore, we cannot expect them to train the new generation of doctors to attain sufficient digital skills. It would be possible to integrate digital skills education during the medical rotations if taught by eHealth or ICT professionals. The skills could then be further developed during medical rotations, providing a solid foundation for the future workforce.

    If eHealth education is implemented early in medical education, this might result in professionals being able to benefit from eHealth. Both bachelor and master students indicated that they feel a lack of this type of education. We concluded from this that it would be beneficial to start training the students during their bachelor phase. This would mean that they would be better prepared to use the skills they have learned during their medical rotations and, therefore, gain practical experience using their skills as soon as possible.

    This study works as a basis to support the need for eHealth education among medical students. There is still a lot of work that needs to be done with regard to a framework that defines which eHealth competencies are needed by future doctors, what eHealth subjects should be prioritized, and how students should be taught these subjects.

    Another factor that may cause insufficient attention with regard to eHealth training is the assumption that today’s students are up to date with technological developments, including an understanding of eHealth, because of the widespread use of technology among this generation [20,44]. However, prior research demonstrates that undergraduate students do not have this knowledge [25-27]. It could mean that health care is missing out on some of the potential benefits of eHealth due to this assumption. Nevertheless, it is important to use the skills that students have already gathered in the digital age while training them for their professions as future doctors. We should, therefore, always invite student panels while creating a future eHealth curriculum.

    Incorporating formal eHealth education into the medical curriculum may contribute to creating a necessary digital mindset [27,45]. This digital mindset means more than just the use of tools [31,32]; it would mean that medical professionals would start to think differently regarding how to provide health care. An example would be to think about how to change from traditional in-hospital care to future health care within patients’ homes. We are increasingly able to gather large amounts of patient data and need medical staff that can think in creative ways in using this data [43]. We hope that by supporting the development of a digital mindset in future medical staff that they will see opportunities in techniques such as data science, machine learning, and deep learning.

    Strengths and Limitations

    The first limitation in this study is a low response rate amounting to 20.64% (303/1468). We used Google Forms to poll our students. This platform provides an easy and accessible way for participants to fill out forms. The downside was that it was impossible to make sure that people did not fill out the form multiple times. However, it seems unlikely that people filled out multiple forms. If duplicates had been filled out by certain individuals, this would change the demography of our sample. However, the demography of the participants matches the demography of our student population. Besides that, we checked the time stamps of all the forms to check if there were identical forms filled out in short succession. After checking time stamps and responses of each questionnaire, we found that there was no evidence that duplicate forms had been filled out.

    Before we created the questionnaire, we looked for similar surveys, but we were not able to find any related to medical education. That is why we compiled our own questionnaire using the WHO definition for eHealth and recent literature. We took a pragmatic approach when creating our questionnaire, and the final questionnaire was not validated. All participants were from a single university in the Netherlands. This may limit the external validity of our results. During our research, we have noticed that there is a limited amount of literature about eHealth training for medical students and, therefore, we assume that most universities do not have a dedicated eHealth program. In this case, our results could inform other universities about the lack of eHealth training in their curriculum.

    What This Paper Adds

    The previous statements saying that eHealth education is necessary were top-down statements. This paper adds the students’ views on eHealth and shows that students feel that they are not prepared to take advantage of the possibilities provided by eHealth. This paper shows that students want more education about eHealth topics.

    Conclusions

    This study demonstrates that students consider themselves insufficiently prepared for the digital aspect of their future medical practices and that they support greater attention to eHealth in the medical curriculum. This study indicates that the lack of eHealth education is not something that is experienced only by researchers who write about eHealth education but also by the medical students themselves.

    Conflicts of Interest

    None declared.

    References

    1. WHO Global Observatory for eHealth. Atlas of eHealth Country Profiles: The Use of eHealth in Support of Universal Health Coverage: Based on the Findings of the Third Global Survey on eHealth 2015. Geneva, Switzerland: World Health Organization; 2015.   URL: https://apps.who.int/iris/rest/bitstreams/908392/retrieve [accessed 2020-09-06]
    2. Lluch M, Abadie F. Exploring the role of ICT in the provision of integrated care: Evidence from eight countries. Health Policy 2013 Jun;111(1):1-13. [CrossRef] [Medline]
    3. Haluza D, Jungwirth D. ICT and the future of health care: Aspects of health promotion. Int J Med Inform 2015 Jan;84(1):48-57. [CrossRef] [Medline]
    4. Linn AJ, Vervloet M, van Dijk L, Smit EG, Van Weert JCM. Effects of eHealth interventions on medication adherence: A systematic review of the literature. J Med Internet Res 2011 Dec 05;13(4):e103 [FREE Full text] [CrossRef] [Medline]
    5. Dellifraine JL, Dansky KH. Home-based telehealth: A review and meta-analysis. J Telemed Telecare 2008;14(2):62-66. [CrossRef] [Medline]
    6. Polisena J, Tran K, Cimon K, Hutton B, McGill S, Palmer K. Home telehealth for diabetes management: A systematic review and meta-analysis. Diabetes Obes Metab 2009 Oct;11(10):913-930. [CrossRef] [Medline]
    7. Radhakrishnan K, Jacelon C. Impact of telehealth on patient self-management of heart failure: A review of literature. J Cardiovasc Nurs 2012;27(1):33-43. [CrossRef] [Medline]
    8. Su JJ, Yu DSF, Paguio JT. Effect of eHealth cardiac rehabilitation on health outcomes of coronary heart disease patients: A systematic review and meta-analysis. J Adv Nurs 2020 Mar;76(3):754-772. [CrossRef] [Medline]
    9. Spijkerman MPJ, Pots WTM, Bohlmeijer ET. Effectiveness of online mindfulness-based interventions in improving mental health: A review and meta-analysis of randomised controlled trials. Clin Psychol Rev 2016 Apr;45:102-114 [FREE Full text] [CrossRef] [Medline]
    10. Deady M, Choi I, Calvo RA, Glozier N, Christensen H, Harvey SB. eHealth interventions for the prevention of depression and anxiety in the general population: A systematic review and meta-analysis. BMC Psychiatry 2017 Aug 29;17(1):310 [FREE Full text] [CrossRef] [Medline]
    11. Downes MJ, Mervin MC, Byrnes JM, Scuffham PA. Telephone consultations for general practice: A systematic review. Syst Rev 2017 Jul 03;6(1):128 [FREE Full text] [CrossRef] [Medline]
    12. Totten AM, Womack DM, Eden KB, McDonagh MS, Griffin JC, Grusing SG, Pacific Northwest Evidence-based Practice Center. Telehealth: Mapping the Evidence for Patient Outcomes From Systematic Reviews. Rockville, MD: Agency for Healthcare Research and Quality (AHRQ); 2016 Jun.   URL: https://www.ncbi.nlm.nih.gov/books/NBK379320/pdf/Bookshelf_NBK379320.pdf [accessed 2020-09-06]
    13. Akiyama M, Yoo B. A systematic review of the economic evaluation of telemedicine in Japan. J Prev Med Public Health 2016 Jul;49(4):183-196 [FREE Full text] [CrossRef] [Medline]
    14. Elbert NJ, van Os-Medendorp H, van Renselaar W, Ekeland AG, Hakkaart-van Roijen L, Raat H, et al. Effectiveness and cost-effectiveness of eHealth interventions in somatic diseases: A systematic review of systematic reviews and meta-analyses. J Med Internet Res 2014 Apr 16;16(4):e110 [FREE Full text] [CrossRef] [Medline]
    15. Wade VA, Karnon J, Elshaug AG, Hiller JE. A systematic review of economic analyses of telehealth services using real time video communication. BMC Health Serv Res 2010 Aug 10;10:233 [FREE Full text] [CrossRef] [Medline]
    16. Edirippulige S, Armfield NR. Education and training to support the use of clinical telehealth: A review of the literature. J Telemed Telecare 2017 Feb;23(2):273-282. [CrossRef] [Medline]
    17. Hoffmann F, Nordgren H. E-health education in undergraduate medical schools in Sweden [article in Swedish]. Lakartidningen 2019 May 07;116:FMA4 [FREE Full text] [Medline]
    18. Edirippulige S, Brooks P, Carati C, Wade VA, Smith AC, Wickramasinghe S, et al. It's important, but not important enough: eHealth as a curriculum priority in medical education in Australia. J Telemed Telecare 2018 Dec;24(10):697-702. [CrossRef] [Medline]
    19. European Health Parliament. Re-thinking European Healthcare Recommendations: Recommendations by the Next Generation. Brussels, Belgium: European Health Parliament; 2016 Jun.   URL: https:/​/www.​healthparliament.eu/​wp-content/​uploads/​2018/​04/​Re-thinking-European-Healthcare-EHP-Book-2015_2016.​pdf [accessed 2020-09-06]
    20. Koch S, Hägglund M. Mutual learning and exchange of health informatics experiences from around the world: Evaluation of a massive open online course in eHealth. Stud Health Technol Inform 2017;245:753-757. [Medline]
    21. O'Shea J, Berger R, Samra C, Van Durme D. Telemedicine in education: Bridging the gap. Educ Health (Abingdon) 2015;28(1):64-67. [CrossRef] [Medline]
    22. Masters K. Health professionals as mobile content creators: Teaching medical students to develop mHealth applications. Med Teach 2014 Oct;36(10):883-889. [CrossRef] [Medline]
    23. Rienits H, Teuss G, Bonney A. Teaching telehealth consultation skills. Clin Teach 2016 Apr;13(2):119-123. [CrossRef] [Medline]
    24. Brockes C, Grischott T, Dutkiewicz M, Schmidt-Weitmann S. Evaluation of the education "clinical telemedicine/e-health" in the curriculum of medical students at the University of Zurich. Telemed J E Health 2017 Nov;23(11):899-904. [CrossRef] [Medline]
    25. Dattakumar A, Gray K, Henderson K, Maeder A, Chenery H. We are not educating the future clinical health professional workforce adequately for e-health competence: Findings of an Australian study. Stud Health Technol Inform 2012;178:33-38. [Medline]
    26. Dattakumar A, Gray K, Maeder A, Butler-Henderson K. Ehealth education for future clinical health professionals: An integrated analysis of Australian expectations and challenges. Stud Health Technol Inform 2013;192:954. [Medline]
    27. Stellefson M, Hanik B, Chaney B, Chaney D, Tennant B, Chavarria EA. eHealth literacy among college students: A systematic review with implications for eHealth education. J Med Internet Res 2011 Dec 01;13(4):e102 [FREE Full text] [CrossRef] [Medline]
    28. Paragraaf 2. Regels voor wetenschappelijk onderzoek met proefpersonen [document in Dutch]. Wet medisch-wetenschappelijk onderzoek met mensen (Medical Research Involving Human Subjects Act). 2020 Jan 01.   URL: https://wetten.overheid.nl/BWBR0009408/2020-01-01#Paragraaf2 [accessed 2020-09-06]
    29. Hage E, Roo JP, van Offenbeek MAG, Boonstra A. Implementation factors and their effect on e-Health service adoption in rural communities: A systematic literature review. BMC Health Serv Res 2013 Jan 12;13:19 [FREE Full text] [CrossRef] [Medline]
    30. Bashshur RL, Shannon GW, Smith BR, Alverson DC, Antoniotti N, Barsan WG, et al. The empirical foundations of telemedicine interventions for chronic disease management. Telemed J E Health 2014 Sep;20(9):769-800 [FREE Full text] [CrossRef] [Medline]
    31. Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: A systematic review of systematic reviews (an update). Implement Sci 2016 Oct 26;11(1):146 [FREE Full text] [CrossRef] [Medline]
    32. Scott Kruse C, Karem P, Shifflett K, Vegi L, Ravi K, Brooks M. Evaluating barriers to adopting telemedicine worldwide: A systematic review. J Telemed Telecare 2018 Jan;24(1):4-12 [FREE Full text] [CrossRef] [Medline]
    33. Codagnone C, Lupiañez-Villanueva F, European Commission. Benchmarking Deployment of eHealth Among General Practitioners (2013) - Final Report. Luxembourg: Publications Office of the European Union; 2013.   URL: http://ec.europa.eu/newsroom/dae/document.cfm?doc_id=4897 [accessed 2020-09-06]
    34. Amjad A, Rehman M. Resistance to change in public organization: Reasons and how to overcome it. Eur J Bus Sci Technol 2018 Nov 14;4(1):56-68 ISSN 2336-6494 [FREE Full text] [CrossRef]
    35. Alami H, Gagnon MP, Wootton R, Fortin JP, Zanaboni P. Exploring factors associated with the uneven utilization of telemedicine in Norway: A mixed methods study. BMC Med Inform Decis Mak 2017 Dec 28;17(1):180 [FREE Full text] [CrossRef] [Medline]
    36. Kooienga S. Rural patients' and primary care clinic staffs' perceptions of EHR implementation: An ethnographic exploration. J Ambul Care Manage 2018;41(1):71-79. [CrossRef] [Medline]
    37. Li J, Talaei-Khoei A, Seale H, Ray P, Macintyre CR. Health care provider adoption of eHealth: Systematic literature review. Interact J Med Res 2013 Apr 16;2(1):e7 [FREE Full text] [CrossRef] [Medline]
    38. Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: A systematic review of systematic reviews (an update). Implement Sci 2016 Oct 26;11(1):146 [FREE Full text] [CrossRef] [Medline]
    39. Gray K, Dattakumar A, Maeder A, Butler-Henderson K, Chenery HJ. Advancing Ehealth Education for the Clinical Health Professions: Final Report 2014. Sydney, Australia: Australian Government Office for Learning and Teaching; 2014 Jan 01.   URL: https:/​/pure.​bond.edu.au/​ws/​portalfiles/​portal/​26149651/​Advancing_Ehealth_Education_for_the_Clinical_Health_Professions_Final_Report_2014.​pdf [accessed 2020-09-06]
    40. National Institute for Health and Clinical Excellence. How to Change Practice: Understand, Identify and Overcome Barriers to Change. London, UK: National Institute for Health and Clinical Excellence; 2007 Dec.   URL: https:/​/www.​nice.org.uk/​media/​default/​about/​what-we-do/​into-practice/​support-for-service-improvement-and-audit/​how-to-change-practice-barriers-to-change.​pdf [accessed 2020-09-06]
    41. Tyler DA, Lepore M, Shield RR, Looze J, Miller SC. Overcoming resistance to culture change: Nursing home administrators' use of education, training, and communication. Gerontol Geriatr Educ 2014;35(4):321-336 [FREE Full text] [CrossRef] [Medline]
    42. Salam M, Alghamdi KS. Nurse educators: Introducing a change and evading resistance. J Nurs Educ Pract 2016 Jun 30;6(11):80-83 [FREE Full text] [CrossRef]
    43. Evers J, Barels W. Business Models & eHealth (Een bedrijfsmatige aanpak in de gezondheidszorg) [bachelor's thesis] [document in Dutch]. Enschede, the Netherlands: University of Twente; 2014 Aug 15.   URL: https://essay.utwente.nl/65656/1/Business%20Models%20%26%20eHealth%20%2815-08-2014%29.pdf [accessed 2020-09-06]
    44. O'Connor S, Hanlon P, O'Donnell CA, Garcia S, Glanville J, Mair FS. Understanding factors affecting patient and public engagement and recruitment to digital health interventions: A systematic review of qualitative studies. BMC Med Inform Decis Mak 2016 Sep 15;16(1):120 [FREE Full text] [CrossRef] [Medline]
    45. Meskó B, Drobni Z, Bényei É, Gergely B, Győrffy Z. Digital health is a cultural transformation of traditional healthcare. Mhealth 2017;3:38 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    ICT: information and communications technology
    WHO: World Health Organization


    Edited by G Eysenbach; submitted 19.11.19; peer-reviewed by S Edirippulige, F Saigí-Rubió, K Ndlovu; comments to author 02.03.20; revised version received 25.05.20; accepted 22.06.20; published 01.10.20

    ©Kjeld Vossen, Jan-Joost Rethans, Sander M J van Kuijk, Cees P van der Vleuten, Pieter L Kubben. Originally published in JMIR Medical Education (http://mededu.jmir.org), 01.10.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on http://mededu.jmir.org/, as well as this copyright and license information must be included.