Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 29.06.15 in Vol 1, No 1 (2015): Jan-Jun

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/4267, first published Jan 21, 2015.

This paper is in the following e-collection/theme issue:

    Original Paper

    Information-Seeking Behaviors of Medical Students: A Cross-Sectional Web-Based Survey

    1Dalhousie University, Halifax, NS, Canada

    2Division of Pediatric Neurology, Dalhousie University, Halifax, NS, Canada

    *these authors contributed equally

    Corresponding Author:

    Aoife Marie O'Carroll, MD, FRCPC

    Dalhousie University

    Division of Pediatric Neurology, IWK Health Centre, Children's Site, 8th Floor

    5850-5980 University Avenue, PO Box 9700

    Halifax, NS,

    Canada

    Phone: 1 902 470 8475

    Fax: 1 902 470 8486

    Email: aoife.ocarroll@dal.ca


    ABSTRACT

    Background: Medical students face an information-rich environment in which retrieval and appraisal strategies are increasingly important.

    Objective: To describe medical students’ current pattern of health information resource use and characterize their experience of instruction on information search and appraisal.

    Methods: We conducted a cross-sectional web-based survey of students registered in the four-year MD Program at Dalhousie University (Halifax, Nova Scotia, and Saint John, New Brunswick, sites), Canada. We collected self-reported data on information-seeking behavior, instruction, and evaluation of resources in the context of their medical education. Data were analyzed using descriptive statistics.

    Results: Surveys were returned by 213 of 462 eligible students (46.1%). Most respondents (165/204, 80.9%) recalled receiving formal instruction regarding information searches, but this seldom included nontraditional tools such as Google (23/107, 11.1%), Wikipedia, or social media. In their daily practice, however, they reported heavy use of these tools, as well as EBM summaries. Accessibility, understandability, and overall usefulness were common features of highly used resources. Students identified challenges managing information and/or resource overload and source accessibility.

    Conclusions: Medical students receive instruction primarily on searching and assessing primary medical literature. In their daily practice, however, they rely heavily on nontraditional tools as well as EBM summaries. Attention to appropriate use and appraisal of nontraditional sources might enhance the current EBM curriculum.

    JMIR Medical Education 2015;1(1):e4

    doi:10.2196/mededu.4267

    KEYWORDS



    Introduction

    The information landscape is expanding rapidly, in large part due to the advent and evolution of the Internet. In the developed world, widespread access to the Internet along with intuitive and user-friendly search tools have made information on a vast range of topics available within moments. Some tools have gained such ubiquity as to become part of common parlance: “Google” is now a dictionary-approved verb [1].

    Medicine has seen a similar trend, and the volume of information is not unequivocally helpful to practice. Authors have previously cited the challenges of staying up to date on a multitude of articles [2] and guidelines [3]; today, even the tools to access evidence proliferate. Whereas evidence-based medicine (EBM) developed as an approach to manage the challenge of translating primary evidence into clinical practice, the field has evolved to define increasingly sophisticated approaches to the body of literature as a whole. Indeed, some authors suggest that information management training may be as important as instruction on searching the primary literature [4,5].

    Instruction in EBM is variable, however, and whether it affects long-term behaviors is uncertain [6-8]. The realities of daily work can present barriers to evidence-based practice, which is often perceived as a time- and effort-intensive pursuit [8-10]. Moreover, access to primary medical literature, summaries, and clinical support tools often comes with costly subscription fees.

    These challenges make the user-friendly, freely accessible tools that are useful for general purpose inquiries appealing. Indeed, studies of medical trainees and practicing physicians support the popularity of general purpose tools for clinical or academic queries [9,11,12]. Notwithstanding concerns regarding reliability, some evidence suggests that general resources such as Google can be effective in answering clinical questions [13-15].

    If these issues are important for the future of medicine, we need to understand how information-seeking behaviors develop. Medical school lays foundations of knowledge and behavior patterns. Students’ active engagement in participatory knowledge building is critical to this process, and thus information acquisition and use have central importance. Students turn to information sources to build background knowledge and will subsequently develop increasingly patient-specific questions and searches. As clinical encounters lead them to continually integrate this knowledge, they gradually build mental maps that enable automatic processing for quick clinical decision making [16]. Meanwhile, however, they must recognize the ongoing need to engage with information sources: to update their mental maps and to supplement when prior knowledge is absent or insufficient [16]. Their medical education must therefore prepare them as managers of information as well as experts in human health.

    Previous studies that focused on medical students in the developed world have considered the use of specific resources [17-19] or technology [20-22]. Others have reported on experimental educational interventions [23-25]. With the current study, we sought a holistic characterization of the information-seeking behaviors of a medical student cohort in the context of an existing formal EBM curriculum.

    We surveyed students enrolled in a medical doctorate training program regarding their use of Internet resources for medical information and the instruction they have received in EBM and information management. We hypothesized that this group would report high use of general purpose resources with minimal instruction in the use or interpretation of such resources.

    The primary objective of this study was to define students’ current patterns of resource use. We also sought to characterize student experiences of current informatics instruction. Lastly, we began an inquiry into student valuations of various resources for medical information.


    Methods

    Survey Instrument

    We developed a web-based survey to assess student information-seeking behavior, formal instruction on information searches, and evaluation of sources of health information. The survey was developed based on a review of relevant published literature and pretested with a convenience sample of ten medical students. The final survey contained 20 questions, predominantly requiring yes/no or rating scale responses; one question required a numeric estimate, one asked students to list their five most-used resources, and two were open-ended. Questions regarding resource use were based on recall of the previous seven days. The list of survey questions is included in Multimedia Appendix 1.

    Participant Recruitment

    All students registered in the Dalhousie University 4-year medical doctorate program, at either the Halifax or the Saint John site, were considered eligible for participation in the survey. At the time of the survey, there were 462 registered students. We considered students in years 1 and 2 preclinical and those in years 3 and 4 clinical because the first two years are classroom-based while the third and fourth years take place predominantly within clinical settings.

    The survey was published online using the Opinio 6 survey platform (ObjectPlanet, Inc) and remained open for two weeks. Two of the study authors (AO and EW) made in-class announcements to each student cohort, and invitation and reminder emails were sent to eligible students.

    Data Analysis

    The survey responses were exported from the survey platform into Excel (Microsoft Corp). Data were analyzed using descriptive statistics, computed by hand. Frequencies were reported as percentages. Where appropriate, 95% confidence intervals were calculated using the .cii command with exact binomial confidence intervals using STATA version 12.1 (StataCorp LP). No correction was made for multiple statistical testing.

    One question asked respondents to list their five most-used resources from the previous seven days. We assessed response frequency, and used Wordle [26] to generate a graphical representation of this data in which type size reflects frequency of occurrence.

    Selected data were subsequently graphed using R (The R Foundation). In keeping with a paper presented at the 2011 Joint Statistical Meeting [27], these Likert data were presented graphically, using diverging stacked bar charts across information sources.

    Open-ended questions were analyzed qualitatively for themes by AO. Responses were read and common themes identified; on second reading, responses were categorized into thematic groups. Responses that reflected more than one theme were included in each relevant thematic group.

    Ethics

    The study was formally reviewed and approved by the IWK Health Centre Research Ethics Board. We also received approval from the Dalhousie Undergraduate Medical Education Curriculum Committee.


    Results

    Participant Recruitment

    Of 462 students invited to respond, 213 (46.1%) provided evaluable responses. Two-thirds of closed-ended questions received response rates of 42.0% or above (193 or more students), and all but one question had responses from at least 39.0% (180 students). Ten students accessed the survey but did not provide any responses; these were considered to be nonrespondents and were not included in the response rate figures.

    Study Participants

    Of the respondents, 56.6% (120/212) were female and 42.0% (89/212) were male; 3 preferred not to answer this question. Preclinical and clinical training levels received comparable representation (110/212, 51.9%, and 102/212, 48.1%, respectively). Roughly half (109/207, 52.7%) had prior experience as contributors to peer-reviewed literature, while relatively few (35/207, 16.9%) had posted information online for the public.

    Instruction on Information Searching

    While 80.9% (165/204; 95% CI 74.8%-86%) of respondents recalled receiving formal instruction on searching for health information, 67.1% (139/207; 95% CI 60.3%-73.5%) also recalled being discouraged from using certain resources. Education regarding bibliographic databases (eg, PubMed) was common (162/206, 78.6%; 95% CI 72.4%-84.0%), whereas respondents seldom had instruction regarding general purpose Internet sources. Only 11.1% (23/207; 95% CI 7.2%-16.2%) reported teaching regarding the general search engine Google, and fewer had education on Wikipedia (a free, online, open-content encyclopedia) or social media. Although self-rated competence in finding information showed some variability, a strong majority (184/207, 88.9%; 95% CI 83.8%-92.8%) felt their skills were good or better.

    Pattern of Resource Use

    We considered a resource to be heavily used if a student reported use 4 or more times in the previous 7 days (Figure 1, Table 1). Most respondents use Google on a daily basis (154/202, 76.2%; 95% CI 69.8%-81.9%), and very few reported using the search engine on fewer than 4 of the prior 7 days (13/202, 6.4%; 95% CI 3.5%-10.8%). Wikipedia, UpToDate (a subscription-only, evidence-based summary source), and personal/provided notes were also heavily used. In contrast, few respondents (28/199, 14.1%; 95% CI, 9.6%-19.7%) had used bibliographic databases on at least 4 days, and just over one-third (68/199, 34.2%; 95% CI, 27.6%-41.2%) had not used such sources at all in the previous week.

    Comparison of preclinical- and clinical-level respondents revealed two prominent differences: clinical-level respondents reported less frequent use of bibliographic databases (P<.001) and more frequent use of UpToDate (P<.001) (Table 1).

    Of note, these results were somewhat different from respondents’ self-generated lists of most used resources. Here, UpToDate was the most-listed single reference, while Google and Wikipedia were the next most commonly cited resources (Figure 2).

    Table 1. Preclinical and clinical student use of resources during the previous 7 days.
    View this table
    Figure 1. Student self-reported use of resources in the previous 7 days.
    View this figure
    Figure 2. Student-identified top resources from the previous 7 days.
    View this figure

    Values and Reasons for Using Sources

    Whereas students valued the general purpose resources Google and Wikipedia highly for their accessibility, understandability, and usefulness, they gave PubMed and other bibliographic databases stronger ratings for accuracy and trustworthiness. As a representative EBM summary, UpToDate appeared to bridge this gap: it received high ratings across values although many students noted elsewhere that access to this resource was limited by cost and lack of an institutional subscription (Figure 3).

    Students identified key factors in their impressions of a source’s trustworthiness: recognition of factual errors in a reference and reputation among mentors and peers, as well as being specifically counseled to use or avoid a reference.

    Figure 3. Student evaluations of specified resources.
    View this figure

    Self-Identified Needs and Abilities

    Students offered valuable insights in their responses to open-ended questions, and several themes emerged. Many spoke of being overwhelmed by the number of available and/or suggested resources as well as by the density of information in these sources; respondents repeatedly reported a need for more basic information and for managing information/source overload. They requested faculty- and/or peer-generated resource recommendations, although some noted that current lists added to a sense of overload. Perhaps in response to this challenge, several requested increased and longitudinally integrated instruction on searching for information and often highlighted a desire to learn how to approach nontraditional, or general purpose, Internet sources. Finally, resource accessibility presented an additional, practical barrier to finding useful information. Respondents commonly identified a need for freer access, including at the point of care; they cited lack of universal Internet access, inconvenience of multiple sign-ins, and subscription-based access as barriers.


    Discussion

    Dalhousie University Medical School Curriculum

    Our survey assessed patterns of information resource use among students in the undergraduate MD program at Dalhousie University. Students in the program must have an undergraduate degree; some have additional study or work experience. The school has two campuses (Halifax, Nova Scotia, and Saint John, New Brunswick); the sites share a uniform curriculum, and many lectures are videoconferenced. All students have remote access to Dalhousie University library electronic resources throughout their training.

    The medical school follows a 4-year curriculum. Two classroom-based preclerkship years aim to provide a foundation of knowledge and basic skills. A two-year clerkship follows, spent mostly in clinical settings.

    A concentrated series of lectures on EBM are delivered at the beginning of year one. On a recent syllabus, seven hours during that unit were devoted to aspects of EBM, including five hours on question formulation and searching for evidence. Lecture notes refer to a standard hierarchy of preappraised evidence and discuss how and when to use various resources [Robin Parker, Sources of evidence, 10 September 2012, Dalhousie University, Canada].

    Information management and EBM continue to receive attention at various points after this introductory unit. Weekly sessions during the first two years address evidence search and appraisal and other topics (ethics, law, population health, and professionalism). The formal curriculum is more limited during the clinical years, and much of the planned teaching time addresses clinical skills; training objectives continue to refer to critical use and appraisal of evidence.

    Students have access to an array of resources and resource guides. They receive reference recommendations for each unit, and notes and links are posted on general and course-specific websites. The library website houses subject guides for each course and clinical rotation, including guides to search strategies for PubMed and Cochrane Library, a subscription-based set of medical and healthcare databases.

    Principal Findings

    Behavior Diverges From Formal Instruction

    In this survey, most students recalled having received formal instruction on health information searches during their training, most of which concerned traditional information sources such as PubMed or other bibliographic databases. Although students commonly reported being discouraged from using certain resources and seldom recalled instruction in the use of general purpose search resources (such as Google), they reported Google as the most frequently used in their actual practice. The contrast between self-reported formal instruction and practice may have several explanations. Medical education takes place on multiple levels: while the formal curriculum transmits intentional and explicit instruction, informal and hidden curricula operate through experience, interactions, and role modeling [28]. A substantial body of literature avers the importance of these latter curricula on the emerging professional and ethical identities of the students. Our respondents likewise rated reputation among peers and mentors as an important factor in their valuation of information sources. Informal curricula may well influence learning behaviors as much as they do ethical ones. If so, given the high rates of general purpose and preappraised resource use among medical residents and qualified physicians [9,11,12]—students’ chief mentors during training—it is not surprising that these students should so frequently use such resources. Moreover, increased informal, experiential interaction with these mentors during the clinical training years would be expected to result in increased use of role-modeled resources.

    Changing Patterns of Resource Use

    Students in their clinical years used Google heavily but reported increased use of UpToDate and less reference to bibliographic databases compared to their preclinical peers. With the shift to a clinical setting, students’ information needs change: they require more patient-oriented information, whereas preclinical students must accumulate basic knowledge of physiology and disease. As well, the real-world, clinical setting demands increased search efficiency, often making review of individual studies impractical [4]. Such needs may have influenced the students in this study and have spurred an industry of preappraised, summarized information sources (including UpToDate). Although criticized for variable design and timeliness [29,30], preappraised summaries have proven superior to primary literature searches in some cases [31,32] and are commonly considered an important part of evidence-based practice [33].

    Access to EBM summaries may have been a contributing factor to the seniority-related variation in resource use. Anecdotally, several students told us that they purchase or share subscriptions to resources, including UpToDate, upon entering clerkship. While our university does not hold a license for UpToDate—the EBM summary most frequently named by students—some clinical settings do.

    Meanwhile, the fact that students less commonly named evidence summaries to which our university does offer access is significant. It may reflect the strength of peer and mentor influences. It could also speak to the problem of information resource overload. Excess information may lead to errors of omission in the clinical context [34,35]: with such a large variety and volume of resources, information may be lost. One medical librarian’s observation that “students may not have an accurate picture of the access they’re entitled to” [personal communication by Kathleen Gadd, 9 March 2015] lends support to this possibility.

    Resource Selection: A Balance of Needs

    The values respondents attributed to various information sources identify ease of understanding and accessibility as common features among the more heavily used resources. Meanwhile, their comments highlight challenges navigating a surplus of information and managing source accessibility. Their practice appears to reflect a trade-off between accessibility/digestibility and accuracy: students appear to believe the balance of benefit lies with summary sources, whether medical or general purpose.

    Such calculations constitute a satisficing approach to searching and source selection. Faced with practical limits on obtaining and analyzing a large volume of relevant data, individuals select what they perceive to be a good enough option [36-38]—for example, selecting Google or UpToDate to locate a piece of needed information instead of conducting a thorough literature review. The concern, of course, is whether students are equipped to know and choose what is actually good enough. Can they appraise the source and the information,adequately to determine whether it meets minimum standards? An optimistic view would hold that this is the case. On the other hand, given the lack of instruction on use and appraisal of their most-used sources, we might question the bases for their strategies and consider the risks of uncritical information seeking.

    Student Self-Assessment

    A strong majority of respondents rated their information seeking skills as good or better—this despite reporting information search practices that diverged from the formal instruction they had received. Perhaps this is because they achieve good results with their current search practices. It is worth noting, however, the limits inherent to self-assessment. Individual self-rating bears little relation to actual competence, and most students overestimate their performance; this is demonstrable among medical students, among others [39-41]. Individuals who believe they have adequate skills are unlikely to seek remediation, whether or not they are truly competent. Furthermore, the students in this survey reported feeling overwhelmed by the information landscape; we know medical professionals consistently choose CME activities that address interests and skills rather than weaknesses and can surmise that medical student behavior is likely similar. It seems unlikely that these students will independently seek out ways to improve their skills, now or possibly even into their careers. If there is a better approach to seeking information, medical educators must offer active guidance.

    Study Limitations

    Our study has limitations. We conducted the survey at a single medical school. The response rate was lower than desired. Our response rate appears comparable, however, to that achieved by previous online surveys addressing technology and information management in medicine [20-22,42].

    Our survey relied on self-reported behaviors, which are known to be subject to social desirability bias. External observation of behavior, however, is not feasible for the current inquiry. A daily diary approach might have offered greater accuracy but at a further cost to the response rate. We asked respondents to report practices from the preceding 7 days as a strategy to obtain more accurate self-reported data than a more general inquiry. Despite this, the difference seen between the frequencies of use reported for different resources and respondents' self-generated lists of most-used resources suggested the possibility of bias in some questions. We would, however, have expected social desirability bias to skew results away from non-traditional, general-purpose sources—instead we saw quite a dramatic favoring of such resources over bibliographic databases.

    Our survey obtained a cross-sectional assessment, with reference to behavior during the preceding seven days. It is possible that student behavior differed from usual practice during the week prior to the survey, rendering nonrepresentative results. In an effort to limit this possibility, we discussed survey timing with the Dalhousie Undergraduate Medical Education Curriculum Committee and student leaders and selected survey dates that fell during the routine session, avoiding examination or recess periods.

    Conclusions

    No tool is optimal for every purpose. Students need to gain skills and familiarity assessing primary literature, but they also must learn to find useful, practical information efficiently. They need to recognize what tool will best serve a given purpose—whether this is getting an overview of a common condition or physiologic process or assessing effectiveness of alternate medical therapies—and to appreciate the limitations inherent to that tool. Thus, whereas current formal instruction on information searching neither reflects nor appears to alter,student behavior, consideration should be given to instruction on information management and appraisal in general. Medical education that includes use and appraisal of primary literature, summary sources, and even general purpose resources might more effectively equip students in their pursuit of lifelong learning.

    Acknowledgments

    This study was unfunded. AO is a subspecialty resident physician at the IWK Health Centre/Dalhousie University. EW is a third-year medical student at Dalhousie University. KEG and JD are staff physicians at the IWK Health Centre and faculty members with the Department of Pediatrics, Dalhousie University. AO contributed to the study’s conception, survey design, data acquisition, analysis, and interpretation; and wrote the manuscript. EW contributed to survey design and implementation and manuscript review. JD and KEG offered supervision to the project and advised regarding study design, data interpretation and statistical analysis, and manuscript review. AO and KEG had full access to all the data in the study, conducted, and take responsibility for the integrity of the data and the accuracy of the data analysis.

    We appreciate support received from other members of Dalhousie University. Poh Chua (Information Technology Services, Dalhousie University) gave technical support. Sylvia Bartlett and Carolyn Pelham (Dalhousie Undergraduate Medical Education administration) and Leo Fares (Dalhousie Medical Student Society President 2013-14) helped to coordinate the survey. Alicia Eakins (UGME Administrator, Dalhousie University) provided curriculum information. Librarians at Dalhousie and its affiliated health care centers gave information on resource availability: Patrick Ellis (Associate University Librarian Resources; Head, W.K. Kellogg Health Sciences Library, Dalhousie University), Jackie Phinney (Information Services Librarian, Dalhousie Medicine NB), Darlene Chapman (Manager, Library and Audiovisual Services, IWK Health Centre, Halifax, NS), Kathleen Gadd (Librarian, Horizon Health Network, NB), and Katie McLean (Librarian Educator; Vice-President of Maritime Health Libraries Association, Capital District Health Authority, NS).

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Survey instrument.

    PDF File (Adobe PDF File), 123KB

    References

    1. Merriam-Webster. Google   URL: http://www.merriam-webster.com/dictionary/google [accessed 2015-06-19] [WebCite Cache]
    2. Davidoff F, Haynes B, Sackett D, Smith R. Evidence based medicine. Brit Med J 1995 Apr 29;310(6987):1085-1086 [FREE Full text] [Medline]
    3. Allen D, Harkins KJ. Too much guidance? Lancet 2005;365(9473):1768. [CrossRef] [Medline]
    4. Slawson DC, Shaughnessy AF. Teaching evidence-based medicine: should we be teaching information management instead? Acad Med 2005 Jul;80(7):685-689. [Medline]
    5. Shaughnessy AF, Gupta PS, Erlich DR, Slawson DC. Ability of an information mastery curriculum to improve residents' skills and attitudes. Fam Med 2012 Apr;44(4):259-264 [FREE Full text] [Medline]
    6. Horsley T, Hyde C, Santesso N, Parkes J, Milne R, Stewart R. Teaching critical appraisal skills in healthcare settings. Cochrane Database Syst Rev 2011(11):CD001270. [CrossRef] [Medline]
    7. Maggio LA, Tannery NH, Chen HC, ten Cate O, O'Brien B. Evidence-based medicine training in undergraduate medical education: a review and critique of the literature published 2006-2011. Acad Med 2013 Jul;88(7):1022-1028. [CrossRef] [Medline]
    8. Yew KS, Reid A. Teaching evidence-based medicine skills: an exploratory study of residency graduates' practice habits. Fam Med 2008 Jan;40(1):24-31 [FREE Full text] [Medline]
    9. Kritz M, Gschwandtner M, Stefanov V, Hanbury A, Samwald M. Utilization and perceived problems of online medical resources and search tools among different groups of European physicians. J Med Internet Res 2013;15(6):e122 [FREE Full text] [CrossRef] [Medline]
    10. van Dijk N, Hooft L, Wieringa-de Waard M. What are the barriers to residents' practicing evidence-based medicine? A systematic review. Acad Med 2010 Jul;85(7):1163-1170. [CrossRef] [Medline]
    11. Duran-Nelson A, Gladding S, Beattie J, Nixon LJ. Should we Google it? Resource use by internal medicine residents for point-of-care clinical decision making. Acad Med 2013 Jun;88(6):788-794. [CrossRef] [Medline]
    12. Hughes B, Joshi I, Lemonde H, Wareham J. Junior physician's use of Web 2.0 for information seeking and medical education: a qualitative study. Int J Med Inform 2009 Oct;78(10):645-655. [CrossRef] [Medline]
    13. Johnson PT, Chen JK, Eng J, Makary MA, Fishman EK. A comparison of world wide web resources for identifying medical information. Acad Radiol 2008 Sep;15(9):1165-1172. [CrossRef] [Medline]
    14. Thiele RH, Poiro NC, Scalzo DC, Nemergut EC. Speed, accuracy, and confidence in Google, Ovid, PubMed, and UpToDate: results of a randomised trial. Postgrad Med J 2010 Aug;86(1018):459-465. [CrossRef] [Medline]
    15. Kim S, Noveck H, Galt J, Hogshire L, Willett L, O'Rourke K. Searching for answers to clinical questions using google versus evidence-based summary resources: a randomized controlled crossover study. Acad Med 2014 Jun;89(6):940-943 [FREE Full text] [CrossRef] [Medline]
    16. Hutchinson A, Maskrey N, Slawson D, Shaughnessy A, Underhill J. The Essential Handbook for GP Training and Education. 2012. Information mastery: decision-making and dealing with information overload   URL: http://www.essentialgptrainingbook.com/resources/web_chapter_05/05%20information%20mastery.pdf [accessed 2015-04-26] [WebCite Cache]
    17. Allahwala UK, Nadkarni A, Sebaratnam DF. Wikipedia use amongst medical students: new insights into the digital revolution. Med Teach 2013 Apr;35(4):337 [FREE Full text] [CrossRef] [Medline]
    18. Usher K, Woods C, Casellac E, Glass N, Wilson R, Mayner L, et al. Australian health professions student use of social media. Collegian 2014 Jun;21(2):95-101. [Medline]
    19. Khalifian S, Markman T, Sampognaro P, Mitchell S, Weeks S, Dattilo J. Medical student appraisal: searching on smartphones. Appl Clin Inform 2013;4(1):53-60 [FREE Full text] [CrossRef] [Medline]
    20. Boruff JT, Storie D. Mobile devices in medicine: a survey of how medical students, residents, and faculty use smartphones and other mobile devices to find information. J Med Libr Assoc 2014 Jan;102(1):22-30 [FREE Full text] [CrossRef] [Medline]
    21. Payne KFB, Wharrad H, Watts K. Smartphone and medical related App use among medical students and junior doctors in the United Kingdom (UK): a regional survey. BMC Med Inform Decis Mak 2012;12:121 [FREE Full text] [CrossRef] [Medline]
    22. Han H, Nelson E, Wetter N. Medical students' online learning technology needs. Clin Teach 2014 Feb;11(1):15-19. [CrossRef] [Medline]
    23. Friederichs H, Marschall B, Weissenstein A. Practicing evidence based medicine at the bedside: a randomized controlled pilot study in undergraduate medical students assessing the practicality of tablets, smartphones, and computers in clinical life. BMC Med Inform Decis Mak 2014;14:113 [FREE Full text] [CrossRef] [Medline]
    24. Boruff JT, Bilodeau E. Creating a mobile subject guide to improve access to point-of-care resources for medical students: a case study. J Med Libr Assoc 2012 Jan;100(1):55-60 [FREE Full text] [CrossRef] [Medline]
    25. Rees E, Sinha Y, Chitnis A, Archer J, Fotheringham V, Renwick S. Peer-teaching of evidence-based medicine. Clin Teach 2014 Jul;11(4):259-263. [CrossRef] [Medline]
    26. Feinberg J. Wordle, Build 1439.: Jonathan Feinberg   URL: http://www.wordle.net/ [accessed 2015-06-19] [WebCite Cache]
    27. Robbins N, Heiberger R. Plotting Likert and other rating scales. 2011 Presented at: Proceedings of the Joint Statistical Meeting (Section on Survey Research Methods); 2011 July 30-August 4; Miami Beach, FL   URL: http://www.amstat.org.ezproxy.library.dal.ca/membersonly/proceedings/2011/papers/300784_64164.pdf
    28. Hafferty FW, Franks R. The hidden curriculum, ethics teaching, and the structure of medical education. Acad Med 1994 Nov;69(11):861-871. [Medline]
    29. Jeffery R, Navarro T, Lokker C, Haynes RB, Wilczynski NL, Farjou G. How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks. J Med Internet Res 2012;14(6):e175 [FREE Full text] [CrossRef] [Medline]
    30. Banzi R, Cinquini M, Liberati A, Moschetti I, Pecoraro V, Tagliabue L, et al. Speed of updating online evidence based point of care summaries: prospective cohort analysis. Brit Med J 2011;343:d5856 [FREE Full text] [Medline]
    31. Sayyah EL, Faghankhani M, Javanbakht A, Ahmadi S, Baradaran HR. To compare PubMed Clinical Queries and UpToDate in teaching information mastery to clinical residents: a crossover randomized controlled trial. PLoS One 2011;6(8):e23487 [FREE Full text] [CrossRef] [Medline]
    32. Hoogendam A, Stalenhoef AFH, Robbé PF, Overbeke AJPM. Answers to questions posed during daily patient care are more likely to be answered by UpToDate than PubMed. J Med Internet Res 2008;10(4):e29 [FREE Full text] [CrossRef] [Medline]
    33. DiCenso A, Bayley L, Haynes RB. ACP Journal Club. Editorial: Accessing preappraised evidence: fine-tuning the 5S model into a 6S model. Ann Intern Med 2009 Sep 15;151(6):JC3-2, JC3. [CrossRef] [Medline]
    34. Ahmed A, Chandra S, Herasevich V, Gajic O, Pickering BW. The effect of two different electronic health record user interfaces on intensive care provider task load, errors of cognition, and performance. Crit Care Med 2011 Jul;39(7):1626-1634. [CrossRef] [Medline]
    35. Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF. Information overload and missed test results in electronic health record-based settings. JAMA Intern Med 2013 Apr 22;173(8):702-704 [FREE Full text] [CrossRef] [Medline]
    36. Simon HA. A Behavioral Model of Rational Choice. Q J Econ 1955 Feb;69(1):99. [CrossRef]
    37. Simon HA. Rational choice and the structure of the environment. Psychol Rev 1956 Mar;63(2):129-138. [Medline]
    38. Maskrey N, Underhill J, Hutchinson A, Shaughnessy A, Slawson D. Getting a better grip on research: the maze of the most busy life. InnovAiT 2010 Feb 11;3(3):172-179. [CrossRef]
    39. Regehr G, Eva K. Self-assessment, self-direction, and the self-regulating professional. Clin Orthop Relat Res 2006 Aug;449:34-38. [CrossRef] [Medline]
    40. Eva KW, Cunnington JPW, Reiter HI, Keane DR, Norman GR. How can I know what I don't know? Poor self assessment in a well-defined domain. Adv Health Sci Educ Theory Pract 2004;9(3):211-224. [CrossRef] [Medline]
    41. Ivanitskaya L, O'Boyle I, Casey AM. Health information literacy and competencies of information age students: results from the interactive online Research Readiness Self-Assessment (RRSA). J Med Internet Res 2006;8(2):e6 [FREE Full text] [CrossRef] [Medline]
    42. Sandars J, Schroter S. Web 2.0 technologies for undergraduate and postgraduate medical education: an online survey. Postgrad Med J 2007 Dec;83(986):759-762 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    EBM: evidence-based medicine


    Edited by G Eysenbach; submitted 21.01.15; peer-reviewed by L Maggio, A Shaughnessy; comments to author 03.03.15; revised version received 06.04.15; accepted 02.05.15; published 29.06.15.

    ©Aoife Marie O'Carroll, Erin Patricia Westby, Joseph Dooley, Kevin E Gordon. Originally published in JMIR Medical Education (http://mededu.jmir.org), 29.06.2015.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on http://mededu.jmir.org/, as well as this copyright and license information must be included.