This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on http://mededu.jmir.org/, as well as this copyright and license information must be included.
Online medical education has relevance to public health literacy and physician efficacy, yet it requires a certain standard of reliability. While the internet has the potential to be a viable medical education tool, the viewer must be able to discern which information is reliable.
Our aim was to perform a literature review to determine and compare the various methods used when analyzing YouTube videos for patient education efficacy, information accuracy, and quality.
In November 2016, a comprehensive search within PubMed and Embase resulted in 37 included studies.
The review revealed that each video evaluation study first established search terms, exclusion criteria, and methods to analyze the videos in a consistent manner. The majority of the evaluators devised a scoring system, but variations were innumerable within each study’s methods.
In comparing the 37 studies, we found that overall, common steps were taken to evaluate the content. However, a concrete set of methods did not exist. This is notable since many patients turn to the internet for medical information yet lack the tools to evaluate the advice being given. There was, however, a common aim of discovering what health-related content the public is accessing, and how credible that material is.
In today’s world, the internet and social media are a part of everyday life. Within seconds, a handheld device can provide more information than one can possibly read. The ease and simplicity of finding information on the internet translates directly to answering health questions and concerns. By 2011, 59% of adults were looking up health information online, and internet access has expanded exponentially since then [
A thorough search was performed within both Embase and PubMed in November 2016. A data management librarian determined the search terms after a preliminary search to find which key words would supply relevant articles. Many search combinations did not generate any articles as this is a relatively new topic and YouTube was not created until 2005. Thus, our inclusion date for articles encompassed anything published after the year 2005. PubMed and Embase were chosen as the literature databases to search, as they are reputable sources of medical literature and PubMed also includes literature from the Medline database. The first search was performed in Embase with the term “patient education” AND “YouTube” OR “Online Videos” OR “Online video.” In PubMed, two separate searches were performed. The first search term was (“Patient Education as Topic” [Medical Subject Headings] OR “patient education”) AND (“YouTube” OR “online videos”), and the second search term was “YouTube health guidelines.” One author analyzed all of the included articles, and the results were reviewed and approved by another author. Each included article was read in its entirety, and the methods as well as unique characteristics for each study were recorded in MS Excel formatting and compared.
The inclusion criteria for the studies to be reviewed were as follows: (1) analysis of videos intended for patients or guardians, (2) contains detailed and repeatable methods of analysis, (3) English language, and (4) analysis of videos that are made available to the public.
The Embase search (“patient education” AND “YouTube” OR “Online Videos” OR “Online video”) generated 65 results, of which 20 were included for review. The first PubMed search (“Patient Education as Topic” [Medical Subject Headings] OR “patient education”) AND (“YouTube” OR “online videos”) had 77 results and 13 articles met the inclusion criteria. The last PubMed search (“YouTube health guidelines”) gave 16 results, of which 4 articles were reviewed. This resulted in a total of 37 studies to be reviewed (see
This study was exempt from requiring Institutional Review Board approval.
Search results and excluded studies.
The same chronological process was generally followed within each piece of literature reviewed, but no two video evaluations were performed in an identical manner. The first step within each study included determining the search term(s) to be used. Multiple search terms were used to ensure that all possible patient searches could be evaluated. For example, “gallbladder disease,” “gallstone disease,” and “gallstone treatment” were all used to assess YouTube videos about gallstone disease, as they are likely terms used by the public [
Methods and techniques used to determine search terms and searching criteria for YouTube videos included multiple search terms (20 studies), autocomplete function within search bar, use of search operators, multiple search dates, limited number of pages within a particular search to be analyzed (30 studies), and changing the video results to be sorted by “most viewed” (3 studies).
The next step involves determining which videos should be included in the study. Some researchers set a maximum time limit for included videos. Only one study excluded videos based on number of views, in which the videos were required to have greater than 2500 views [
Most studies had multiple reviewers and stated the qualifications of the reviewers, which included students, residents, or physicians. The most rigorous qualification requirements involved a 1-month clinical rotation in the department of allergy and clinical immunology and successful completion of a series of learning objectives [
The source of upload allowed for categorization of videos. An analysis of educational videos on children’s dental caries separated the videos into health care professionals, academic institutes, professional organizations, individual users, and product companies [
To determine the accuracy of the videos, 22 of the studies created a novel scoring system. These scoring systems and other methods are summarized in
Ten studies evaluated the quality of the video presentation, of which five assessed video quality according to global quality score guidelines. This rates the quality from a score of 1-5 while taking into account video flow and usefulness [
The most common video characteristics recorded were number of views, followed by source of upload. These data, along with the frequencies of other parameters taken into account by the various studies, are summarized in
The most common sequence of methods performed is as follows: (1) determine a search term(s); (2) establish inclusion criteria for videos; (3) determine video reliability scoring/what parameters will be taken into account; (4) review videos individually; (5) convene to discuss discrepancies and determine final results; and (6) analyze results and determine the reliability or usefulness of videos and which characteristics determine that quality.
Methods for determining accuracy and usefulness of videos.
Method | Description |
Creation of a novel scoring system (22 studies) | Formation of guidelines based on scientific literature and physician expertise with a corresponding point system |
HONcodea | Health on the Net Foundation guidelines for websites adapted for YouTube videos |
Adaptation of these guidelines to be implemented for YouTube videos | |
Judgment as useful, misleading, or personal experiences | Subjective categorization by the researchers based on knowledge of the topic as well as on predetermined criteria |
aHealth on the Net (HON) Foundation created a set of 8 principles for websites to abide by called the HONcode.
Video data collected by various studies as of Nov 2016 (results based on all 37 studies reviewed).
Our review found that defining a search term, determining how to judge or score the videos, and determining the reliability of the video sources and information were the primary methods discussed throughout the studies. There were also many steps taken to ensure that the evaluations were indicative of how the general public and patients would receive and understand the information given in the YouTube videos. For example, YouTube content is constantly being modified, thus researchers performed content searches at later dates to give insight into the evolution of viewership [
Throughout the reviewed literature, there was considerable focus on determining which video characteristics could be quantified and compared to reveal a positive correlation with video accuracy. The most commonly statistically analyzed parameters were video score versus number of likes, and video score versus source of upload. One study discovered that younger patients as well as patients with higher education are more likely to use the internet as source of health information due to their increased ability to search the Web and identify reliable information and sources [
This paper is a comprehensive review, but it is not a systematic review. All efforts were taken to include all articles possible, but we cannot guarantee that some were not missed. In addition, this is a newly popular topic and it is likely that use of these search terms at a later date will result in an increased number of results.
Social media has the potential to aid in closing the health literacy gap and can present information in novel ways that allow even illiterate populations to learn [
Titles of the included studies that complied with all inclusion criteria.
Health on the Net
BD is the primary author of the paper and performed the review and analysis. EC determined the search terms and the databases employed. AS developed the concept and approved the final draft for submission.
None declared.