On July 1, 2021, the digital Web of Science (WoS) core collection database was accessed and queried with the following search string: TOPIC: (“eTeaching*” OR “online teaching*” OR “E-teaching*” OR “electronic teaching*” OR “computer-assisted teaching*” OR “computer-mediated teaching*” OR “computer-based teaching*” OR “digital teaching*” OR “online course*” OR “eLearning*” OR “online learning*” OR “E-learning*” OR “electronic learning*” OR “computer-assisted learning*” OR “computer-mediated learning*” OR “computer-based learning*” OR “digital learning*”) AND TOPIC: (“medic*”). The query identified publications mentioning these words and their derivatives in the title, abstract and, keywords. The “Analyze” and “Create Citation Report” functions of WoS were deployed for initial analyses and frequency counting. The full records of the resultant publications were exported to VOSviewer, version 1.6.15 (Leiden University) for further bibliometric analyses. Normalized citations were computed in VOSviewer by considering the average number of citations received by the documents published in the same year and included in the data set (a score of >1 indicates higher-than-average citations compared with the documents published in the same year). As an exploratory analysis, we further analyzed publications with authors based on low-income economies according to the World Bank [22].

The VOSviewer [23] generated bubble maps that visualized the recurring terms and their citation per publication (CPP). Terms that appeared in at least 1% of the analyzed publications (n≥40) were visualized. Similarly, author keywords that appeared in at least 3 publications were visualized.

Two-sample t tests were conducted to analyze if the CPP showed a significant difference between original articles and reviews, and between open access (OA) and non-OA papers. Statistical tests were performed with SPSS, version 26.0 (IBM Corp). The results were deemed significant if P<.05.

Our literature search yielded a total of 3978 documents, which collectively accumulated 35,104 citations (Figure 1), reflecting a CPP of 8.82 and h-index of 65. The first paper on this topic was published in Lancet in 1976, reporting the experimentation of computer-assisted learning among 5th year medical students at Glasgow University [24]. The study reported that 79 out of 80 students were keen to have further such tuition. Meanwhile, in 2020, when the COVID-19 pandemic affected the whole world, the yearly publication count on digital teaching suddenly increased to 515, up from 200-300 in the prior 7 years. About 72.1% of the retrieved documents were original articles (n=2870, CPP=9.8), whereas 6.0% were reviews (n=239, CPP=21.0). The remaining were mainly proceedings papers, editorial materials, and meeting abstracts. Hence, the article-to-review ratio was 12:1. Less than half (41.5%) were OA (n=1649, CPP=9.3), whereas over half were non-OA (n=2329, CPP=8.5). Reviews were significantly more often cited (P<.001) than original articles. Moreover, the citations between OA vs non-OA papers did not significantly differ (P=.331).

The most productive authors, organizations, journals, and journal categories are listed in Table 1. The contributors were mostly from Europe and North America. The reports were mainly published in medical education journals.

The recurring terms in the titles and abstracts of the papers are depicted in Figure 2. Some themes were more highly cited (yellow bubbles), including general terms such as innovation (n=129 [3.2%], CPP=16.4), trial (n=220 [5.5%], CPP=14.3), effectiveness (n=474 [11.9%], CPP=14.8); terms describing modalities of digital teaching such as virtual reality (n=47 [1.2%], CPP=16.9), simulation (n=241 [6.1%], CPP=12.6), and massive open online course (MOOC), n=57 [1.4%], CPP=11.8); terms characterizing teaching disciplines such as anatomy (n=163 [4.1%], CPP=16.2), nursing (n=122 [3.1%], CPP=13.3), and surgery (n=129 [3.2%], CPP=8.8). It seemed that more studies were conducted on trainee (n=198 [5%], CPP=8.5) than undergraduate student (n=62 [1.6%], CPP=10.1). The recurring author keywords are depicted in Figure 3A (note that, for clarity, the following dominating keywords were omitted from the figure: e-learning [n=1010], medical education [n=500], education [n=352], online learning [n=240], blended learning [n=162], and elearning [n=108]). Different aspects in medical education were implied in digital teaching, such as gross anatomy education (n=50 [1.3%], CPP=31.0), histology (n=14 [0.4%], CPP=17.7), complementary medicine (n=6 [0.2%], CPP=3.0), medicinal chemistry (n=17 [0.4%], CPP=5.1), and basic life support (n=4 [0.1%], CPP=6.0). The term “COVID-19” had a rather low CPP. If we computed average normalized citations by normalizing the citations by the mean number of citations received by the documents published in the same year and included them in the data set, the recency yet importance of COVID-19 could be illustrated (normalized citation=1.95, where the citation rate achieved is equal to 1) (Figure 3B). Top 10 recurring keywords are listed in Table 2.

For completeness, Table 3 lists the top 10 most cited papers based on total and yearly citation count, respectively. Ranking by yearly citation count showed that many of the top 10 papers concerned COVID-19.

By examining the data, we noticed that one of the keywords with highest CPP was “low- and middle-income countries” (n=3 [0.1%], CPP=57.7), listed by 2 original articles and 1 review [31,41,42]. Hence, we searched for this phrase in the entire data set (not limited to author keywords) and identified 19 original articles (CPP=6.9) and 7 reviews (CPP=44.1). These numbers suggested that the original research works on this aspect were not highly cited. For instance, the most cited work was a survey among students, residents, and lecturers in a medical faculty in Cameroon (38 citations) [43]. This work found that 84% of students and 58% of residents never had access to digital teaching resources but viewed digital teaching positively and wished to have it in their school. Thus, a high need for digital resources for medical teaching exists, at least in some parts of the world. To address such needs, the University of Dundee and the British Society for Antimicrobial Chemotherapy developed a MOOC on microbiology to cater education need in low- and middle-income countries, and found that 13% of participants were from Africa, 16% from Asia, 8% from Australia, 49% from Europe, 9% from North America, and 5% from South America [44].

Regarding publications with authors based in low-income economies according to the World Bank [22], we were able to identify a total of 29 publications from low-income economies with 262 citations, a CPP of 9.0, and an h-index of 7. A publication was included if it had an author based in low-income economies, irrespective of their position in the author list. The first document was published in 2009, an editorial that introduced a web-based learning environment by Omdurman Islamic University in Sudan [45]. The United States was involved in 8 (27%) of these 29 papers, whereas Sudan (n=7 [24%]), Ethiopia (n=6 [21%]), and Uganda (n=5 [17%]) were the most productive low-income countries. The most productive organization was Addis Ababa University in Ethiopia (n=5 [17%]), and the most productive journal was BMC Medical Education (n=3 [10%]). Figure 4 shows the recurring terms in the titles and abstracts (n≥2) from these publications. Terms reflected more basic concepts, such as resource-limited setting (n=2 [7%], CPP=0), computer (n=2 [7%], CPP=89.0), and medical education partnership initiative (n=4 [14%], CPP=5.0). Meanwhile, Table 2 shows that COVID-19 and training were recurring keywords shared by these papers and the entire data set.

This bibliometric analysis of 3978 publications on digital education research in medicine revealed that this field began to grow rapidly in terms of both publication and citation counts in the 2000s. Original articles accounted for 72.1% of the literature. The article-to-review ratio was 12:1. This ratio was higher than that for literature sets of virtual reality application in medicine (5.9:1) [21] and medical errors (8.1:1) [19]. This implied that researchers working in digital teaching in medical education tended to conduct original research and find novelty instead of summarizing evidence from existing literature. The literature had heavy contributions from North America, Europe, and Oceania. Low-income countries together accounted for 0.9% of the publications, and their works were infrequently cited. This situation was similar in the emergency medicine literature, for which low-income countries published only 0.1% of all articles [46]. The small contribution by low-income countries was also identified in cardiovascular [47] and anesthesia literature [48].

With the current COVID-19 pandemic, digital teaching could prove itself very beneficial for medical education. As a reflection of these benefits and the wide application of digital education during the pandemic, COVID-19 was the most frequently occurring author keyword in the analyzed literature data set (Table 2). During the COVID-19 pandemic, many reports were published to share the perspectives as well as the impact and challenges of a sudden switch to digital teaching in the local settings, such as in Malaysia [49], Jordan [50], and Saudi Arabia [51]. Importantly, not all populations can readily access the internet for digital teaching. In Jordan, a survey on 652 medical students found that the overall satisfaction rate in medical digital teaching was only 26.8%, because 69.1% of them faced a main challenge of internet streaming quality and coverage [50]. Based on a focus group interview of 60 students, medical students in Saudi Arabia also faced some internet connection and synchronicity issues, but digital teaching was well accepted overall [51]. In Europe, poor internet connection was encountered by only 21.5% of 2721 surveyed medical students [52]. Although digital teaching can take place in many formats, internet accessibility remains to be a mandatory requirement. In countries and regions where medical students cannot connect to the internet anytime and anywhere, perhaps asynchronous formats will be more suitable, such as a MOOC course. A MOOC course that teaches emergency medical practice may deliver the teaching with good student satisfaction and, at the same time, effectively reduce the cross-infection risk between teachers, clinical staff, medical students, and patients [53].

The following discussion covers the principal findings from the most cited original articles in the literature set. For instance, in the setting of problem-based learning (PBL), a blended version was found to be as effective as the traditional face-to-face approach in terms of test results; it was also significantly superior in terms of subjective learning gains and satisfaction with easy revisits to the web-based learning modules [54]. A similar study on learning acid-base physiology found that students not only had higher satisfaction regarding the web-based PBL compared with the traditional PBL, but also a significantly higher test score with a medium effect size [55]. Performance enhancement was similarly observed for basic life support learning with web-based virtual patient encounters over standard training [56]. Regarding the web tools, it was advocated that YouTube (YouTube LLC) could be a very useful platform to store and disseminate tailor-made teaching videos such as those dedicated to human anatomy [57]. Moreover, a learning period as short as 30 minutes with a mobile phone with augmented reality blended learning environment could already bring about a greater knowledge gain than a traditional textbook [58]. Another benefit of digital teaching was cost-effectiveness: it was estimated that blended learning could cut costs by 24% compared to the traditional face-to-face approach [59]. However, initial costs of creation and preparation of digital teaching tools, including provision of an adequate information technology environment, may exceed those of traditional face-to-face approaches and may therefore act as a possible barrier to implementation.

Taken together, this short overview of the most cited original articles in the analyzed literature set is illustrative of the diversity of digital tools that could be used for medical education and the benefits that they are offering.

To the authors’ knowledge, no previous bibliometric analysis on digital teaching research in the medical literature has been published. A recent study on 10 e-learning journals found that the field shifted its foci from online learning, distance education, and pedagogical issues to mobile learning and interactive learning environments [60]. Meanwhile, research on e-learning in higher education was predominated by Spain and published in EDULEARN Proceedings [61]. These entities ranked 6th and 3rd in our predefined literature set, as listed in Table 1. By contrast, current results were consistent to a general e-learning literature analysis that identified the United States and the United Kingdom as the most productive countries; however, in both countries, chemical and engineering journals predominated instead of medical education ones [62,63]. Further, it was found that e-learning literature could be clustered into 7 foci: social sciences, psychology, medicine, health professions, life sciences, physical sciences or engineering, and computer science [64]. Moreover, it seemed that the contribution of computer science was on the decline whereas that of social sciences was gradually increasing in the scientific literature on digital education [65]. Finally, the current results were similar to that of the health sciences literature, where BMC Medical Education and Medical Teacher were the top 2 most productive journals, and COVID-19 was one of the most frequently mentioned keywords [66].

Some papers might not be indexed by WoS and may thus be missed from the analysis presented in this study. Besides, it was not possible to assess the methodology and reporting quality of each experimental study due to the large number of publications involved. Moreover, the contributions from the low-income countries might be underestimated, as their works might be published in local journals or journals not indexed by WoS. This limitation could be partially addressed by extending the search to other databases such as Scopus and Education in Africa, hosted by AfricaBib. However, different databases count publication and citation data differently, which hinders merging such data for the kind of bibliometric analysis applied in our work. Nevertheless, readers should be aware that searching other databases with the same search string is expected to result in additional relevant publications (eg, identical PubMed search identified 5383 papers as compared with the 3978 papers identified in WoS, which was analyzed in this study) since WoS has more stringent requirements for indexing, requiring more time to achieve indexing for new journals. However, with respect to the latter consideration, it is a reasonable assumption that the most significant and impactful scientific works would be more often published in established journals already indexed in WoS (on average, the WoS data set analyzed by us consist of studies with higher significance and impact, and this is not necessarily a limitation; rather, it can also be seen as a kind of filtering that excludes papers from less established journals). Along this line of thought, we should however emphasize that “less established journals” would not necessarily imply inferior journal quality; while other databases sometimes index journals that simply do not meet the stringent quality criteria of WoS, at the same time, there are examples of newly emerging journals of high quality, which are well on their way to being indexed in WoS. One highly relevant example for this area of research would be JMIR Medical Education, edited by Nabil Zary, who was one of the most productive researchers identified in our study (Table 1). Furthermore, it should be noticed that the CPP data were based on all publication types, not just original articles. Therefore, a high CPP reflected in our study does not necessarily mean that only original research articles concerning certain terms were highly cited; this parameter is also influenced by the citation rates of relevant reviews, proceedings papers, editorial material, and meeting abstracts. Readers should be aware of these limitations when interpreting the results. Moreover, considering that Scopus and Google Scholar are becoming increasingly used for the assessment of academic performance in different environments (often in low- and middle-income countries), future studies assessing the publication practices based on these databases are expected to gain further insights.

The analyzed literature in the field of digital teaching research in medicine contained 3978 publications. The literature received worldwide contributions with the most productive countries being the United States and the United Kingdom. Reviews were significantly more cited, but the citations between OA vs non-OA papers did not significantly differ. Some themes were more highly cited, such as virtual reality, innovation, trial, effectiveness, and anatomy. Different aspects in medical education were experimented for digital teaching, such as gross anatomy education, histology, complementary medicine, medicinal chemistry, and basic life support. Some studies have shown that digital teaching could increase learning satisfaction, knowledge gain, and even cost-effectiveness. Digital teaching in medical education is expected to flourish in the future, especially in light of the COVID-19 pandemic occurrence, which brought the advantages of the digital education approach to the spotlight. This would be particularly useful for clinical teaching during pandemics, gaining insights into highly infectious diseases or rare diseases that do not have available cases in a local setting.