The scientometric analysis framework of this paper is shown in Figure 1, which is mainly divided into 4 parts: data collection, data preprocessing, topic modeling, and results analysis.

HIE-related scientific articles from 2014 to 2023 were retrieved from Web of Science Core Collection (WoSCC), using the search strategy “Topic=health informatics education, Year Published=2014-2023, Language=English, Document Type=Article.” The search was conducted using the following query: TS = (health informatics education) AND PY = (2014-2023) AND LA = (English) AND DT = (Article).

Only WoSCC was used to retrieve articles owing to its comprehensive collection of multidisciplinary academic journals, its higher quality than other databases, and its status as an authoritative source for citation information, making it a primary scientific database and the most preferred destination for researchers from all domains [16,17]. The search was restricted to this term and did not include other terms related to health informatics, such as medical informatics, nursing informatics, or consumer health informatics, as this search strategy has been shown to be effective for scientometric analysis of health informatics [18,19]. In addition, only research articles were included as they tend to make more original contributions when compared to other types of publications [20]. As a result, a total of 577 articles were retrieved.

It is commonly agreed that titles, abstracts, and article keywords are suitable for conceptual reviews because they usually represent the noteworthy content of articles [21,22]. In particular, abstracts are able to present summaries of articles in terms of research aims and problems, as well as major findings [23,24]. Therefore, in this study, the primary materials of topic modeling were the title, keywords, and abstract of each article. Of the articles collected, 2 without abstracts were excluded, and 575 articles with abstract information were selected for topic modeling.

To improve data quality before conducting topic modeling, preprocessing was performed, which involved the following 3 aspects.

First, the names of countries/regions, institutions, and researchers were extracted from the authors’ addresses, which were further preprocessed to ensure unified expressions. Terms were extracted from the titles, keywords, and abstracts, and were consolidated when they had multiple spellings.

Second, numbers, punctuations, symbols, and stop words (eg, “me,” “I,” “or,” “him,” “a,” and “they”) were deleted to enhance consistency and reduce computational load [25]. Punctuations were removed, and all the remaining terms were converted to lowercase.

Third, following the previous work [20], the weights 0.4, 0.4, and 0.2 were assigned to terms extracted from titles, keywords, and abstracts, respectively. Since in topic modeling, the estimations of document-topic and topic-term distributions are based on the document-term distribution, which is frequency-based, the assignment of weights was performed by multiplying the frequency by weight. For example, for each document, if f_1wi, f_2wi, and f_3wi denote the frequencies of the word w_i in the keywords, title, and abstract, respectively, then the weighted frequency of the word w_i in the document is denoted by 0.4×f_1wi+0.4×f_2wi+0.2×f_3wi.

The major research topics within the 575 research articles in relation to HIE were uncovered by using the structural topic model (STM) [26]. The STM belongs to the class of topic models seeking to discover latent thematic clusters within a collection of texts, and is an extension of the latent Dirichlet allocation (LDA) model [27] and the correlated topic model (CTM) [28]. Traditional topic modeling techniques, such as LDA, assume that topics are discrete and independent. However, in many cases, topics may not be independent, and documents may have complex relationships between topics. The STM addresses these limitations by incorporating structural information into the topic modeling process. Besides identifying topics, the STM estimates the prevalence of the found topics by incorporating document-level covariates (eg, time of publication or number of authors).

A diagram of the STM, adapted from a recent study using the STM to analyze smart health care [29], is presented in Figure 2. The process of topic modeling is to estimate the θ and β based on the observed term W, which represent document topic and topic word distributions, respectively, where n∈{1,2,…,N} denotes terms covering a document, k∈{1,2,…,K} is the topic, d∈{1,2,…,D} represents the document number, and z represents the underlying topic allocation of each term.

During topic modeling, the decision of a value indicating the number of topics is a priority. Previous work has demonstrated that an exclusive reliance on statistical measures could result in a less meaningful model parameter choice [30]. Thus, following the suggestion of previous studies [31], 10 candidate models were adopted with the numbers of topics ranging from 4 to 15. Two domain experts, who have taught health informatics for more than 10 years, independently compared the outputs of the 10 models by evaluating the representative terms and articles to select the best-fitted model based on the following criteria adopted in previous work [32]: (1) Representative terms in each topic could form a meaningful topic together; (2) The top representative articles for each topic were in close relation to the topic; (3) There was limited overlap between topics within a topic model; and (4) All obvious research topics in HIE were included. In this study, both the experts chose the 7-topic model. The output of the model involved topic content and topic prevalence. The 7-topic model showed good semantic consistency within topics and exclusivity between topics.

Topics were labeled manually according to the inspection of the highest ranking 12 terms and 20 articles that consisted of each topic. The labels were individually examined by the 2 domain experts, who then reached an agreement through discussion. Specifically, they first examined the most representative terms for each topic and interpreted the discriminating terms within topics based on their semantic meanings. Then, they examined a sample of representative articles for each of the topics. Finally, they compared the labeling results and discussed the inconsistent labels to unify the results.

In addition to the STM, other methods and tools have also been adopted in this study. On the one hand, several scientometric indicators, namely, article count, citation count, and average citations per article, were used to evaluate the performance of involved publication sources, subjects, institutions, and countries/regions. On the other hand, scientific collaborations among countries/regions and institutions were explored by social network analysis using VOSviewer. VOSviewer generated collaborative networks by representing countries/regions and institutions as nodes, and the collaborative relationships were reflected using lines. The node size indicated the number of articles in which 2 linked countries/regions or institutions collaborated, and colors reflected the countries/regions or institutions.

The annual trends of the article counts are depicted in Figure 3. Research in HIE has been growing during the studied period, particularly since 2020, reaching 84 articles in 2023.

Although there were fluctuations in the data, it is reasonable to conclude that HIE has received a growing interest from the academic community. Therefore, it is highly likely that the amount of HIE research will continue to increase in the near future.

In the analyses concerning countries/regions and institutions, all the countries/regions and institutions contributing to each article were included in the data, and the most prolific ones were evaluated using 3 scientometric indicators, namely, article count, citation count, and average citations per article.

A total of 70 countries/regions contributed to the 575 articles, and the top 10 countries/regions ranked by the article count are shown in Table 1. United States had 326 articles, followed by Canada (50 articles) and Australia (45 articles). These top 3 countries were also ranked among the top 3 in terms of citation count, with 4705, 673, and 514 citations, respectively.

Regarding average citations per article, the top 3 countries/regions included Germany (17.61), United States (14.43), and Canada (13.46). Although Germany had fewer articles (n=28) as compared to the top 3 countries, their articles had higher quality as indicated by the average citations per article. By considering all the abovementioned indicators, the performance of Germany is worth highlighting, indicating its great contribution to the research field of HIE.

The 575 articles concerning HIE were contributed by 200 institutions, among which the top 10 are shown in Table 2. Among the listed 10 institutions, 9 are from the United States, which again indicates the dominant position and active role of the United States in the field of HIE. Harvard University was the most prolific institution with 33 articles, followed by University of Texas System (28 articles) and University System of Ohio (24 articles).

Regarding citation count, the top 3 institutions were University of California System (521 citations), University of Minnesota System (394 citations), and Indiana University System (327 citations). These top 3 institutions were also ranked among the top 3 in terms of the average citations per article, with values of 26.05, 19.7, and 16.35, respectively.

By considering all these indicators, the performance of University of California System is worth highlighting, indicating its great contribution to the field of HIE. If we put Harvard University and Harvard Medical School together, we get an institution with the most influence regarding the indicators of article count and citation count.

The 575 articles concerning HIE were contributed by 197 publication sources, among which the top 20 with a minimum article count of 5 are shown in Table 3, which together contributed to 51.3% (295/575) of the total studied articles. Journal of Medical Internet Research was the most prolific in the field with 42 articles, followed by Journal of the American Medical Informatics Association (37 articles), Applied Clinical Informatics (30 articles), and BMJ Open (28 articles). Regarding citation count, the top 3 journals were Journal of Medical Internet Research (1213 citations), Journal of the American Medical Informatics Association (442 citations), and Academic Medicine (328 citations). Regarding average citations per article, the top 3 journals were Academic Medicine (41.00), Journal of Medical Internet Research (28.88), and JMIR mHealth and uHealth (25.56). By considering all these indicators, the performance of Journal of Medical Internet Research is worth highlighting, indicating its significant contribution to the research field of HIE.

The 575 articles concerning HIE were distributed in 80 Web of Science categories, among which the top 20 with a minimum article count of 7 are shown in Table 4. Medical Informatics was the most relevant category in this field with 229 articles, followed by Health Care Sciences Services (189 articles) and Nursing (103 articles). The latter 2 categories also ranked among the top 2 from the perspective of citation count. Regarding average citations per article, the top 3 categories were Dentistry Oral Surgery Medicine (40.00), Education Scientific Disciplines (18.42), and Health Care Sciences Services (15.87). Although the article count of Dentistry Oral Surgery Medicine was only 7, its citation count was relatively high. That is because an article titled “Artificial Intelligence in Dentistry: Chances and Challenges” received widespread attention. This article reviewed artificial intelligence (AI) in dentistry and suggested that dental education needs to accompany the introduction of clinical AI solutions by fostering digital literacy in the future dental workforce.

The macro picture of collaboration among countries in the research field of HIE from 2014 to 2023 is shown in Figure 4, indicating that related research in countries has benefited from international cooperation.

The collaborative scientific research networks among countries/regions and institutions are visualized in Figures 5 and 6, respectively. In these collaborative networks, countries/regions and institutions are represented by nodes, and collaborations between countries/regions or institutions are represented by edges. The size of each node indicates the number of articles from each country/region or institution, with a larger node being associated with a larger number of articles. Additionally, the color of a node is determined by the cluster to which the node belongs, and the lines between nodes represent collaborative relationships. The distance between 2 nodes in the visualization approximately indicates the relatedness of the countries/regions or institutions in terms of collaborative links. In general, the closer 2 countries/regions or institutions are located to each other, the stronger their relatedness.

For country/region collaborations, the minimum number of articles of a country/region was set to 2, and 50 countries met the threshold out of the 72 countries. For each of the 50 countries, the total strength of the co-authorship links with other countries was calculated. The results of the top 10 countries/regions are shown in Table 5. Of these 10 countries/regions, United States, England, and Canada collaborated with the most countries/regions, with 138, 85, and 82 collaborations, respectively.

For institution collaborations, the minimum number of articles of an institution was set to 2, and 36 institutions met the threshold out of the 1203 institutions. For each of the 36 institutions, the total strength of the co-authorship links with other institutions was calculated. The results of the top 10 institutions are shown in Table 6. Of these 10 institutions, Vanderbilt University, Harvard Medical School, and University of Washington collaborated with the most institutions, with 31, 28, and 27 collaborations, respectively.

Table 7 shows the 7-topic STM results with representative terms under 4 different metrics, topic proportion within the whole corpus, and suggested topic labels. Among the 4 metrics, “Highest Prob” means terms with the highest probability, “FREX” weights words by their overall frequency and how exclusive they are to the topic, “Lift” weights words through division by their frequency in other topics, and “Score” divides the log frequency of the word in the topic by the log frequency of the word in other topics [26]. Topic proportion refers to the ratio of the number of articles on each topic to all articles. The results showed that Clinical Informatics Education (130/575, 22.6%) was the most researched topic among authors of HIE studies, followed by Mobile Application in Health Education & Healthcare (123/575, 21.4%) and Consumer Health Informatics Education (99/575, 17.2%). Additionally, a synopsis for each topic label has been provided to illustrate its meaning in the last column of Table 7.

Figure 7 shows the trends of the annual proportions of detected topics, which suggest that HIE research has become more diverse over time, with many different issues being of concern. The temporal dynamics of the distributions of all topics clearly indicate the unique foci for each year and demonstrate how the popularity of each topic has changed relative to other topics over time, depicting the process of development for health informatics research. For example, researchers showed continuous interest in Clinical Informatics Education (Topic 1) and Discipline of Biomedical and Health Informatics (Topic 6). In 2018, the predominant foci of the community were mainly related to Professional Teaching of Biomedical and Health Informatics (Topic 4). Years 2017 and 2022 witnessed a dramatic interest in issues concerning Mobile Application in Health Education & Healthcare (Topic 2). Some topics with high proportions at earlier time points showed reductions at the latest time point, such as Consumer Health Informatics Education (Topic 3) and Nursing Informatics Education (Topic 7). Since the year 2019, scholars in the field of HIE began to show interest in Public Health Informatics Education & Digital Health (Topic 5).

This study conducted a scientometric analysis on research of HIE from 2014 to 2023. Over the last decade, research on HIE has appeared to be increasing steadily. This field has been progressively developing and maturing, and has received a growing interest from the academic community. It is currently experiencing a phase of rapid growth.

The United States has made substantial contributions to the growth of publications in this research field. Additionally, Canada, Australia, England, Germany, China, Brazil, Sweden, and Spain demonstrated a high interest as well. Harvard University and Harvard Medical School represented the most influential institutions, and the most vigorous institutions included University of Texas System, University System of Ohio, Indiana University System, and University of California System. Research on HIE has been predominantly favored by journals related to health informatics, as indicated by article counts and citation counts. Journal of Medical Internet Research, Journal of The American Medical Informatics Association, and Applied Clinical Informatics were found to be the most influential and productive publication sources in terms of average citations per article and article counts. Collaboration among universities in the United States, England, and Canada was found to be high. Collaboration was identified to be mainly concentrated in the Americas, Europe, and Asia, with limited international collaboration from developing countries, suggesting that international collaboration in this research field needs to be strengthened.

Topic modeling analysis over time identified the knowledge structure and development trends in this research field. Seven topics were identified using R package STM, and representative terms were provided for each topic under 4 different metrics. Clinical Informatics Education was found to be the most researched topic among authors of HIE studies, followed by the topics of Mobile Application in Health Education & Healthcare and Consumer Health Informatics Education. Additionally, the temporal dynamics of the distributions of all topics have been presented, which demonstrated how the popularity of each topic has changed relative to other topics over time and thus depicted the process of development for health informatics research.

For a profound understanding of the identified research topics of HIE, the most representative research works for each topic were analyzed.

As IT is applied widely in health care and medical education, some studies have examined the evolution of the intellectual and conceptual structure in the field of HIE. For example, Monkman et al [62] conducted a scoping review to identify health informatics competencies from research articles and grey literature originating from professional associations, and proposed a health informatics core competencies framework that could be used to inform Canadian health informatics programs to ensure graduates would be prepared for careers in health informatics. To outline the history of medical informatics education in China, Hu et al [63] systematically analyzed the status of medical informatics education at different academic levels and suggested reasonable strategies for the further development of the field in China. Asiri [64] explored the challenges that face HIE and training in the Kingdom of Saudi Arabia by examining the state of health IT education, with a focus on HIE. Jeon et al [65] presented the status of nursing informatics education and the content covered in nursing informatics courses in Korea, and concluded that a greater focus was needed on training faculty and developing the courses. Similarly, Masic et al [66] investigated the status of medical informatics education in Bosnia and Herzegovina, and Europe.

Although these studies made invaluable contributions to the understanding of HIE, they tended to focus on specific regions or disciplines and did not provide a global overview of research in HIE. Additionally, they did not use any topic modeling methods to reveal topic structure and evolutionary trends of this research field, and thus, the methodologies employed in these studies had some limitations. Meanwhile, there was a scarcity of comprehensive scientometric analysis in this research field. Therefore, this study provided a topic modeling–based scientometric analysis of scientific articles concerning HIE, and it analyzed aspects such as productive countries and institutions, scientific collaborations, and topics and trends of HIE research.

The main limitation of this study was that only 1 database was searched to identify studies for inclusion, meaning that potentially relevant studies could have been overlooked. Although WoSCC is a widely used academic database globally, other medical-oriented databases may provide more significant academic and practical information. Subsequent studies should conduct searches in commonly used databases, such as PubMed, Scopus, IEEE, and Google Scholar, to ensure a more comprehensive analysis. Another constraint was that only journal articles were retrieved for analysis, excluding other relevant publication types in this field, such as reviews, book chapters, and conference proceedings. This could potentially affect the summarization and generalization of the results.

As AI has covered most of the research topics in health informatics, academic research on the intersection of AI and HIE has emerged in recent years. For example, Paranjape et al [67] summarized the state of medical education and recommended a framework to include AI education. Yang et al [68] examined the impact of an AI system tutoring course on clinical training and found an increased improvement in the expert-led AI tutoring group. However, AI did not appear as a representative term in any of these topics. The main reason may be that the time span chosen for this study was long at 10 years, and AI did not dominate during this period. Overall, the application of AI in HIE should be taken seriously, and formal training for clinical and health informatics programs should consist of AI technology and its applications.

The field of HIE is increasingly promising and active, with academic products flourishing. This study presents the results and findings from a topic-driven scientometric analysis of academic research articles concerning HIE published from 2014 to 2023. This study not only highlights the major research concerns and issues among scholars but also identifies the tendencies of article and citation counts; identifies major countries/regions, institutions, publication sources, and subjects; and outlines academic collaboration relationships. The topic-based scientometric analysis contributed to HIE by providing a comprehensive overview of its community. The contributions of this study with regard to HIE have been summarized.

First, it helps researchers, policymakers, and practitioners better understand the historical, extant, and upcoming structure of research in HIE. The exploration of major topics and their trends is helpful for the identification and comparison of current and potential scientific strength. Research governors or funding agencies can optimize research policies to promote potential competitive research areas and enhance scientific collaborations in specific research areas in order to bolster scientific and educational activities. Second, findings regarding prolific countries/regions and institutions can assist scholars in sharing research, conducting collaborations, and identifying important actors from whom to learn. This is valuable for the research community of HIE in terms of understanding its research status and trends, especially by translating the findings into directions for future research. Third, this study provides a clear outline of HIE, which could help interested educators as well as researchers who wish to continue working in this area. The results of prolific publication sources and subjects provide suggestions about sources to which scholars can submit their work and will help scholars to better position their research in the field.

As it explores the structures and research topics within the area of HIE, this study serves as a starting point for further in-depth research into HIE. The research scope of HIE is relatively broad, and this study is expected to provide editors and conference organizers with ways to identify gaps in the field, leading them to create special issues or conference sessions about topics that are generally not co-occurring, which could create potential new synergies in HIE.