Original Paper
Abstract
Background: Academic and educational institutions are making significant contributions toward training health informatics professionals. As research in health informatics education (HIE) continues to grow, it is useful to have a clearer understanding of this research field.
Objective: This study aims to comprehensively explore the research topics and trends of HIE from 2014 to 2023. Specifically, it aims to explore (1) the trends of annual articles, (2) the prolific countries/regions, institutions, and publication sources, (3) the scientific collaborations of countries/regions and institutions, and (4) the major research themes and their developmental tendencies.
Methods: Using publications in Web of Science Core Collection, a scientometric analysis of 575 articles related to the field of HIE was conducted. The structural topic model was used to identify topics discussed in the literature and to reveal the topic structure and evolutionary trends of HIE research.
Results: Research interest in HIE has clearly increased from 2014 to 2023, and is continually expanding. The United States was found to be the most prolific country in this field. Harvard University was found to be the leading institution with the highest publication productivity. Journal of Medical Internet Research, Journal of The American Medical Informatics Association, and Applied Clinical Informatics were the top 3 journals with the highest articles in this field. Countries/regions and institutions having higher levels of international collaboration were more impactful. Research on HIE could be modeled into 7 topics related to the following areas: clinical (130/575, 22.6%), mobile application (123/575, 21.4%), consumer (99/575, 17.2%), teaching (61/575, 10.6%), public health (56/575, 9.7%), discipline (55/575, 9.6%), and nursing (51/575, 8.9%). The results clearly indicate the unique foci for each year, depicting the process of development for health informatics research.
Conclusions: This is believed to be the first scientometric analysis exploring the research topics and trends in HIE. This study provides useful insights and implications, and the findings could be used as a guide for HIE contributors.
doi:10.2196/58165
Keywords
Introduction
Health informatics is at the crossroad of information science, computer science, data science, medicine, and health care, with great improvements in a wide range of health-related areas, including nursing, health care, public health, clinical decision, and biomedicine [
]. There is a growing need for professionals to be able to address health informatics issues in health-related fields through the design, development, implementation, evaluation, and use of innovative information technologies and applications.Consequently, many academic and educational institutions are making significant contributions toward training health informatics professionals by offering educational programs or courses in the field [
]. The nature of health informatics attracts both health care and IT professionals as specialists in a highly interdisciplinary field. Health care professionals, on the other hand, need to grasp the opportunities and limitations that technology can bring to health care [ ]. There are several ways to enter health informatics education (HIE) in different countries around the world, with the most common pathway being postgraduation following the completion of a bachelor’s degree in health informatics or a related degree in health care, life sciences, cognitive science, computer science and IT, health information management, and others, with associated work experience [ - ].Due to its multidisciplinary foundations and diverse application areas, HIE has captured the interest of researchers from numerous areas. Engagement from these researchers has been beneficial to the field as it has led to the development of a rich research literature in many journals. This has resulted in the vast and flourishing development of this field, with the research community demonstrating significant growth and a constant increase in the scientific literature. As research in HIE continues to grow, it is essential to have a clearer understanding of its status, trends, and topics across the research field for researchers and participants.
This study explores the topic structure and evolutionary trends of HIE research by conducting a topic-based scientometric analysis of scientific articles concerning HIE from 2014 to 2023. Specifically, scientometric analysis is an effective way to evaluate research characteristics and frontiers in a specific research field, with the application of mathematical and statistical methodologies [
]. In particular, it can be employed to obtain a better understanding of what has been investigated in the past and further make predictions about what will happen in the future. Such analysis can depict the trends and distributions of the literature involving scientific impacts and productivity [ ]. Meanwhile, topic modeling is a dimension reduction technology to extract hidden themes within a textual dataset, which detects term co-occurrence and assigns terms to the hidden themes [ ]. It has been widely applied in scientific research trend analysis and in the identification of emerging topics within a particular research area [ , ]. Although scientometric and topic models have been popularly implemented in a variety of scientific fields, they have not been applied in the research field of HIE. Thus, this study contributes to deepening the understanding of HIE and provides valuable references and recommendations for researchers and potential contributors, focusing on the following questions:- What research topics were the HIE community interested in?
- How did such research topics evolve over time?
- Which countries/regions and institutions were the major contributors to HIE?
- What were the scientific collaborations between countries/regions and institutions?
- What were the main research concerns of prolific contributors?
Methods
Analysis Framework
The scientometric analysis framework of this paper is shown in
, which is mainly divided into 4 parts: data collection, data preprocessing, topic modeling, and results analysis.Data Collection
HIE-related scientific articles from 2014 to 2023 were retrieved from Web of Science Core Collection (WoSCC), using the search strategy “Topic=health informatics education, Year Published=2014-2023, Language=English, Document Type=Article.” The search was conducted using the following query: TS = (health informatics education) AND PY = (2014-2023) AND LA = (English) AND DT = (Article).
Only WoSCC was used to retrieve articles owing to its comprehensive collection of multidisciplinary academic journals, its higher quality than other databases, and its status as an authoritative source for citation information, making it a primary scientific database and the most preferred destination for researchers from all domains [
, ]. The search was restricted to this term and did not include other terms related to health informatics, such as medical informatics, nursing informatics, or consumer health informatics, as this search strategy has been shown to be effective for scientometric analysis of health informatics [ , ]. In addition, only research articles were included as they tend to make more original contributions when compared to other types of publications [ ]. As a result, a total of 577 articles were retrieved.It is commonly agreed that titles, abstracts, and article keywords are suitable for conceptual reviews because they usually represent the noteworthy content of articles [
, ]. In particular, abstracts are able to present summaries of articles in terms of research aims and problems, as well as major findings [ , ]. Therefore, in this study, the primary materials of topic modeling were the title, keywords, and abstract of each article. Of the articles collected, 2 without abstracts were excluded, and 575 articles with abstract information were selected for topic modeling.Data Preprocessing
To improve data quality before conducting topic modeling, preprocessing was performed, which involved the following 3 aspects.
First, the names of countries/regions, institutions, and researchers were extracted from the authors’ addresses, which were further preprocessed to ensure unified expressions. Terms were extracted from the titles, keywords, and abstracts, and were consolidated when they had multiple spellings.
Second, numbers, punctuations, symbols, and stop words (eg, “me,” “I,” “or,” “him,” “a,” and “they”) were deleted to enhance consistency and reduce computational load [
]. Punctuations were removed, and all the remaining terms were converted to lowercase.Third, following the previous work [
], the weights 0.4, 0.4, and 0.2 were assigned to terms extracted from titles, keywords, and abstracts, respectively. Since in topic modeling, the estimations of document-topic and topic-term distributions are based on the document-term distribution, which is frequency-based, the assignment of weights was performed by multiplying the frequency by weight. For example, for each document, if f1wi, f2wi, and f3wi denote the frequencies of the word wi in the keywords, title, and abstract, respectively, then the weighted frequency of the word wi in the document is denoted by 0.4×f1wi+0.4×f2wi+0.2×f3wi.Data Analysis
The major research topics within the 575 research articles in relation to HIE were uncovered by using the structural topic model (STM) [
]. The STM belongs to the class of topic models seeking to discover latent thematic clusters within a collection of texts, and is an extension of the latent Dirichlet allocation (LDA) model [ ] and the correlated topic model (CTM) [ ]. Traditional topic modeling techniques, such as LDA, assume that topics are discrete and independent. However, in many cases, topics may not be independent, and documents may have complex relationships between topics. The STM addresses these limitations by incorporating structural information into the topic modeling process. Besides identifying topics, the STM estimates the prevalence of the found topics by incorporating document-level covariates (eg, time of publication or number of authors).A diagram of the STM, adapted from a recent study using the STM to analyze smart health care [
], is presented in . The process of topic modeling is to estimate the θ and β based on the observed term W, which represent document topic and topic word distributions, respectively, where n∈{1,2,…,N} denotes terms covering a document, k∈{1,2,…,K} is the topic, d∈{1,2,…,D} represents the document number, and z represents the underlying topic allocation of each term.During topic modeling, the decision of a value indicating the number of topics is a priority. Previous work has demonstrated that an exclusive reliance on statistical measures could result in a less meaningful model parameter choice [
]. Thus, following the suggestion of previous studies [ ], 10 candidate models were adopted with the numbers of topics ranging from 4 to 15. Two domain experts, who have taught health informatics for more than 10 years, independently compared the outputs of the 10 models by evaluating the representative terms and articles to select the best-fitted model based on the following criteria adopted in previous work [ ]: (1) Representative terms in each topic could form a meaningful topic together; (2) The top representative articles for each topic were in close relation to the topic; (3) There was limited overlap between topics within a topic model; and (4) All obvious research topics in HIE were included. In this study, both the experts chose the 7-topic model. The output of the model involved topic content and topic prevalence. The 7-topic model showed good semantic consistency within topics and exclusivity between topics.Topics were labeled manually according to the inspection of the highest ranking 12 terms and 20 articles that consisted of each topic. The labels were individually examined by the 2 domain experts, who then reached an agreement through discussion. Specifically, they first examined the most representative terms for each topic and interpreted the discriminating terms within topics based on their semantic meanings. Then, they examined a sample of representative articles for each of the topics. Finally, they compared the labeling results and discussed the inconsistent labels to unify the results.
In addition to the STM, other methods and tools have also been adopted in this study. On the one hand, several scientometric indicators, namely, article count, citation count, and average citations per article, were used to evaluate the performance of involved publication sources, subjects, institutions, and countries/regions. On the other hand, scientific collaborations among countries/regions and institutions were explored by social network analysis using VOSviewer. VOSviewer generated collaborative networks by representing countries/regions and institutions as nodes, and the collaborative relationships were reflected using lines. The node size indicated the number of articles in which 2 linked countries/regions or institutions collaborated, and colors reflected the countries/regions or institutions.
Results
Annual Trends of Publications
The annual trends of the article counts are depicted in
. Research in HIE has been growing during the studied period, particularly since 2020, reaching 84 articles in 2023.Although there were fluctuations in the data, it is reasonable to conclude that HIE has received a growing interest from the academic community. Therefore, it is highly likely that the amount of HIE research will continue to increase in the near future.
Prolific Countries/Regions and Institutions
In the analyses concerning countries/regions and institutions, all the countries/regions and institutions contributing to each article were included in the data, and the most prolific ones were evaluated using 3 scientometric indicators, namely, article count, citation count, and average citations per article.
A total of 70 countries/regions contributed to the 575 articles, and the top 10 countries/regions ranked by the article count are shown in
. United States had 326 articles, followed by Canada (50 articles) and Australia (45 articles). These top 3 countries were also ranked among the top 3 in terms of citation count, with 4705, 673, and 514 citations, respectively.Regarding average citations per article, the top 3 countries/regions included Germany (17.61), United States (14.43), and Canada (13.46). Although Germany had fewer articles (n=28) as compared to the top 3 countries, their articles had higher quality as indicated by the average citations per article. By considering all the abovementioned indicators, the performance of Germany is worth highlighting, indicating its great contribution to the research field of HIE.
The 575 articles concerning HIE were contributed by 200 institutions, among which the top 10 are shown in
. Among the listed 10 institutions, 9 are from the United States, which again indicates the dominant position and active role of the United States in the field of HIE. Harvard University was the most prolific institution with 33 articles, followed by University of Texas System (28 articles) and University System of Ohio (24 articles).Regarding citation count, the top 3 institutions were University of California System (521 citations), University of Minnesota System (394 citations), and Indiana University System (327 citations). These top 3 institutions were also ranked among the top 3 in terms of the average citations per article, with values of 26.05, 19.7, and 16.35, respectively.
By considering all these indicators, the performance of University of California System is worth highlighting, indicating its great contribution to the field of HIE. If we put Harvard University and Harvard Medical School together, we get an institution with the most influence regarding the indicators of article count and citation count.
Country/region | Article count | Citation count (ranking) | Average citations per article (ranking) | Web of Science category (article count) |
United States | 326 | 4705 (1) | 14.43 (2) |
|
Canada | 50 | 673 (2) | 13.46 (3) |
|
Australia | 45 | 514 (3) | 11.42 (6) |
|
England | 33 | 369 (5) | 11.18 (7) |
|
Germany | 28 | 493 (4) | 17.61 (1) |
|
Netherlands | 25 | 327 (6) | 13.08 (4) |
|
China | 23 | 289 (7) | 12.57 (5) |
|
Brazil | 20 | 168 (8) | 8.40 (10) |
|
Sweden | 17 | 153 (9) | 9.00 (9) |
|
Spain | 13 | 145 (10) | 11.15 (8) |
|
Institution | Country | Article count | Citation count (ranking) | Average citations per article (ranking) | Web of Science category (article count) |
Harvard University | United States | 33 | 321 (4) | 9.73 (9) |
|
University of Texas System | United States | 28 | 274 (7) | 9.79 (8) |
|
University System of Ohio | United States | 24 | 317 (5) | 13.21 (5) |
|
Harvard Medical School | United States | 20 | 252 (8) | 12.60 (6) |
|
Indiana University System | United States | 20 | 327 (3) | 16.35 (3) |
|
University of California System | United States | 20 | 521 (1) | 26.05 (1) |
|
University of Minnesota System | United States | 20 | 394 (2) | 19.70 (2) |
|
University of Pennsylvania | United States | 19 | 226 (9) | 11.89 (7) |
|
State University System of Florida | United States | 18 | 282 (6) | 15.67 (4) |
|
University of Toronto | Canada | 18 | 146 (10) | 8.11 (10) |
|
Top Publication Sources and Subjects
The 575 articles concerning HIE were contributed by 197 publication sources, among which the top 20 with a minimum article count of 5 are shown in
, which together contributed to 51.3% (295/575) of the total studied articles. Journal of Medical Internet Research was the most prolific in the field with 42 articles, followed by Journal of the American Medical Informatics Association (37 articles), Applied Clinical Informatics (30 articles), and BMJ Open (28 articles). Regarding citation count, the top 3 journals were Journal of Medical Internet Research (1213 citations), Journal of the American Medical Informatics Association (442 citations), and Academic Medicine (328 citations). Regarding average citations per article, the top 3 journals were Academic Medicine (41.00), Journal of Medical Internet Research (28.88), and JMIR mHealth and uHealth (25.56). By considering all these indicators, the performance of Journal of Medical Internet Research is worth highlighting, indicating its significant contribution to the research field of HIE.The 575 articles concerning HIE were distributed in 80 Web of Science categories, among which the top 20 with a minimum article count of 7 are shown in
. Medical Informatics was the most relevant category in this field with 229 articles, followed by Health Care Sciences Services (189 articles) and Nursing (103 articles). The latter 2 categories also ranked among the top 2 from the perspective of citation count. Regarding average citations per article, the top 3 categories were Dentistry Oral Surgery Medicine (40.00), Education Scientific Disciplines (18.42), and Health Care Sciences Services (15.87). Although the article count of Dentistry Oral Surgery Medicine was only 7, its citation count was relatively high. That is because an article titled “Artificial Intelligence in Dentistry: Chances and Challenges” received widespread attention. This article reviewed artificial intelligence (AI) in dentistry and suggested that dental education needs to accompany the introduction of clinical AI solutions by fostering digital literacy in the future dental workforce.Publication source | Article count | Citation count (ranking) | Average citations per article (ranking) | Web of Science category (article count) |
Journal of Medical Internet Research | 42 | 1213 (1) | 28.88 (2) |
|
Journal of The American Medical Informatics Association | 37 | 442 (2) | 11.95 (7) |
|
Applied Clinical Informatics | 30 | 245 (5) | 8.17 (12) |
|
BMJ Open | 28 | 177 (8) | 6.32 (14) |
|
CIN: Computers Informatics Nursing | 27 | 280 (4) | 10.37 (10) |
|
International Journal of Medical Informatics | 17 | 67 (14) | 3.94 (17) |
|
BMC Medical Informatics and Decision Making | 15 | 142 (9) | 9.47 (11) |
|
Nurse Education Today | 10 | 216 (7) | 21.60 (4) |
|
JMIR Medical Informatics | 9 | 26 (17) | 2.89 (19) |
|
JMIR mHealth and uHealth | 9 | 230 (6) | 25.56 (3) |
|
Methods of Information in Medicine | 9 | 98 (12) | 10.89 (8) |
|
Academic Medicine | 8 | 328 (3) | 41.00 (1) |
|
International Journal of Environmental Research and Public Health | 8 | 84 (13) | 10.50 (9) |
|
Journal of Interprofessional Care | 8 | 62 (15) | 7.75 (13) |
|
BMC Medical Education | 7 | 23 (19) | 3.29 (18) |
|
Health Informatics Journal | 7 | 40 (16) | 5.71 (15) |
|
Informatics for Health Social Care | 7 | 104 (11) | 14.86 (6) |
|
Frontiers in Public Health | 6 | 108 (10) | 18.00 (5) |
|
Journal of Nursing Education | 6 | 24 (18) | 4.00 (16) |
|
Healthcare | 5 | 13 (20) | 2.60 (20) |
|
Web of Science category | Article count | Citation count (ranking) | Average citations per article (ranking) |
Medical Informatics | 229 | 371 (9) | 1.62 (20) |
Health Care Sciences Services | 189 | 3000 (1) | 15.87 (3) |
Nursing | 103 | 1326 (2) | 12.87 (7) |
Computer Science Interdisciplinary Applications | 71 | 873 (3) | 12.30 (8) |
Computer Science Information Systems | 70 | 697 (5) | 9.96 (12) |
Medicine General Internal | 60 | 517 (8) | 8.62 (14) |
Public Environmental Occupational Health | 46 | 593 (6) | 12.89 (5) |
Information Science Library Science | 41 | 528 (7) | 12.88 (6) |
Education Scientific Disciplines | 38 | 700 (4) | 18.42 (2) |
Health Policy Services | 30 | 348 (10) | 11.60 (15) |
Pharmacology Pharmacy | 14 | 221 (12) | 15.79 (4) |
Environmental Sciences | 12 | 89 (15) | 7.42 (16) |
Medicine Research Experimental | 10 | 95 (14) | 9.50 (13) |
Pediatrics | 10 | 73 (18) | 7.30 (17) |
Radiology Nuclear Medicine Medical Imaging | 10 | 121 (13) | 12.10 (9) |
Education Educational Research | 8 | 26 (20) | 3.25 (19) |
Multidisciplinary Sciences | 8 | 41 (19) | 5.13 (18) |
Oncology | 8 | 88 (16) | 11.00 (10) |
Primary Health Care | 8 | 85 (17) | 10.63 (11) |
Dentistry Oral Surgery Medicine | 7 | 280 (11) | 40.00 (1) |
Scientific Collaboration
The macro picture of collaboration among countries in the research field of HIE from 2014 to 2023 is shown in
, indicating that related research in countries has benefited from international cooperation.The collaborative scientific research networks among countries/regions and institutions are visualized in
and , respectively. In these collaborative networks, countries/regions and institutions are represented by nodes, and collaborations between countries/regions or institutions are represented by edges. The size of each node indicates the number of articles from each country/region or institution, with a larger node being associated with a larger number of articles. Additionally, the color of a node is determined by the cluster to which the node belongs, and the lines between nodes represent collaborative relationships. The distance between 2 nodes in the visualization approximately indicates the relatedness of the countries/regions or institutions in terms of collaborative links. In general, the closer 2 countries/regions or institutions are located to each other, the stronger their relatedness.For country/region collaborations, the minimum number of articles of a country/region was set to 2, and 50 countries met the threshold out of the 72 countries. For each of the 50 countries, the total strength of the co-authorship links with other countries was calculated. The results of the top 10 countries/regions are shown in
. Of these 10 countries/regions, United States, England, and Canada collaborated with the most countries/regions, with 138, 85, and 82 collaborations, respectively.For institution collaborations, the minimum number of articles of an institution was set to 2, and 36 institutions met the threshold out of the 1203 institutions. For each of the 36 institutions, the total strength of the co-authorship links with other institutions was calculated. The results of the top 10 institutions are shown in
. Of these 10 institutions, Vanderbilt University, Harvard Medical School, and University of Washington collaborated with the most institutions, with 31, 28, and 27 collaborations, respectively.Country/region | Total link strength | Article count |
United States | 138 | 326 |
England | 85 | 33 |
Canada | 82 | 50 |
Netherlands | 76 | 25 |
Australia | 75 | 45 |
Germany | 67 | 28 |
Sweden | 59 | 17 |
Greece | 54 | 10 |
Norway | 52 | 9 |
Spain | 52 | 13 |
Institution | Total link strength | Article count |
Vanderbilt University | 31 | 18 |
Harvard Medical School | 28 | 20 |
University of Washington | 27 | 16 |
Oregon Health Science University | 25 | 11 |
Columbia University | 23 | 16 |
Johns Hopkins University | 22 | 13 |
University of Texas Health Science Center Houston | 22 | 13 |
University of Minnesota System | 20 | 20 |
Indiana University System | 18 | 20 |
University of Utah | 18 | 14 |
Major Research Topics and Trends
shows the 7-topic STM results with representative terms under 4 different metrics, topic proportion within the whole corpus, and suggested topic labels. Among the 4 metrics, “Highest Prob” means terms with the highest probability, “FREX” weights words by their overall frequency and how exclusive they are to the topic, “Lift” weights words through division by their frequency in other topics, and “Score” divides the log frequency of the word in the topic by the log frequency of the word in other topics [ ]. Topic proportion refers to the ratio of the number of articles on each topic to all articles. The results showed that Clinical Informatics Education (130/575, 22.6%) was the most researched topic among authors of HIE studies, followed by Mobile Application in Health Education & Healthcare (123/575, 21.4%) and Consumer Health Informatics Education (99/575, 17.2%). Additionally, a synopsis for each topic label has been provided to illustrate its meaning in the last column of .
shows the trends of the annual proportions of detected topics, which suggest that HIE research has become more diverse over time, with many different issues being of concern. The temporal dynamics of the distributions of all topics clearly indicate the unique foci for each year and demonstrate how the popularity of each topic has changed relative to other topics over time, depicting the process of development for health informatics research. For example, researchers showed continuous interest in Clinical Informatics Education (Topic 1) and Discipline of Biomedical and Health Informatics (Topic 6). In 2018, the predominant foci of the community were mainly related to Professional Teaching of Biomedical and Health Informatics (Topic 4). Years 2017 and 2022 witnessed a dramatic interest in issues concerning Mobile Application in Health Education & Healthcare (Topic 2). Some topics with high proportions at earlier time points showed reductions at the latest time point, such as Consumer Health Informatics Education (Topic 3) and Nursing Informatics Education (Topic 7). Since the year 2019, scholars in the field of HIE began to show interest in Public Health Informatics Education & Digital Health (Topic 5).
Topics | Topic labels | Representative termsa | Topic (N=575), n (%) | Synopsis of topic labels |
Topic 1 | Clinical Informatics Education |
| 130 (22.6) | Integrating clinical informatics into the full spectrum of medical education |
Topic 2 | Mobile Application in Health Education & Healthcare |
| 123 (21.4) | Mobile apps are effective learning tools for medical education |
Topic 3 | Consumer Health Informatics Education |
| 99 (17.2) | Health informatics from multiple consumer or patient perspectives, including health literacy and consumer education |
Topic 4 | Professional Teaching of Biomedical and Health Informatics |
| 61 (10.6) | Competencies of health informatics for health care professionals and others working in this field |
Topic 5 | Public Health Informatics Education & Digital Health |
| 56 (9.7) | Courses on digital health to train public health professionals |
Topic 6 | Discipline of Biomedical and Health Informatics |
| 55 (9.6) | Practices in teaching and learning in the discipline of biomedical and health informatics |
Topic 7 | Nursing Informatics Education |
| 51 (8.9) | Digital competencies for nurses and nursing care in this digital age |
a“Highest Prob” means terms with the highest probability, “FREX” weights words by their overall frequency and how exclusive they are to the topic, “Lift” weights words through division by their frequency in other topics, and “Score” divides the log frequency of the word in the topic by the log frequency of the word in other topics.
Discussion
Principal Findings
This study conducted a scientometric analysis on research of HIE from 2014 to 2023. Over the last decade, research on HIE has appeared to be increasing steadily. This field has been progressively developing and maturing, and has received a growing interest from the academic community. It is currently experiencing a phase of rapid growth.
The United States has made substantial contributions to the growth of publications in this research field. Additionally, Canada, Australia, England, Germany, China, Brazil, Sweden, and Spain demonstrated a high interest as well. Harvard University and Harvard Medical School represented the most influential institutions, and the most vigorous institutions included University of Texas System, University System of Ohio, Indiana University System, and University of California System. Research on HIE has been predominantly favored by journals related to health informatics, as indicated by article counts and citation counts. Journal of Medical Internet Research, Journal of The American Medical Informatics Association, and Applied Clinical Informatics were found to be the most influential and productive publication sources in terms of average citations per article and article counts. Collaboration among universities in the United States, England, and Canada was found to be high. Collaboration was identified to be mainly concentrated in the Americas, Europe, and Asia, with limited international collaboration from developing countries, suggesting that international collaboration in this research field needs to be strengthened.
Topic modeling analysis over time identified the knowledge structure and development trends in this research field. Seven topics were identified using R package STM, and representative terms were provided for each topic under 4 different metrics. Clinical Informatics Education was found to be the most researched topic among authors of HIE studies, followed by the topics of Mobile Application in Health Education & Healthcare and Consumer Health Informatics Education. Additionally, the temporal dynamics of the distributions of all topics have been presented, which demonstrated how the popularity of each topic has changed relative to other topics over time and thus depicted the process of development for health informatics research.
Implications of the 7 Topics
For a profound understanding of the identified research topics of HIE, the most representative research works for each topic were analyzed.
Topic 1: Clinical Informatics Education
Education in clinical informatics is important for not only medical students but also other clinical staff at all professional levels of education. Integrating clinical informatics into the full spectrum of medical education is a vital step required for the understanding and practice of modern medicine. Education in clinical informatics is present throughout the spectrum of formal medical education, extending from high school to postgraduate training. Mai et al [
] presented a model of a clinical informatics curriculum that has been in place at the Children’s Hospital of Philadelphia. This educational model can support resident involvement in hospital-wide informatics efforts. Similar models to promote resident engagement in clinical informatics across training programs have the potential to accelerate health care improvements and develop the clinical informaticians of the future.Topic 2: Mobile Application in Health Education & Healthcare
Current evidence suggests an important knowledge gain among health science students and professionals with mobile devices (m-learning). Matos Lamenha-Lins et al [
] conducted a study to verify whether a mobile app was a good learning tool for improving dental students’ knowledge. Mobile apps are effective learning tools that have a significant role in transferring information and knowledge to clinicians and nurses. The use of mobile technologies to improve health outcomes or mobile health is rapidly evolving. Garner et al [ ] demonstrated that a mobile health app was effective in improving diabetes health literacy. Nezamdoust et al [ ] performed a study to identify the factors affecting nurses’ use of practical health-related mobile apps in education and patient interaction based on the combined technology acceptance model (TAM) and diffusion of innovation (DOI).Topic 3: Consumer Health Informatics Education
The American Medical Informatics Association defines consumer health informatics (CHI) as a field devoted to informatics from multiple consumers or patient views and which includes patient-focused informatics, health literacy, and consumer education [
]. Scholars mainly conducted research from 3 aspects, including consumer health needs [ ], the effect of health education materials [ , ], and consumer health behavior and literacy [ , ].Topic 4: Professional Teaching of Biomedical and Health Informatics
Health informatics has acquired the attributes of a profession, such as a well-defined set of competencies, a certification of fitness to practice, shared professional identity, life-long commitment, and a code of ethics. Research on this topic involves the following: health informatics curricula at different education levels [
, ], essential skills and competencies for health care professionals and others working in the field of health informatics to support the development and implementation of health informatics in health care [ , , ], and the quality of teaching and training with delivered HIE programs [ - ].Topic 5: Public Health Informatics Education & Digital Health Education
A generally accepted definition of public health informatics is as follows: “the systematic application of information, computer science, and technology to public health practice, research, and learning” [
]. Public health has a strong historical basis in handling health information, and much of public health activity continues to involve collecting, analyzing, and disseminating information about the health of the public. Moreover, public health is inherently interdisciplinary, and the sciences underlying public health encompass a set of skills and knowledge that are recognized to be at the core of health informatics [ ]. Meanwhile, digital health has been recognized as an innovative health solution that enables public health action to stop the rapid spread of a virus during a pandemic [ ]. Digital health is a key driver in addressing public health challenges and achieving sustainable health systems and universal health coverage [ ]. Therefore, there is an urgent need to analyze how to establish systematic courses on digital health in medical schools to train the next generation of doctors and public health managers in the integration of health theory and digital technology [ ].Topic 6: Discipline of Biomedical and Health Informatics
Health informatics became a professional discipline nearly 2 decades ago [
]. Being an interdisciplinary discipline according to its definition, health informatics draws upon different disciplines. Professionals entering the field have diverse backgrounds, and HIE at various levels needs to support a range of work roles. To help inform programs on practices in teaching and learning, a survey of master’s programs in health and biomedical informatics in the United States was conducted to determine the national landscape of culminating experiences, including capstone projects, research theses, internships, and practicums [ ]. Bellazzi et al [ ] provided a description of the fundamental design principles of 3 medical informatics education programs and explored some aspects of the teaching modules, highlighting their positive aspects.Topic 7: Nursing Informatics Education
Nursing informatics is a subcomponent of health informatics pertaining to nurses and nursing care [
]. Nurses need digital competencies to integrate new technologies in their professional activities. The foundations for these digital competencies are acquired during their vocational training/undergraduate education and ongoing training, and are deepened in their workplaces [ ]. Most research involving this topic was focused on improving competencies for nursing in the digital age [ , ].Comparison With the Literature
As IT is applied widely in health care and medical education, some studies have examined the evolution of the intellectual and conceptual structure in the field of HIE. For example, Monkman et al [
] conducted a scoping review to identify health informatics competencies from research articles and grey literature originating from professional associations, and proposed a health informatics core competencies framework that could be used to inform Canadian health informatics programs to ensure graduates would be prepared for careers in health informatics. To outline the history of medical informatics education in China, Hu et al [ ] systematically analyzed the status of medical informatics education at different academic levels and suggested reasonable strategies for the further development of the field in China. Asiri [ ] explored the challenges that face HIE and training in the Kingdom of Saudi Arabia by examining the state of health IT education, with a focus on HIE. Jeon et al [ ] presented the status of nursing informatics education and the content covered in nursing informatics courses in Korea, and concluded that a greater focus was needed on training faculty and developing the courses. Similarly, Masic et al [ ] investigated the status of medical informatics education in Bosnia and Herzegovina, and Europe.Although these studies made invaluable contributions to the understanding of HIE, they tended to focus on specific regions or disciplines and did not provide a global overview of research in HIE. Additionally, they did not use any topic modeling methods to reveal topic structure and evolutionary trends of this research field, and thus, the methodologies employed in these studies had some limitations. Meanwhile, there was a scarcity of comprehensive scientometric analysis in this research field. Therefore, this study provided a topic modeling–based scientometric analysis of scientific articles concerning HIE, and it analyzed aspects such as productive countries and institutions, scientific collaborations, and topics and trends of HIE research.
Limitations
The main limitation of this study was that only 1 database was searched to identify studies for inclusion, meaning that potentially relevant studies could have been overlooked. Although WoSCC is a widely used academic database globally, other medical-oriented databases may provide more significant academic and practical information. Subsequent studies should conduct searches in commonly used databases, such as PubMed, Scopus, IEEE, and Google Scholar, to ensure a more comprehensive analysis. Another constraint was that only journal articles were retrieved for analysis, excluding other relevant publication types in this field, such as reviews, book chapters, and conference proceedings. This could potentially affect the summarization and generalization of the results.
As AI has covered most of the research topics in health informatics, academic research on the intersection of AI and HIE has emerged in recent years. For example, Paranjape et al [
] summarized the state of medical education and recommended a framework to include AI education. Yang et al [ ] examined the impact of an AI system tutoring course on clinical training and found an increased improvement in the expert-led AI tutoring group. However, AI did not appear as a representative term in any of these topics. The main reason may be that the time span chosen for this study was long at 10 years, and AI did not dominate during this period. Overall, the application of AI in HIE should be taken seriously, and formal training for clinical and health informatics programs should consist of AI technology and its applications.Conclusion
The field of HIE is increasingly promising and active, with academic products flourishing. This study presents the results and findings from a topic-driven scientometric analysis of academic research articles concerning HIE published from 2014 to 2023. This study not only highlights the major research concerns and issues among scholars but also identifies the tendencies of article and citation counts; identifies major countries/regions, institutions, publication sources, and subjects; and outlines academic collaboration relationships. The topic-based scientometric analysis contributed to HIE by providing a comprehensive overview of its community. The contributions of this study with regard to HIE have been summarized.
First, it helps researchers, policymakers, and practitioners better understand the historical, extant, and upcoming structure of research in HIE. The exploration of major topics and their trends is helpful for the identification and comparison of current and potential scientific strength. Research governors or funding agencies can optimize research policies to promote potential competitive research areas and enhance scientific collaborations in specific research areas in order to bolster scientific and educational activities. Second, findings regarding prolific countries/regions and institutions can assist scholars in sharing research, conducting collaborations, and identifying important actors from whom to learn. This is valuable for the research community of HIE in terms of understanding its research status and trends, especially by translating the findings into directions for future research. Third, this study provides a clear outline of HIE, which could help interested educators as well as researchers who wish to continue working in this area. The results of prolific publication sources and subjects provide suggestions about sources to which scholars can submit their work and will help scholars to better position their research in the field.
As it explores the structures and research topics within the area of HIE, this study serves as a starting point for further in-depth research into HIE. The research scope of HIE is relatively broad, and this study is expected to provide editors and conference organizers with ways to identify gaps in the field, leading them to create special issues or conference sessions about topics that are generally not co-occurring, which could create potential new synergies in HIE.
Acknowledgments
This work was supported by Zhejiang Provincial Philosophy and Social Sciences Planning Project (reference number: 22JCXK11YB) and Zhejiang Provincial Education Sciences Planning Project (reference number: 2023SCG422).
Data Availability
The data and analysis code are available upon reasonable request.
Conflicts of Interest
None declared.
References
- Bichel-Findlay J, Koch S, Mantas J, Abdul SS, Al-Shorbaji N, Ammenwerth E, et al. Recommendations of the International Medical Informatics Association (IMIA) on education in biomedical and health informatics: second revision. Int J Med Inform. Feb 2023;170:104908. [CrossRef] [Medline]
- Amlung J, Huth H, Cullen T, Sequist T. Modernizing health information technology: lessons from healthcare delivery systems. JAMIA Open. Oct 2020;3(3):369-377. [FREE Full text] [CrossRef] [Medline]
- Friedman C, Wyatt J, Ash J. Evaluation Methods in Biomedical and Health Informatics. Cham. Springer; 2022.
- Hersh W. A stimulus to define informatics and health information technology. BMC Med Inform Decis Mak. May 15, 2009;9:24. [FREE Full text] [CrossRef] [Medline]
- Kushniruk A, Lau F, Borycki E, Protti D. The School of Health Information Science at the University of Victoria: towards an integrative model for health informatics education and research. Yearb Med Inform. Mar 07, 2018;15(01):159-165. [CrossRef]
- Kolokathi A, Hasman A, Chronaki C, Madsen I, Moen A, Randell R, et al. Education in biomedical and health informatics: a European perspective. Stud Health Technol Inform. Aug 21, 2019;264:1951-1952. [CrossRef] [Medline]
- Mantas J. The European educational programs in biomedical and health informatics. Stud Health Technol Inform. Jun 26, 2020;272:482-483. [CrossRef] [Medline]
- Alshammari R, Al-Shorbaji N, Ahmed Z, Al-Hamadan Z, Alshammari A, El Hassan O, et al. Middle East and North African Health Informatics Association (MENAHIA): a new opportunity for the region. Yearb Med Inform. Apr 25, 2019;28(01):292-296. [CrossRef]
- Murray PJ, Betts HJ, Wright G, Tshayingca-Mashiya NV. Health informatics education and capacity building in Eastern Cape Province, South Africa. Yearb Med Inform. Mar 07, 2018;18(01):158-163. [CrossRef]
- Were MC, Gong W, Balirwa P, Balugaba BE, Yeung A, Pierce L, et al. Comparative analysis of competency coverage within accredited master's in health informatics programs in the East African region. J Am Med Inform Assoc. Aug 13, 2021;28(9):1843-1848. [FREE Full text] [CrossRef] [Medline]
- Chen X, Xie H. A structural topic modeling-based bibliometric study of sentiment analysis literature. Cogn Comput. Jul 31, 2020;12(6):1097-1129. [CrossRef]
- Chen X, Wang S, Tang Y, Hao T. A bibliometric analysis of event detection in social media. OIR. Feb 11, 2019;43(1):29-52. [CrossRef]
- Lester CA, Wang M, Vydiswaran VGV. Describing the patient experience from Yelp reviews of community pharmacies. J Am Pharm Assoc (2003). 2019;59(3):349-355. [CrossRef] [Medline]
- Chen X, Zou D, Xie H. Fifty years of British Journal of Educational Technology: A topic modeling based bibliometric perspective. Brit J Educational Tech. Feb 05, 2020;51(3):692-708. [CrossRef]
- Hao T, Chen X, Li G, Yan J. A bibliometric analysis of text mining in medical research. Soft Comput. Sep 6, 2018;22(23):7875-7892. [CrossRef]
- Falagas ME, Pitsouni EI, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB J. Feb 2008;22(2):338-342. [CrossRef] [Medline]
- Wang Q, Waltman L. Large-scale analysis of the accuracy of the journal classification systems of Web of Science and Scopus. Journal of Informetrics. May 2016;10(2):347-364. [CrossRef]
- Saheb T, Saheb M. Analyzing and visualizing knowledge structures of health informatics from 1974 to 2018: a bibliometric and social network analysis. Healthc Inform Res. Apr 2019;25(2):61-72. [FREE Full text] [CrossRef] [Medline]
- Zou LX, Sun L. Visualization analysis of health informatics research from 2001 to 2018. Current Science. 2020;118(5):714-721. [FREE Full text]
- Chen X, Xie H, Wang FL, Liu Z, Xu J, Hao T. A bibliometric analysis of natural language processing in medical research. BMC Med Inform Decis Mak. Mar 22, 2018;18(Suppl 1):14. [FREE Full text] [CrossRef] [Medline]
- Cretchley J, Rooney D, Gallois C. Mapping a 40-year history with Leximancer: themes and concepts in the Journal of Cross-Cultural Psychology. Journal of Cross-Cultural Psychology. Apr 14, 2010;41(3):318-328. [CrossRef]
- Zhong S, Geng Y, Liu W, Gao C, Chen W. A bibliometric review on natural resource accounting during 1995–2014. Journal of Cleaner Production. Dec 2016;139:122-132. [CrossRef]
- Lee K, Jung H, Song M. Subject–method topic network analysis in communication studies. Scientometrics. Sep 30, 2016;109(3):1761-1787. [CrossRef]
- Yan E. Research dynamics, impact, and dissemination: A topic‐level analysis. Asso for Info Science & Tech. Jun 2015;66(11):2357-2372. [CrossRef]
- Hoffman M, Blei D, Bach F. Online learning for Latent Dirichlet Allocation. In: NIPS'10: Proceedings of the 23rd International Conference on Neural Information Processing Systems - Volume 1. 2010. Presented at: 23rd International Conference on Neural Information Processing Systems; December 6-9, 2010; Vancouver, British Columbia, Canada. [CrossRef]
- Roberts ME, Stewart BM, Tingley D. stm: An R package for structural topic models. J. Stat. Soft. 2019;91(2):1-40. [CrossRef]
- Blei D, Ng A, Jordan M. Latent dirichlet allocation. In: NIPS'01: Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic. 2001. Presented at: 14th International Conference on Neural Information Processing Systems: Natural and Synthetic; December 3-8, 2001; Vancouver, British Columbia, Canada. [CrossRef]
- Blei D, Lafferty J. Correlated topic models. In: NIPS'05: Proceedings of the 18th International Conference on Neural Information Processing Systems. 2005. Presented at: 18th International Conference on Neural Information Processing Systems; December 5-8, 2005; Vancouver, British Columbia, Canada. [CrossRef]
- Chen X, Xie H, Li Z, Cheng G, Leng M, Wang FL. Information fusion and artificial intelligence for smart healthcare: a bibliometric study. Information Processing & Management. Jan 2023;60(1):103113. [CrossRef]
- Levy KEC, Franklin M. Driving regulation. Social Science Computer Review. Dec 02, 2013;32(2):182-194. [CrossRef]
- Tvinnereim E, Fløttum K. Explaining topic prevalence in answers to open-ended survey questions about climate change. Nature Clim Change. Jun 1, 2015;5(8):744-747. [CrossRef]
- Chen X, Zou D, Cheng G, Xie H. Detecting latent topics and trends in educational technologies over four decades using structural topic modeling: A retrospective of all volumes of Computers and Education. Computers & Education. Jul 2020;151:103855. [CrossRef]
- Mai MV, Luo BT, Orenstein EW, Luberti AA. A model for clinical informatics education for residents: addressing an unmet need. Appl Clin Inform. Apr 2018;9(2):261-267. [FREE Full text] [CrossRef] [Medline]
- Matos Lamenha-Lins R, Maria de Carvelho Pugliesi D, José Camello de Lima F, Regina Oliveira Moreira A, Gonçalves Correia de Leite de Marcelos P, Dos Santos VE. Mobile application as a learning tool for improving dental students' knowledge regarding dental trauma. Eur J Dent Educ. Nov 2022;26(4):700-706. [CrossRef] [Medline]
- Garner SL, Young P, Fendt M, Koch H, George CE, Hitchcock J, et al. Effectiveness of a culturally responsive mHealth gaming application to improve diabetes health literacy in India: a randomized controlled trial. Comput Inform Nurs. Oct 01, 2023;41(10):796-804. [CrossRef] [Medline]
- Nezamdoust S, Abdekhoda M, Rahmani A. Determinant factors in adopting mobile health application in healthcare by nurses. BMC Med Inform Decis Mak. Feb 22, 2022;22(1):47. [FREE Full text] [CrossRef] [Medline]
- Informatics: Research and Practice. AMIA - American Medical Informatics Association. URL: https://www.amia.org/about-amia/why-informatics/informatics-research-and-practice [accessed 2024-02-05]
- Pian W, Khoo CS, Chi J. Automatic classification of users' health information need context: logistic regression analysis of mouse-click and eye-tracker data. J Med Internet Res. Dec 21, 2017;19(12):e424. [FREE Full text] [CrossRef] [Medline]
- Kraschnewski JL, Chuang CH, Poole ES, Peyton T, Blubaugh I, Pauli J, et al. Paging "Dr. Google": does technology fill the gap created by the prenatal care visit structure? Qualitative focus group study with pregnant women. J Med Internet Res. Jun 03, 2014;16(6):e147. [FREE Full text] [CrossRef] [Medline]
- Vamos CA, Merrell L, Detman L, Louis J, Daley E. Exploring women's experiences in accessing, understanding, appraising, and applying health information during pregnancy. J Midwifery Womens Health. Jul 2019;64(4):472-480. [CrossRef] [Medline]
- Wang W, Sun R, Mulvehill AM, Gilson CC, Huang LL. Handling internet-based health information: improving health information web site literacy among undergraduate nursing students. J Nurs Educ. Feb 01, 2017;56(2):110-114. [CrossRef] [Medline]
- Lim HM, Wong SS, Yip KC, Chang FWS, Chin AJZ, Teo CH, et al. Online health information-seeking behaviour of patients attending a primary care clinic in Malaysia: a cross-sectional study. Fam Pract. Jan 19, 2022;39(1):38-45. [CrossRef] [Medline]
- Jidkov L, Alexander M, Bark P, Williams JG, Kay J, Taylor P, et al. Health informatics competencies in postgraduate medical education and training in the UK: a mixed methods study. BMJ Open. Mar 30, 2019;9(3):e025460. [FREE Full text] [CrossRef] [Medline]
- Berner ES, Dorsey AD, Garrie RL, Qu H. Assessment-based health informatics curriculum improvement. J Am Med Inform Assoc. Jul 2016;23(4):813-818. [FREE Full text] [CrossRef] [Medline]
- Valenta AL, Berner ES, Boren SA, Deckard GJ, Eldredge C, Fridsma DB, et al. AMIA Board White Paper: AMIA 2017 core competencies for applied health informatics education at the master's degree level. J Am Med Inform Assoc. Dec 01, 2018;25(12):1657-1668. [FREE Full text] [CrossRef] [Medline]
- Sapci AH, Sapci HA. Teaching hands-on informatics skills to future health informaticians: a competency framework proposal and analysis of health care informatics curricula. JMIR Med Inform. Jan 21, 2020;8(1):e15748. [FREE Full text] [CrossRef] [Medline]
- Alzghaibi H. The gap between bachelor's degree graduates in health informatics and employer needs in Saudi Arabia. BMC Med Educ. Jun 26, 2023;23(1):475. [FREE Full text] [CrossRef] [Medline]
- McLane TM, Hoyt R, Hodge C, Weinfurter E, Reardon EE, Monsen KA. What industry wants: an empirical analysis of health informatics job postings. Appl Clin Inform. Mar 31, 2021;12(2):285-292. [FREE Full text] [CrossRef] [Medline]
- Hovenga E, Grain H. Learning, training and teaching of health informatics and its evidence for informaticians and clinical practice. Stud Health Technol Inform. 2016;222:336-354. [Medline]
- Yasnoff WA, O'Carroll PW, Koo D, Linkins RW, Kilbourne EM. Public health informatics: improving and transforming public health in the information age. J Public Health Manag Pract. Nov 2000;6(6):67-75. [CrossRef] [Medline]
- Gamache R, Kharrazi H, Weiner JP. Public and population health informatics: the bridging of big data to benefit communities. Yearb Med Inform. Aug 2018;27(1):199-206. [FREE Full text] [CrossRef] [Medline]
- Crawford A, Serhal E. Digital health equity and COVID-19: the innovation curve cannot reinforce the social gradient of health. J Med Internet Res. Jun 02, 2020;22(6):e19361. [FREE Full text] [CrossRef] [Medline]
- Global strategy on digital health 2020-2025. World Health Organization. 2021. URL: https://www.who.int/publications/i/item/9789240020924 [accessed 2024-12-01]
- Poncette A, Glauert DL, Mosch L, Braune K, Balzer F, Back D. Undergraduate medical competencies in digital health and curricular module development: mixed methods study. J Med Internet Res. Oct 29, 2020;22(10):e22161. [FREE Full text] [CrossRef] [Medline]
- Roberts J. Developing health informatics as a recognised professional domain supporting clinical and health management activity. World Hosp Health Serv. 2006;42(4):38-40. [Medline]
- Cox SM, Johnson SB, Shiu E, Boren S. The national landscape of culminating experiences in master's programs in health and biomedical informatics. J Am Med Inform Assoc. Mar 18, 2021;28(4):856-861. [FREE Full text] [CrossRef] [Medline]
- Bellazzi R, Cecconi M, Costantino ML, Veltri P. Bioengineering and medical informatics education in MD programs: perspectives from three Italian experiences. Int J Med Inform. Apr 2023;172:105002. [FREE Full text] [CrossRef] [Medline]
- Moen A, Mæland Knudsen L. Nursing informatics: decades of contribution to health informatics. Healthc Inform Res. Jun 2013;19(2):86-92. [FREE Full text] [CrossRef] [Medline]
- Risling T. Educating the nurses of 2025: Technology trends of the next decade. Nurse Educ Pract. Jan 2017;22:89-92. [CrossRef] [Medline]
- Jouparinejad S, Foroughameri G, Khajouei R, Farokhzadian J. Improving the informatics competency of critical care nurses: results of an interventional study in the southeast of Iran. BMC Med Inform Decis Mak. Sep 11, 2020;20(1):220. [FREE Full text] [CrossRef] [Medline]
- Egbert N, Thye J, Hackl WO, Müller-Staub M, Ammenwerth E, Hübner U. Competencies for nursing in a digital world. Methodology, results, and use of the DACH-recommendations for nursing informatics core competency areas in Austria, Germany, and Switzerland. Inform Health Soc Care. 2019;44(4):351-375. [CrossRef] [Medline]
- Monkman H, Mir S, Borycki EM, Courtney KL, Bond J, Kushniruk AW. Updating professional competencies in health informatics: A scoping review and consultation with subject matter experts. Int J Med Inform. Feb 2023;170:104969. [CrossRef] [Medline]
- Hu D, Sun Z, Li H. An overview of medical informatics education in China. Int J Med Inform. May 2013;82(5):448-466. [CrossRef] [Medline]
- Asiri HA. Challenges of the health informatics education in the Kingdom of Saudi Arabia: What stands in our way? Journal of Health Informatics in Developing Countries. 2014;8(1):114. [FREE Full text]
- Jeon E, Kim J, Park H, Lee J, Kim J, Jin M, et al. Current status of nursing informatics education in Korea. Healthc Inform Res. Apr 2016;22(2):142-150. [FREE Full text] [CrossRef] [Medline]
- Masic I, Pandza H. Medical informatics education - past, today and future. ejbi. 2018;14(2):40-45. [CrossRef]
- Paranjape K, Schinkel M, Nannan Panday R, Car J, Nanayakkara P. Introducing artificial intelligence training in medical education. JMIR Med Educ. Dec 03, 2019;5(2):e16048. [FREE Full text] [CrossRef] [Medline]
- Yang Y, Shulruf B. Expert-led and artificial intelligence (AI) system-assisted tutoring course increase confidence of Chinese medical interns on suturing and ligature skills: prospective pilot study. J Educ Eval Health Prof. 2019;16:7. [FREE Full text] [CrossRef] [Medline]
Abbreviations
AI: artificial intelligence |
HIE: health informatics education |
LDA: latent Dirichlet allocation |
STM: structural topic model |
WoSCC: Web of Science Core Collection |
Edited by B Lesselroth; submitted 08.03.24; peer-reviewed by J Wang, MS Zami; comments to author 19.05.24; revised version received 13.09.24; accepted 24.11.24; published 11.12.24.
Copyright©Qing Han. Originally published in JMIR Medical Education (https://mededu.jmir.org), 11.12.2024.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on https://mededu.jmir.org/, as well as this copyright and license information must be included.