AI Literacy Among Chinese Medical Students: Cross-Sectional Examination of Individual and Environmental Factors

Introduction

Artificial intelligence’s (AI’s) rapid advancement in diagnostic support, treatment decision-making, personalized care, and health system management is quickly reshaping health care service delivery [1-4]. To prepare medical trainees in using these tools [5], medical training programs need to efficiently leverage their limited resources to set up the relevant training approaches and programs to meet the most important needs. However, instead of identifying such gaps, research has mostly focused on applying AI in medical education across tasks, including, for example, admission, research, and evaluation [6-17]. The developments of AI-targeted curricula and competencies have so far advanced without high-quality evidence of trainees’ needs [18-22], highlighting the questionable appropriateness of such proposals.

Among the various AI skills and competencies, clarifying trainees’ AI literacy appears is especially important. AI literacy refers to a set of competencies that enable individuals to critically evaluate AI technologies, communicate and collaborate effectively with AI, and use AI as a tool across various contexts [23]. While various frameworks for AI literacy exist, there is growing consensus that it comprises cognitive knowledge, attitudes, and practical behaviors toward AI tools [5,24-26].

Building on this consensus, we defined AI literacy as a multidimensional construct encompassing knowledge, attitude, and behavior, integrating both classical and contemporary theoretical models. Specifically, this tripartite structure aligns with the ABC model of attitudes in social psychology (Affect–Behavior–Cognition), which conceptualizes human responses as comprising cognitive, affective, and behavioral components [27-29].

Current evidence on AI literacy in medical trainees has been hindered by theoretical and methodological deficiencies. First, the theoretical weakness stems from the lack of recognition that AI literacy has multiple dimensions. Most have focused on AI knowledge [30-33] or attitudes toward AI applications [34-38]. This misses the important dimension of AI behaviors [5,24,25,39,40], which account for the actual usage patterns of AI tools. Second, methodologically, the few studies that captured all three dimensions suffered from capturing only a small number of medical schools and sample size [41,42]. This limits their generalizability. Third, the studies often fail to examine the factors associated with subdimensions of AI literacy, which limits the ability to design targeted interventions within a limited budget. Existing studies have often focused mainly on individual-level variables, overlooking environmental factors associated with the digital divide [41,43-45].

We addressed these gaps by drawing on the comprehensive quantitative database of Chinese medical students, the 2024 China Medical Student Survey (CMSS). We report the distribution of AI literacy across cognitive, attitudinal, and behavioral dimensions among 80,355 Chinese medical students and their associated individual and environmental factors. This large sample size across all three dimensions of AI literacy provides a more meaningful representation of the potential educational needs. We further analyzed the potential factors associated with AI literacy, enabling medical educators to design targeted interventions.

Methods

Data Source and Study Sample

Since 2019, China’s National Center for Health Professions Education Development has conducted an annual nationwide survey of medical students in China known as the CMSS. It aims to support medical education development by collecting comprehensive information across the entire educational journey, including students’ demographic backgrounds, preadmission experiences, academic training, and postgraduation plans [46]. The survey takes place between May and July each year. Additional details on the context of medical education in China and the CMSS are provided in Items 1 and 2 in Multimedia Appendix 1. All data analyses adhered to the (STROBE) Strengthening the Reporting of Observational Studies in Epidemiology guidelines for cross-sectional studies.

This study used data from the 2024 CMSS, which included participants from 109 medical schools across 28 provinces in China. The survey achieved a 73.9% response rate (80,355/108,710 surveyed) and captured 109 of 202 medical schools (109/202, 53.9%). The geographical distribution of participating institutions roughly reflected the national pattern. Comparing the dataset and the national distribution, respectively, 49.5% (54/109) and 45.0% (91/202) of the schools were from the eastern region, 28.4% (31/109) and 30.7 % (62/202) from the central region, and 22.0% (24/109) and 24.3% (49/202) from the western region. As for the tiers of institutions, the data included universities’ designation as ‘Double First-Class’ (DFC) institutions. These institutions garner priority investment in China’s bid to establish world-leading educational institutions [47] (see Item 3 in Multimedia Appendix 1 for more details). Further, the proportion of DFC universities was similar in the dataset and nation was (25.7%, 28/109 vs 23.3%, 47/202), respectively.

Ethical Considerations

The project was approved by the Peking University Institutional Review Board (Beijing, China; approval Number: IRB00001052- 20069). Participation was voluntary, and completion and submission of the questionnaire were considered to constitute informed consent. To protect participant privacy and confidentiality, no personally identifiable information was collected, and all responses were analyzed in an anonymized form. The data were stored securely and were accessible only to the research team. Participants did not receive any financial or other compensation for their participation.

Measurement of AI Literacy

AI literacy is a multidimensional construct comprising cognitive, attitudinal, and behavioral dimensions [24,25,40]. In this study, we refer to the cognitive dimension as “AI knowledge” to improve clarity and align with common usage in educational research. AI knowledge was assessed through four items evaluating students’ self-perceived proficiency in core areas of medical AI: basic concepts of AI, machine learning tools, multimodal medical data analysis, and ethics issues related to AI. Responses were rated on a three-point scale coded as 1 (low), 2 (medium), and 3 (high) proficiency. Students’ attitude towards AI was measured using two items: students’ self-perceived view of using AI tools in (1) teaching and (2) learning. Responses were rated on a three-point scale, coded as 1 (negative), 2 (neutral), and 3 (positive), respectively. AI behavior refers to actual use of AI tools rather than behavioral intention and was evaluated through self-perceived usage frequency and application patterns. Usage frequency was assessed on a six-point scale (ranging from “never used” to “multiple times daily,” coded one to six, where higher scores indicated more frequent academic use). Usage patterns were measured with two items: extent of using AI for (1) professional knowledge learning and (2) writing tasks (options included “never used,” “generating initial drafts,” “writing specific sections,” “editing and polishing,” “outlining,” and “brainstorming ideas”). The Cronbach α values for cognitive, attitudinal, and behavioral dimension were 0.81, 0.86, and 0.79, respectively, indicating high internal consistency. The Kaiser-Meyer-Olkin values were 0.76, 0.70, and 0.75 for the three dimensions. Additionally, Bartlett’s test of sphericity was significant (P<.001), supporting the construct validity of the measurement scales. Further confirmatory factor analysis showed that the measurement instrument loaded well onto the three identified dimensions of AI literacy. Relevant statistics include comparative fit index (CFI)=0.994, Tucker-Lewis index (TLI)=0.991, Root Mean Square Error of Approximation (RMSEA)=0.025, and statistically significant loading on all factors (P<.001).

Factors Associated With AI Literacy

Existing research suggests that both individual and environmental factors associate with access to, and use of digital technologies [45]. For individual factors, we examined demographic characteristics, family background, intellectual ability, and enrollment motivation [47-49]. Demographic characteristics included sex (female or male), ethnicity (Han Chinese or other), only child (yes or no). Family background included hometown (urban or rural), father’s education, mother’s education, having at least one parent as a physician (yes or no), and high-income families (total family income from the previous year>150,000 RMB, approximately US $21,385, yes or no). We used the National College Entrance Examination scores to measure the respondents’ intellectual ability. We assessed enrollment motivation influencing students’ choice of clinical medicine following previous work [47]. Intrinsic motivation was measured by students’ reported interest in medicine, confidence in achieving success in the field, and strong performance in relevant high school subjects. Extrinsic motivation was evaluated through responses about employment prospects, encouragement (or requirements) from significant others, and anticipated convenience in accessing health care resources for themselves and family members.

Regarding environmental factors, we considered students’ affiliation with DFC universities, enrollment in long-track programs (ie, 5+3 or 8 y medical education program instead of 5 y medical education program; yes or no), and the educational phase (preclinical or clinical).

Statistical Analysis

We used the principal component analysis to aggregate the AI knowledge and AI attitude items, followed by Min-Max normalization to standardize the scores. Based on expert consultation and author consensus, we aggregated the AI behavior index using a weighted arithmetic mean of three key indicators: (1) usage frequency (50% weight), (2) extent of AI integration in professional knowledge learning (30% weight), and (3) AI utilization in writing tasks (20% weight). For robustness check, we also applied equal weights across all three indicators. We then applied the Min-Max normalization method, scaling the data so that the minimum and maximum values correspond to 0 and 100, respectively. Given that students were clustered within schools, we first fitted an unconditional (null) multilevel model to estimate the intraclass correlation coefficient (ICC) and assess the proportion of variance in the standardized AI literacy scores attributable to between-school differences. The ICCs for the three dimensions of AI literacy—knowledge, attitude, and behavior—were 4.0%, 1.0%, and 4.1%, respectively. These results indicate that only a small proportion of the total variance was explained by school-level clustering, suggesting minimal between-school effects. Therefore, we proceeded with multivariate linear regression to examine the associations between individual factors (ie, demographic characteristics, family background, and enrollment motivation) and environmental factors (ie, educational phase, type of education program, and tier of education program) and AI literacy. We conducted subgroup analyses and examined formal interaction terms based on institutional tier. Both classical and school-clustered standard errors were used to calculate 95% CI, and statistical significance was set at a two-sided P< 0.05. All analyses were performed using Stata (version 18.0; StataCorp LLC) between December 19, 2024, and March 18, 2025.

Results

Participant Characteristics

Of the 80,355 medical students included, most were Han (69,333 students, 86.3%) and female (41,227, 51.3%). A total of 28,885 students (35.9%) reported being the only child; 48,716 (60.6%) were from urban areas; 29,152 (36.3%) reported having at least one parent as a physician. In addition, 13,736 (17.1%) came from high-income families, 36,041 (44.9%) were in the clinical phase of training, and 5823 (7.2%) were enrolled in long-track medical education programs (Table 1).

When stratified by DFC status, students from DFCs came from relatively more advantaged families—with higher household incomes, more parental education, and greater likelihood of urban origin—and also had significantly higher college entrance examination scores (Table 1).

Distribution of the Three Dimensions of AI Literacy

In our analysis of the three dimensions of AI literacy among Chinese medical students (Table 2), the scores declined from knowledge (mean 76.0, SD 26.9) to attitude (mean 71.6, SD 24.4) and then further to behavior (mean 32.5, SD 28.5). When we stratified by institutional tier, students from DFC universities reported lower AI knowledge (mean 72.6, SD 28.6 vs mean 76.5, SD 26.7; P<.001) but higher attitude (mean 72.6, SD 24.6 vs mean 71.4, SD 24.4; P<.001), and behavior (mean 35.0, SD 28.4 vs mean 32.1, SD 28.5; P<.001).

Figure 1 further illustrates the item-level patterns within each dimension of AI literacy. The descending trend—from higher scores in knowledge to lower scores in attitude and the lowest in behavior—was consistent across institutional tiers. Notably, students from DFC universities scored higher on AI behavior subcomponents, especially in professional learning and writing tasks. More detailed results on each institution are provided in Figure S1 in Multimedia Appendix 1.

‎

The figure shows the distribution of the subcomponents of the three dimensions of AI literacy by institutional tier. The AI knowledge dimension comprised students’ self-reported proficiency in core areas of medical AI, including (1) basic concepts of AI, (2) machine learning tools, (3) multimodal medical data analysis, and (4) ethics issues related to AI. The AI attitude dimension included students’ view of using AI tools in (1) teaching and (2) learning. The AI behavior dimension was measured by usage frequency and application patterns, including AI use for professional knowledge learning and writing tasks.

Factors Associated With AI Knowledge

Figure 2 shows the results of regression analyzes of the individual and environmental factors associated with AI literacy (more details including robustness check results are available in Tables S1-S4 in Multimedia Appendix 1). Male students were slightly more likely to report higher AI knowledge (β=0.02, 95% CI –0.02 to 0.06; P=.30). Both intrinsic (β=0.17, 95% CI 0.14 to 0.19; P<.001) and extrinsic motivation (β=0.12, 95% CI, 0.10 to 0.14; P<.001) were positively associated with AI knowledge. Students enrolled in long-track programs reported lower AI knowledge than those in the five-year programs (β=−0.10, 95% CI −0.19 to −0.01; P=.04). When stratified by DFC status, these patterns remained largely consistent. However, students from DFC universities reported significantly less AI knowledge (β=−0.10, 95% CI −0.16 to −0.03; P=.003).

‎

Factors Associated With AI Attitude

Male students (β=0.05, 95% CI 0.04 to 0.06; P<.001) demonstrated significantly more positive attitudes toward AI. Both intrinsic (β=0.09, 95% CI 0.08 to 0.10; P<.001) and extrinsic motivation (β=0.07, 95% CI 0.06 to 0.08; P<.001) were positively associated with AI attitude. Students from high-income families (β=0.05, 95% CI 0.03 to 0.07; P<.001), with at least one physician parent (β=0.03, 95% CI 0.01 to 0.04; P<.001), and urban backgrounds (β=0.04, 95% CI 0.02 to 0.06; P<.001) showed more positive AI attitude. As for environmental factors, students enrolled in long-track programs (β=0.04, 95% CI 0.01 to 0.07; P<.006) and in clinical phase of training (β=0.02, 95% CI 0.01 to 0.03; P<.003) showed significantly more positive attitudes toward AI. These patterns remained largely consistent in subgroup analyses by DFC status; however, among the non-DFC students, the associations of family income and parental medical background on AI attitude were significantly stronger.

Factors Associated With AI Behavior

Males (β=0.20, 95% CI 0.19 to 0.22; P<.001) demonstrated higher levels of AI behavior. Students from high-income families (β=0.07, 95% CI 0.05 to 0.09; P<.001) and those with at least one physician parent (β=0.03, 95% CI 0.01 to 0.04; P<.001) showed higher AI behavior scores. Both intrinsic (β=0.08, 95% CI 0.07 to 0.08; P<.001) and extrinsic motivation (β=0.05, 95% CI 0.04 to 0.05; P<.001) were positively associated with AI behavior. Students enrolled in long-track programs (β=0.18, 95% CI 0.15 to 0.20; P<.001) exhibited higher AI behavior. Again, these patterns were largely held in subgroup analysis by DFC status. In the non-DFC group, however, the effect of family income and parental medical background was again stronger.

Figure 3 illustrates the associations between individual factors and AI literacy across different institutional environments.

‎

Discussion

Principal Findings

Despite the recognized importance of preparing medical students to use AI tools effectively, systematic evidence on the subcomponents of AI literacy and the associated factors is limited. To our knowledge, this is the first large-scale, multicenter, national-level study on the multidimensional constructs of AI literacy and their associated individual and environmental factors among medical students. Three key results emerged. First, medical students reported moderate to high AI knowledge, moderate AI attitude, but only low AI behavior scores. Second, both individual and environmental factors were associated with different dimensions of AI literacy. Third, compared to less favorable environmental conditions, the association between worse AI literacy and family income and parental medical background may be weaker.

This study provides several significant contributions. First, the use of a large, multicenter, and nationally representative sample—covering more than 50% (109/202) of the medical schools in China—expands beyond the small, single-center settings from previous studies. Second, by comprehensively examining the cognitive, attitudinal, and behavioral dimensions of AI literacy, this study provides a more holistic understanding of medical students’ AI literacy profiles, identifying nuanced gaps and interventional needs within AI literacy, particularly around attitude and behaviors. Third, our comprehensive examination of the individual differences and environmental factors associated with AI literacy provides an evidence-based foundation for developing tailored interventions. Understanding these associations enables medical educators to design targeted programs that meet the diverse needs of student populations, potentially reducing disparities in AI literacy and ensuring that all future physicians are adequately prepared to engage with an increasingly AI-integrated health care system.

Our findings consistently revealed that self-perceived AI knowledge scores surpassed attitude and behavior. This suggests that while students developed a foundational understanding of AI and generally perceived AI integration in medicine positively, they still lacked practical experience in its application. Although this defies common logic that deeper knowledge should portend more use [50], this trend aligns with previous international studies of medical staff and trainees who held generally positive attitudes toward AI despite low exposure in formal use [36,51].

The potential barriers underlying this knowledge–use gap are as follows. First, structural barriers—such as limited institutional infrastructure, insufficient curricular integration, and time constraints—may constrain students’ ability to translate AI knowledge into practice [52]. Such challenges are particularly salient in the Chinese healthcare systems, where physician shortages drive each health care provider to carry significant patient load [53]. These contextual limitations can, in turn, dampen students’ autonomous motivation to explore alternative tools [54]. Second, sociocultural norms may also constrain AI use [55]. A recent study showed that nearly half of Chinese medical staff (1320/2705, 48.8%) expressed concerns about the ethical safety, accuracy, interpretability, and fairness of AI systems [56]. Such ethical reservations from staff providers to medical trainees, leading them to perceive AI use as potentially inappropriate, dishonest, unprofessional, or harmful. This may then limit the students’ willingness to use the AI tools despite being knowledgeable of AI tools’ capabilities. suggesting that moral and risk considerations may underlie the limited AI use observed among some medical students. Third, our survey was conducted between May and July 2024, when most mature and user-friendly AI tools were developed abroad. Chinese students may have faced barriers related to access [57] and cultural appropriateness [58].

At an individual level, students’ demographics, family background, and motivation were all associated with AI literacy. The patterns generally aligned with the literature. Demographically, male medical students scored higher in AI attitude and behavior. This is consistent with previous findings [50,59] and the phenomenon of a “technological gender gap” [60]. Male students tended to perceive AI tools as more useful and easier to use [61-64]. Additionally, individuals from advantaged family background (ie, urban residents, those with at least one physician parent, and students from high-income families) perceived AI more favorably and used it more frequently. These advantaged groups hold greater material, cognitive, social, and cultural capital that can facilitate technological adoption [65]. Although technological innovations eventually reach broader populations, disparities in access and adoption persist during the diffusion process, with advantaged groups generally adopting innovations earlier and using them more effectively, thus creating a digital divide between socioeconomic groups [65,66].

Notably, both intrinsic and extrinsic motivation were positively associated with all dimensions of AI literacy, with intrinsic motivation showing particularly strong effects. According to self-determination theory, intrinsic motivation is associated with better performance and enhanced subjective well-being, while extrinsic motivation can also drive performance, even filtering into intrinsic motivation under certain conditions [67-69]. Intrinsic motivation serves as a natural source of AI learning and achievement, fostering greater autonomy, exploration, and reflection that promote deeper AI engagement. In contrast, extrinsically motivated students may preferentially recognize the instrumental value of AI but limit autonomous exploration of AI tools.

Our study highlights the significant role of the environment in shaping AI literacy. Specifically, DFC students demonstrated higher AI behavior scores. The DFC Project represents China’s willingness to invest financial and human resources in these institutions to develop world-class universities and disciplines [47,70]. The stronger student–faculty and peer interactions and higher-quality student efforts may promote deeper engagement with AI [71]. The seemingly paradoxical finding that DFC students reported lower AI knowledge but higher AI behavior may stem from the Dunning–Kruger effect [72,73]. More competent individuals tend to recognize the complexity of a domain and their own knowledge limitations, leading to more cautious self-assessments, while those less competent may overestimate their understanding. DFC students, exposed to higher levels of academic research, may be more aware of the challenges inherent in AI, whereas non-DFC students, with relatively limited exposure, may exhibit overconfidence in their basic knowledge. Similar cognitive bias patterns have been observed among health professions students; for example, low-performing medical students have been shown to substantially overestimate their academic performance relative to peers [74,75].

Importantly, in these DFC supportive environments, the students’ family income and parental medical background showed less pronounced associations with AI attitude and behavior, suggesting that the supportive learning environment may have mitigated the detrimental relationship between students’ background and AI literacy [48].

In addition, the students’ program also appears to matter. Students in the long-track program demonstrated significantly higher AI attitude and behavior scores. They typically represent China’s elite medical students, attending research-oriented universities that provide enhanced research training opportunities and funding [76], potentially including familiarization with AI technologies that led to more positive perceptions and use of AI tools. The students’ phase of training also appeared to be associated with their AI literacy. Students undergoing clinical training reported higher attitude scores but lower knowledge and behavior scores. Perhaps the complexity of clinical practice and the pressure of professional development precipitated cognitive overload and a preferential reliance on traditional tools [77]. The technical limitations and evolving normative and legal thresholds around AI technologies may further exacerbate technology avoidance [78-82].

Tying together the findings, we provide five sets of recommendations to help medical schools maneuver through the current lack of structured AI education and training [83]. First, carefully integrating AI-related content into the curriculum to generate practical AI exposure may help bridge the knowledge-use gap. This should be mindful of the potentially detrimental association with excessive cognitive offloading [84] while ensuring a minimal knowledge and skills threshold [85]. As regulations and ethical frameworks surrounding AI use in health care continue to evolve to meet the technical, ethical, and legal challenges [83,86,87], developing interdisciplinary training on practical risks, benefits, and guidelines on AI usage may help establish norms around AI usage. Second, developing and implementing scalable AI education frameworks that enable targeted training based on learners’ backgrounds can potentially reduce inequities in AI literacy driven by individual and environmental differences while maximizing educational impact.

The next three recommendations focus specifically on digital divide, and we draw on Han and Kumwenda’s framework for mitigating digital divide. The framework considers first-order barriers relate to infrastructure and access; second-order barriers include motivation and engagement; and third-order barriers reflect skill and training disparities [88].

First, medical educators and policy makers should prioritize investments in digital infrastructure to reduce disparities in technological access across institutional and regional contexts [89]. Targeted funding and digital resource allocation can ensure that students in under-resourced settings are not excluded from AI-driven learning environments.

Second, improving motivation and engagement may incorporate experiential learning opportunities [87,90,91]. Providing all students—particularly those from disadvantaged backgrounds—with the chance to work directly with AI tools and integrate them into their coursework can foster greater motivation, engagement, and confidence in using AI technologies.

Third, structured and scalable AI education frameworks that offer tiered and targeted training that adapts to learners’ backgrounds. Such approaches would focus on ensuring that disadvantaged students can develop foundational AI literacy before advancing to higher-order skills. Moreover, the development of multilingual and culturally adaptive AI models [92] can further ensure inclusivity and contextual relevance in AI-enhanced medical education.

Limitations

First, the reliance on self-reported data potentially exposes the study to social desirability bias. It is necessary to develop objective instruments to measure medical students’ AI literacy. Such a tool can help evaluate AI proficiency based on observable AI use. Performance- and scenario-based assessments might be potential avenues to operationalize such evaluation. Second, given the rapid development of AI technologies, the AI literacy of medical students is likely to evolve rapidly. To capture these dynamic shifts of AI literacy, we are actively exploring another round of survey. Third, this study employed a cross-sectional design, which precludes causal inference. Although most individual and environmental factors were determined before the measurement of AI literacy, reverse causality cannot be fully excluded, and unmeasured confounders may have influenced both the predictors and outcomes, potentially biasing the observed associations. Fourth, even though the unique premedical training structure limits the generalizability of the findings, the methodological framework and multidimensional conceptualization of AI literacy (knowledge, attitude, and behavior) may still provide a useful reference for future comparative studies or adaptations in other educational systems.

Conclusions

This cross-sectional study revealed that medical students exhibited the highest performance in AI knowledge, followed by attitude, and then behavior. Both individual characteristics and environmental factors were significantly associated with AI literacy, and environmental factors moderated individual variations in AI literacy. Integrating practical AI-related training into medical curricula through interdisciplinary collaboration, coupled with targeted interventions for students according to their backgrounds, may help prepare future physicians to effectively engage with AI technologies in medical practice.