Adaptation of the Japanese Version of the 12-Item Attitudes Towards Artificial Intelligence Scale for Medical Trainees: Multicenter Development and Validation Study

Introduction

Artificial intelligence (AI) is undergoing rapid development and integration into health care [1,2]. Although the use of AI has raised a number of potential concerns, including a lack of control over the rapidly progressing technology [3], issues of privacy and data protection [4], ethical problems [5], and the potential loss of human jobs [6], the benefits of AI are expected to far outweigh these concerns. AI contributes to improving patient care through enhancing diagnostic accuracy and providing more personalized therapeutic plans [7]. Moreover, AI has the potential to enhance the training and education of medical professionals by offering benefits including virtual simulation and training, remote education, and the recording of teaching videos [8,9]. Consequently, the use of AI has increased in both clinical practice and medical education.

Individuals may hold divergent views on the prospects and risks of AI and adopt varied attitudes toward it [10]. The attitudes of physicians and medical trainees toward AI will have a major impact on its integration into medical practice and educational activities [11]. Negative attitudes toward AI can lead to skepticism and concerns, thereby impeding its adoption [12]. Conversely, positive attitudes toward AI likely foster trust [13], leading individuals to embrace AI systems and amplify their benefits and possibly resulting in their integration into health care [12].

Thus, assessing physicians’ and medical trainees’ attitudes toward AI is critical to identifying potential barriers and fostering acceptance. Accordingly, instruments with robust psychometric properties are required. Although several measures for assessing attitudes toward AI have been developed, only a few have been comprehensively evaluated for reliability and validity. In 2024, Stein et al [10] developed the 12-item Attitudes Towards Artificial Intelligence scale (ATTARI-12). This is a unidimensional scale that integrates affective, behavioral, and cognitive facets into a single measure. The authors subsequently confirmed its psychometric properties for the general population [10], and its wide use in future studies is anticipated.

Despite the potential global applicability of the ATTARI-12, there is currently no validated Japanese version. Japan has a distinct cultural context, and attitudes toward AI are likely influenced by cultural and societal norms. For instance, Japanese culture’s high uncertainty avoidance tends to result in only a gradual adoption of new technologies [14]. In the absence of a culturally adapted measure, assessments may not accurately reflect the true sentiments of Japanese health care workers and trainees. This gap hinders research and educational efforts aimed at understanding and improving AI integration into the Japanese health care system.

In this study, we developed a Japanese version of the ATTARI-12, which was originally created for the general population, and examined whether it is applicable to medical students and resident physicians. We anticipated that the development of this scale in Japan would promote future research and educational courses that effectively assess and improve the integration of AI into clinical practice and the medical education system.

Methods

This study formed part of a larger research project regarding AI education for medical trainees. We conducted this study in the following 2 steps.

Step 1: Translation and Cross-Cultural Adaptation

In accordance with an international guideline [15], we translated the original ATTARI-12 into Japanese. First, the first author (HF) asked the creator of the original ATTARI-12 and corresponding author of the article that reported it to allow us to develop the Japanese version. The creator willingly provided permission for our translation of the scale. Second, 2 translators (HF and KK) conducted forward translations independently. Both translators were familiar with Western and Japanese cultures and had rich experience in developing translated versions of scales in the field of health profession education. In particular, KK is a fluent speaker of Japanese and English. Third, the translators performed a synthesis of their translations. Discrepancies were resolved through discussion (version 1). Fourth, HF asked professional bilingual translators who were not involved in our study to translate the Japanese text back into English. HF and KK then compared the back-translated and original English versions item by item and then revised the Japanese version (version 2). Fifth, an expert review of version 2 was conducted by an AI expert (YY) and a health profession education expert (YN) at HF’s request. These experts concluded that no amendment was required. Sixth, HF contacted the creator of the original scale again and asked him to check version 2. The creator concluded that no revision was necessary. Seventh, pilot testing was performed with 2 medical trainees, which indicated no problematic items. Finally, version 2 was confirmed as the finalized Japanese version of the ATTARI-12 (J-ATTARI-12).

Step 2: Investigation of Psychometric Properties

Participants

Between June 2025 and July 2025, we recruited study participants from 5 universities and 9 hospitals across Japan, each of which varied in type and location (Multimedia Appendix 1). We asked the medical education directors of the universities and residency training directors of the hospitals to distribute our anonymous online self-administered questionnaire. An email with a link to the online questionnaire was sent to medical students at the universities and resident physicians at the hospitals via the respective directors. The participants were provided with a brief explanation of the study and indicated their consent to take part by checking the consent box. They were then able to access and complete the questionnaire. The survey duration was 1 month. To maximize the response rate, we sent reminders several times during the survey period.

Measures

The original version of the ATTARI-12 has 12 items [10], with responses on a 5-point Likert scale (1=“strongly disagree”; 5=“strongly agree”) [10]. Items 2, 4, 7, 8, 10, and 12 are reverse items and, therefore, are reverse scored [10]. The sum of all item scores is averaged to create a total score ranging from 1 to 5, with higher scores indicating more positive attitudes toward AI.

Statistical Analysis

To investigate the structural validity of the instrument, we performed exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Because conducting these 2 types of factor analysis on the same dataset would potentially raise concerns [16], we adopted the split-half validation approach, in which the participants were randomly divided into 2 groups, half (group A) for EFA and the other half (group B) for CFA. As mentioned above, we aimed to develop the scale in a manner that was culturally adapted to the Japanese medical education context. Consequently, EFA was conducted first, followed by CFA.

To determine the appropriateness of EFA, we checked the Kaiser-Meyer-Olkin (KMO) coefficient and Bartlett sphericity test. Running EFA requires a KMO value over 0.80 and a significant result in the Bartlett test [17,18]. We applied EFA to the responses of group A to explore the factor structure of the J-ATTARI-12 using maximum likelihood estimation and Promax rotation. We determined the final factor solution using the results of parallel analysis and the factor loading values (cutoff value=0.30).

We subsequently conducted CFA on group B to confirm the model obtained in EFA. CFA was performed using the maximum likelihood estimator method. Model fitness of CFA is commonly conducted using indexes, including the comparative fit index (CFI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR), with higher CFI (>0.900), lower RMSEA (<0.080), and lower SRMR (<0.080) values indicating a good fit [19-21]. In this study, we compared these indexes between the 2 models, namely, the model suggested by the EFA (2-factor model; described in detail below) and the 1-factor model.

Convergent validity was evaluated through hypothesis testing. On the basis of a previous finding that ATTARI-12 scores were positively linked with specific attitudes toward robots [10], we investigated this relationship using Pearson correlation coefficients. With reference to previous studies [10,22], attitudes toward robots were assessed using 3 items (eg, “Robots are necessary as they can do jobs that are too hard or too dangerous for people”), each of which was answered on a 4-point Likert scale (Multimedia Appendix 2). The scores were computed by averaging the responses to the 3 items (Cronbach α=0.78; mean 3.25, SD 0.55 in our dataset), with higher scores indicating a more positive attitude toward robots. The Pearson correlation coefficients are deemed meaningful if they are greater than 0.30 [23].

We used Cronbach α values to evaluate the internal consistency reliability of the scale. In this study, we computed the values using the responses of the entire sample (N=326; described in detail below). Cronbach α values above 0.70 are acceptable [24].

The possible influence of participant gender and year group on ATTARI-12 scores was explored using descriptive statistics and comparison analyses (independent 2-tailed t test or 1-way ANOVA). With regard to gender, as outlined later, the “Others” category comprised only 4 participants, rendering it inadequate for inclusion in a 1-way ANOVA. Given the high likelihood of unstable estimates and heterogeneity of variance associated with such a small group, we compared scores between only the 2 larger groups (man and woman) via independent t test. The undergraduate medical curriculum in Japan has traditionally consisted of 2 phases: the first 4 years of preclinical education and the subsequent 2 years of clinical education (ie, clinical clerkship) [25]. Accordingly, medical trainees were divided into preclinical medical students (ie, first- to fourth-year students), clinical medical students (ie, fifth- and sixth-year students), and medical residents. All statistical analyses were performed using R (version 4.5.1; R Foundation for Statistical Computing) and SPSS (version 30.0; IBM Corp), with 2-sided P values of <.05 considered statistically significant.

Ethical Considerations

This study was conducted according to the ethical standards and principles of the Declaration of Helsinki and was approved by the ethics committee of Juntendo University Faculty of Medicine (E25-0028). All participants checked the consent box at the beginning of the questionnaire to indicate their informed consent to take part in the study. To ensure confidentiality, all participant data were anonymized before analysis. Participants were entered into a draw for 1 of 10 ¥5000 (approximately US $30) gift cards.

Results

Overview

In total, 9.2% (326/3551) of the eligible participants responded to the survey. There were no missing data. Table 1 shows the participants’ characteristics, and Table 2 shows the responses to each item.

Structural Validity Analysis

We performed EFA on group A using maximum likelihood estimation with Promax rotation (154/326, 47.2% of the participants). The KMO value was 0.83, and the Bartlett test was significant (P<.001). Table 3 shows the results of the EFA. EFA suggested a 2-factor solution. After discussion among the team members, we named the factors as follows: factor 1 was AI anxiety and aversion, and factor 2 was AI optimism and acceptance.

We then performed CFA on group B (the remaining 172/326, 52.8% of the participants) using the maximum likelihood estimator method. The 2-factor model suggested by the EFA yielded better goodness-of-fit results (CFI=0.914; RMSEA=0.075; SRMR=0.056) than the 1-factor model (CFI=0.804; RMSEA=0.113; SRMR=0.078). Multimedia Appendix 3 shows the CIs for the RMSEA. Figure 1 shows the path diagram. Therefore, we adopted the 2-factor model.

‎

Convergent Validity Analysis

We computed the Pearson correlation coefficient between the ATTARI-12 scores and the attitude toward robots scores. The coefficient value was 0.52 (P<.001), which indicated a positive correlation between these 2 scores.

Internal Consistency Reliability Analysis and Descriptive Statistics

Table 4 shows the descriptive statistics with the Cronbach α values. The Cronbach α for all 12 items was 0.84. The Cronbach α values for factors 1 and 2 were 0.80 and 0.76, respectively. These values were above the cutoff (0.70).

Table 5 shows descriptive data and comparison by gender and year group. The comparison analyses did not show any significant differences in ATTARI-12 scores.

Discussion

Principal Findings

In this study, we translated the ATTARI-12, originally developed for the general population, into Japanese in accordance with an international guideline [15] and then validated its structural and convergent validity and internal consistency reliability for medical trainees. Applying this scale in the context of medical trainees in Japan has the potential to stimulate future research and educational interventions. Such initiatives would serve to effectively assess and enhance the integration of AI in clinical practice and the medical education system.

Our study found that the internal consistency reliability of the J-ATTARI-12 was good. The findings were consistent with those of the original ATTARI-12 developmental study [10]. The original study reported Cronbach α values of 0.93 for a US sample and 0.90 for a German sample [10], which suggested that the ATTARI-12 is likely a helpful measure with good internal consistency reliability across countries.

Factor analysis indicated that the J-ATTARI-12 had a 2-factor structure, in contrast to the original English-language version’s unidimensional structure. The original ATTARI-12 was conceptualized as a unidimensional scale. The original article by Stein et al [10] suggested that the developers intentionally balanced positively and negatively worded items. This design makes it unsurprising that the Japanese version yielded 2 factors corresponding to negative (factor 1: AI anxiety and aversion) and positive (factor 2: AI optimism and acceptance) item valence. Therefore, we should acknowledge that, although 2 factors emerged, they simply reflect positive versus negative wording and that, for the purpose of international comparison, using a total score based on a 1-factor assumption may remain preferable. At the same time, the content of the 2 factors (ie, AI anxiety and aversion vs AI optimism and acceptance) may reflect attitudinal domains that are influenced by broader cultural characteristics, such as uncertainty avoidance, technophilia, or collectivism. To determine whether a 2-factor structure offers conceptual or psychometric advantages beyond a unidimensional model and whether this pattern is observed across different cultural contexts, future cross-cultural validation studies with larger and more diverse samples are required.

We note a couple of potential limitations of our study. First, the response rate was relatively low. In addition, the survey was conducted using convenience sampling and included only 14 institutions, and it is likely that participating institutions or individuals had a greater interest in AI. It is increasingly challenging to obtain high response rates to online surveys, and rates frequently drop to 10% [26,27]. Nevertheless, the literature suggests that the response rate to our survey may have been sufficient to provide reliable data [28,29]. Additionally, it should be noted that, despite the absence of accurate statistics on the demographic variables of medical trainees across Japan, recent reports have indicated that the proportion of female physicians aged ≤29 years stands at approximately 40%, with a steady upward trend observed in recent years [30]. Thus, the finding that 43.6% (142/326) of the respondents were women does not generate concerns regarding the representativeness of the sample. Second, in this study, although we evaluated the structural validity, convergent validity, and internal consistency reliability of the scale, other determinants of validity (eg, discriminant validity and predictive validity) and reliability (eg, test-retest reliability) have yet to be examined. Future studies should examine these psychometric properties. Third, the use of a gift card lottery can raise concerns about bias. Nevertheless, this method is commonly used and appears to be acceptable [31-34]. Fourth, our results may indicate a ceiling effect. For example, 41.4% (135/326) of the respondents selected the highest score (5) for item 5. However, according to the original ATTARI-12 paper, responses tend to be skewed toward a score of 4, suggesting that the scale inherently elicits generally positive attitudes toward AI. Therefore, these findings do not represent an unusual deviation but are consistent with the response patterns reported in the original study.

Despite these limitations, this study produced the first Japanese version of the ATTARI-12, which is likely to be used as a novel measure for assessing AI attitudes among medical students and residents. Two outcomes are expected. First, by incorporating the J-ATTARI-12 into medical curricula, medical educators will be able to develop a more comprehensive understanding of the trainees’ readiness for AI adoption. This will likely be advantageous in the design of customized educational interventions using AI based on the level of each trainee’s attitude toward AI [35] and will support curriculum development by enabling medical educators to identify learners who may need more foundational AI exposure or targeted support. Second, the J-ATTARI-12 will facilitate medical education research in Japan. The scale allows researchers to examine the impact of specific educational interventions on attitude change concerning AI, and it is suitable for repeated administration in longitudinal studies to track how attitudes evolve over the course of training. In addition, the availability of the J-ATTARI-12 will promote cross-cultural comparison of attitudes toward AI and its possible outcomes internationally. For these reasons, we expect that the J-ATTARI-12 will play a pivotal role in facilitating data-informed curriculum development and contribute to the expanding body of medical education research in this era of AI.

Conclusions

In this study, we developed the J-ATTARI-12, originally developed for the general population, in accordance with an international guideline. A validation survey revealed that the structural and convergent validity, as well as the internal consistency reliability, were good for medical trainees in Japan. The developed measure can be used for customized educational initiatives using AI based on the level of each trainee’s attitude toward AI. It will also provide helpful information to medical education researchers in Japan in this era of AI.