Potential study participants were recruited by convenience sampling through the authors’ contacts in Jordan. The survey link was sent through WhatsApp and Facebook groups targeted to students in health schools in the Arab-speaking country. The survey was open from February 28, 2023, and was closed on March 31, 2023. Participation was voluntary and did not involve incentives. The inclusion criteria that were outlined explicitly in the introductory section of the questionnaire before the informed consent item included (1) being 18 years of age or older, (2) being concurrently enrolled in a Jordanian university, and (3) having a very good comprehension of the Arabic language. The exclusion criteria included (1) being younger than 18 years of age, (2) studying in non–health care-related disciplines, (3) having a poor comprehension of the Arabic language.

The minimum sample size was estimated to be 360 participants following the established guidelines for survey validation studies, considering 36 items with 10 participants per item [42-44].

This study was approved by the institutional review board of the School of Pharmacy at the Applied Science Private University (2023-PHA-3), and approval was granted on January 24, 2023. Participation was voluntary and anonymous.

The survey instrument development process involved an extensive literature review and expert validation, followed by item development and pilot testing to ensure clarity [25,26,45-49]. Following an internal discussion among the authors with previous experience in survey construction and validation (MS, MB, DM, and SH), the survey tool was created based on the TAM framework. This internal discussion led to the identification of potential domains for inclusion in the final questionnaire: perceived usefulness, ease of use, risks, anxiety, attitude toward the technology, social influence, and cognitive and behavioral factors [25,26].

Herein, we refer to this edited TAM model in the context of ChatGPT adoption as the TAME-ChatGPT (Technology Acceptance Model Edited to Assess ChatGPT Adoption) survey instrument. Face and content validity were assessed by subjective evaluation, with an assessment of the clarity, comprehensiveness, and relevance of the initial items that were adopted. Additionally, any potential biases or issues with the wording of the items (eg, vague wording or complex items) were assessed [50].

Then, forward and backward translations were conducted by 3 authors (MS, NAS, and MB). Afterward, the survey was distributed among 6 participants representing a pilot test, followed by minor language modifications to improve clarity. The construct validity was checked following survey distribution using 13 TAM-based items evaluated among the respondents who heard of ChatGPT before the study. An additional 23 TAM-based items were evaluated among the respondents who used ChatGPT before the study.

The survey was introduced with a full explanation of the aims and a mandatory electronic consent item for the successful completion of the survey. The introductory section explicitly explained the guaranteed participant anonymity and privacy by refraining to request any personal details such as names or emails. This was followed by items to assess age, sex, university (public vs private), nationality (Jordanian vs non-Jordanian), school (health vs scientific vs humanities), and current educational level (undergraduate vs postgraduate). Then, a single item followed (“Have you heard of ChatGPT before the study?”) with a “yes” response required to move into the next item, while the answer of “no” resulted in survey submission. The next item was “Have you used ChatGPT before the study?” with “yes” resulting in the presentation of the full 36 items. An answer of “no” resulted in the presentation of the first 13 TAM items. The complete phrasing of the included items is presented in Table S1 of Multimedia Appendix 1.

Each item was evaluated on a 5-point Likert scale with the following responses: strongly agree scored as 5, agree scored as 4, neutral/no opinion scored as 3, disagree scored as 2, and strongly disagree scored as 1. The scoring was reversed for the items implying a negative attitude toward ChatGPT.

The statistical analysis was performed using SPSS software (V22.0; IBM Corp). To explore the factor structure of the TAME-ChatGPT construct comprising a total of 36 items, we conducted an exploratory factor analysis (EFA) using principal component analysis (PCA) as the extraction method and oblimin rotation to determine the correlations between factors. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and the Bartlett test of sphericity were used to assess the suitability of the data for EFA. The internal consistency of the subscales and the TAME-ChatGPT was checked using Cronbach α. The level of statistical significance was set at P<.05.

Descriptive statistics included the measures of distribution (mean and median), dispersion (SD), and IQR. For the scale variables, and considering the relatively small sample size, the Shapiro-Wilk test was used to assess the normality of the scale variables.

The associations between categorical variables were assessed using the chi-square (χ²) test, while the associations between categorical and scale variables were assessed by the Mann-Whitney (M-W) U test for nonnormally distributed scale variables. The level of statistical significance was P=.05.

A total of 480 responses were received over a 1-month period. A total of 9 individuals declined to participate in the study. Moreover, 5 respondents attending humanities schools and 8 science students were excluded. Thus, the final study sample comprised a total of 458 participants.

The study sample had a mean age of 21 (SD 3.3) years and a median age of 20 (IQR 19-22) years. Characteristics of the study sample are shown in Table 1. Out of the 458 participants, only 109 (23.8%) had heard of ChatGPT prior to the study, and only 55 (11.3%) self-reported ChatGPT use before the study.

In the whole study sample, older age, male sex, and postgraduate education were associated with a higher probability of hearing about ChatGPT before the study (Table 2). On the other hand, the differences lacked statistical significance upon comparing the different categories in the tested variables with the probability of ChatGPT usage before the study (Table 2).

The EFA was conducted on a set of 13 items to identify underlying factors that accounted for the variance in the responses. The sample comprised the participants who heard of ChatGPT before the study (n=109, 23.8%). The Bartlett test of sphericity was significant (χ²₇₈=779.2) P<.001), indicating the factorability of the correlation matrix. The KMO measure of the sampling adequacy was 0.823, indicating that the data were suitable for factor analysis.

The EFA was performed using PCA and oblimin rotation to account for potential correlations between factors. The scree plot showed that the optimal number of factors was 3, which explained a cumulative total of 69.3% of the variance in the data (Figure S1 of Multimedia Appendix 1). The eigenvalues for the 3 factors were 4.695, 3.148, and 1.168, respectively. All 13 items loaded significantly on 1 of the 3 factors, with factor loadings ranging from 0.65 to 0.87 (Table 3).

Based on the original TAM constructs, factor 1 was labeled “perceived risk” and included 5 items. Factor 2 was labeled “technology/social influence” and included 5 items related to attitude toward technology and social influence. Factor 3 was labeled “anxiety” and included 3 items related to anxiety and fear from ChatGPT.

The 3 factors demonstrated good internal consistency, with Cronbach α values of .876, .858, and .827, respectively, indicating that they could be used to measure these constructs in future research.

The EFA was conducted on a set of 14 items to identify underlying factors that accounted for the variance in the responses. The sample comprised the participants who used ChatGPT before the study (n=55, 11.3%). The Bartlett test of sphericity was significant (χ²₉₁=427.1; P<.001), indicating the factorability of the correlation matrix. The KMO measure of sampling adequacy was 0.702, indicating that the data were suitable for factor analysis.

Similar to the approach used for the attitude scale, the EFA was performed using PCA and oblimin rotation. The scree plot indicated that the optimal number of factors was 4, which explained a cumulative total of 72% of the variance in the data (Figure S2 of Multimedia Appendix 1). The eigenvalues for the 4 factors were 5.296, 1.979, 1.577, and 1.269, respectively. All 14 items loaded significantly on 1 of the 4 factors, with factor loadings ranging from 0.59 to 0.94 (Table 4).

Factor 1 was labeled “perceived usefulness” and included 6 items related to perceived usefulness. Factor 2 was labeled “perceived risk” and included 3 items related to perceived risk. Factor 3 was labeled “perceived ease of use” and included 2 items related to ease of use. Factor 4 was labeled “behavior” and included 3 items related to cognitive and behavioral aspects of ChatGPT use.

The 4 factors demonstrated good internal consistency (Cronbach α values of .885, .718, .824, and .781, respectively) and could be used to measure these constructs in future research.

The 3 TAME-ChatGPT attitude subscales were evaluated at first. The possible range of the perceived risks subscale was between 5 and 25, with higher values indicating low perceived ChatGPT risks due to reverse coding of these items and a score of 15 indicating a neutral attitude toward ChatGPT.

Among the participants who have heard of ChatGPT before the study (n=109, 23.9%), the mean perceived risks score was 12.5 (SD 4.8), indicating a general agreement with the items assessing the perceived ChatGPT risks. Higher perceived risks were seen among females (P=.036, M-W; Table 5). No statistically significant differences were seen based on age, nationality, university, or self-reported ChatGPT use (Table 5).

For the technology/social influence subscale, the possible range was 5 to 25, with higher values indicating a positive attitude toward technology exemplified by ChatGPT and a score of 15 indicating a neutral attitude. The mean attitude toward technology score was 19.3 (SD 4.1), indicating a positive attitude toward ChatGPT technology. Higher technology subscale scores were seen among the participants who used ChatGPT before the study (mean 21, SD 3.6 vs mean 17.6, SD 3.9 among those who have not used it before the study; P<.001, M-W), and among males (mean 20.1, SD 4 vs mean 18.3, SD 4.2 among females; P=.023, M-W). No statistically significant differences were seen based on age, nationality, university, and educational level (Table 5).

For the anxiety subscale, the possible range was 3 to 15, with higher values indicating lower anxiety toward ChatGPT due to the reverse coding of these items and a score of 9 indicating a neutral attitude. The mean anxiety score was 6.6 (SD 2.9), indicating an anxious attitude regarding ChatGPT in the study sample. No statistically significant differences were seen based on age, sex, nationality, university, educational level, and self-reported ChatGPT use (Table 5).

The 4 TAME-ChatGPT usage subscales were evaluated. The possible range of the perceived usefulness subscale was 6 to 30, with higher values indicating a higher perceived usefulness of ChatGPT and a score of 18 indicating a neutral attitude.

The mean perceived usefulness score was 24.2 (SD 4.9), indicating high perceived usefulness of ChatGPT among the participants who used it before the study. No statistically significant differences were seen based on age, sex, nationality, university, and educational level (Table 6).

For the perceived risk subscale, the possible range was 3 to 15, with higher values indicating lower perceived risks from ChatGPT use due to reverse coding of these items and a score of 9 indicating a neutral attitude. The mean perceived risk score was 7.2 (SD 2.8), indicating a slightly high perceived risk from ChatGPT use. No statistically significant differences were seen based on age, sex, nationality, university, and educational level (Table 6).

For the perceived ease of use subscale, the possible range was 2 to 10, indicating higher perceived ease of ChatGPT use, and a score of 6 indicated a neutral attitude. The mean perceived ease of use was 8.9 (SD 1.6), indicating the high perceived easiness of ChatGPT use in the study sample. No statistically significant differences were seen based on age, sex, nationality, university, and educational level (Table 6).

For the behavior subscale, the possible range was 3 to 15, with higher values indicating a positive behavior toward ChatGPT use due to reverse coding of these items and a score of 9 indicating a neutral attitude. The mean behavior was 9.8 (SD 3.3), indicating a slightly positive behavior toward ChatGPT leaning toward a neutral attitude. No statistically significant differences were seen based on age, sex, nationality, university, and educational level (Table 6).

The main finding of this study demonstrated the reliability and validity of TAME-ChatGPT as a possible valuable tool for assessing health care students’ attitudes toward ChatGPT. The findings emphasized the need to account for risk perceptions, usefulness, ease of use, attitudes toward technology, and behavioral factors to successfully implement ChatGPT in health care education. These insights can guide AI developers, academics, and policy makers to formulate suitable strategies to ensure the ethical and optimal deployment of ChatGPT while addressing potential implementation challenges.

The availability of ChatGPT as an example of LLMs carries transformative societal implications, especially in health care settings, making its adoption in health care education seemingly inevitable [9,11,51-54]. Students will increasingly explore this innovative AI-based technology, with an already growing literature highlighting its significance in health care education through personalized learning with immediate feedback and impressive performance in medical exams [9,18,40,55-60]. Additionally, a recent study indicated a growing tendency among the general public to employ ChatGPT for self-diagnosis [61]. Therefore, the initial step toward the effective integration of ChatGPT in health care education involves evaluating attitudes toward this novel technology as well as the factors influencing its acceptance and usage.

However, before achieving this relevant aim, it is imperative to use a survey instrument that is validated to reach reliable conclusions based on the tested variables. Thus, this study represents one of the initial efforts to construct and validate a survey instrument assessing the attitudes toward ChatGPT among health care students in Jordan.

In this study, the major domains that were inferred through EFA included the perceived risks associated with ChatGPT, the attitude toward technology/social influence, and the anxiety that ChatGPT creates for the participants who have heard of ChatGPT. For the participants who used ChatGPT, EFA showed that 4 TAM-based domains were crucial factors driving ChatGPT use, which included the perceived usefulness, perceived risks, perceived ease of use, and behavior driving the use of technology.

The emergence of perceived risks as a major construct driving the attitude toward ChatGPT and its use is understandable. This is related to the potential for LLMs exemplified by ChatGPT to generate biased, inaccurate, or harmful content [9]. ChatGPT, among other LLMs, depends on huge training data sets; nevertheless, there is a general lack of transparency regarding the origin of these data [9,37]. Subsequently, there is a possibility that LLMs could learn and reproduce biased and incorrect content, which can have severe consequences in health care settings [9,36,37,62-64].

Risk perception plays a crucial role in decision-making, including the adoption of novel technologies like ChatGPT [65-68]. Recent studies highlighted the potential risks associated with ChatGPT risks including performance and privacy concerns [9,41]. Consequently, the participating students’ knowledge, beliefs, and prior experience with similar technologies significantly influenced their risk perception of ChatGPT. Unintended negative consequences, such as inappropriate or inaccurate content, pose significant risks in health care settings, necessitating careful consideration before its adoption in health care education [9,69-71].

This study demonstrated that risk perception significantly influenced health care students’ attitudes and usage of ChatGPT. This emphasizes the need for developers to address potential biases in ChatGPT, in addition to the need to address possible technological flaws to prevent cybersecurity threats and data breaches. Policy makers and AI-chatbot developers should prioritize transparent risk management strategies to promote responsible ChatGPT adoption in health care education [9,18,72]. Suggested measures to address ChatGPT’s perceived risks include student education on ChatGPT’s limitations and risks, establishing ethical guidelines for its responsible use, considering ethical and legal aspects, and promoting the development of high-quality training data [9,41].

The second construct driving the attitude toward ChatGPT found in this study was the attitude toward technology, alongside social influence. This construct refers to the perception and readiness to embrace technological innovations. Consistent with the previous evidence, positive attitudes facilitate the adoption of new technology adoption [73,74]. Thus, to promote a wider adoption of educational chatbots, providing training and education on the technology, highlighting its benefits, and ensuring accurate outputs are crucial [75,76].

Social influence can significantly impact attitudes toward ChatGPT adoption, including the opinions of the social circle and peers [77,78]. Additionally, media, public figures, and technology leaders play a role in shaping positive attitudes toward such applications. For example, the public opinions of prominent figures in the technology and business sectors can influence the widespread adoption and use of ChatGPT [79,80].

The third construct found in this validation study was the anxiety ChatGPT might provoke. The global availability of ChatGPT can be a transformative paradigm shift akin to the introduction of the internet and mobile phones, inducing fear, uncertainty, or discomfort [79,81,82]. Therefore, the elicited anxiety from such novel technology should be regarded as a significant factor driving its adoption [83,84].

In the second part of the TAM-based survey assessing ChatGPT usage determinants, the results showed that the perceived usefulness and ease of use as important factors influencing ChatGPT use among health care students. These psychological factors have been identified previously to play a critical role in shaping attitudes toward the adoption of new technologies [74,85-87]. Additionally, the perceived usefulness and effectiveness of technologies in achieving their intended goals could significantly influence the overall attitude of users, since an efficient and user-friendly technology encourages a more positive attitude toward its adoption [87-89]. Consequently, the impact of perceived usefulness and ease of use on students’ attitudes toward ChatGPT appear crucial for predicting and encouraging its successful adoption. In this exploratory study, we observed a high level of ease of use among the small group of participants who reported using ChatGPT, likely due to its user-friendly nature and free accessibility [17,71,90].

In this study, following the TAM model, the behavioral and cognitive factors emerged as key drivers of ChatGPT usage among health care students. ChatGPT can provide quick and easy access to information and services, reducing the need for human interaction, which is advantageous for busy health care students dealing with massive information and packed learning schedules [18,91]. Therefore, the ease of access provided by ChatGPT compared to traditional methods of education is a significant advantage [9,18,91,92]. Additionally, educational chatbots offer the potential to enhance self-confidence and communication skills, particularly for students facing challenges in social communication, highlighting its value as a conversational interface that simulates human interactions and fosters a sense of companionship among students [93,94].

On the other hand, one of the negative driving factors for ChatGPT use is the potential for dependence or even addiction [95]. This problem is of particular concern for individuals who may be susceptible to compulsive behavior [96]. This addiction can lead to decreased productivity, social withdrawal, and other negative consequences severely affecting the students’ later interactions with patients. The use of ChatGPT can also be associated with a deterioration in empathy and social skills [9]. The reliance on ChatGPT may result in hindering the development of the skills needed to interpret and respond to social cues, which should be considered in health care education [9,91].

The limited sample size used in this study is a major limitation; however, the complexity of the scale required the participants to spend considerable time and effort, which can limit the number of participants that are willing to complete the survey due to respondent fatigue [97]. Selection bias should also be considered based on the adoption of convenience-based sampling, and this issue should be addressed in future studies aiming to confirm the findings of this study and evaluate the attitudes of health care students toward ChatGPT and its use. The female predominance might be due to selection bias, but it aligns with the fact that dentistry, pharmacy, and nursing fields in Jordan have a majority of female students, as anticipated. Importantly, despite the utilization of the TAM framework, a significant limitation of this study is the potential bias in the tested constructs, which should be considered in future validation studies.

Following the initial validation of TAME-ChatGPT as a tool to assess the attitude and usage of ChatGPT among health care students as indicated by the results of this study, a follow-up multinational project will ensue to conduct a confirmatory factor analysis and determine the major determinants of the attitude toward ChatGPT. This can help to guide the efforts needed for the successful adoption of ChatGPT in health care education.

In this study, we showed that the validated TAME-ChatGPT scales have good reliability and validity with usefulness to test the following domains covered by 13 items to determine the attitude toward ChatGPT: perceived risks from ChatGPT, the attitude toward technology/social influence, and the anxiety that ChatGPT creates. Additionally, 4 constructs can be helpful to determine the factors driving ChatGPT use comprising 14 items: usefulness, perceived risks, perceived ease of use, and behavior driving the use of ChatGPT. Future studies are recommended to guide the successful adoption of ChatGPT in health care education.

Overall, the results of this study highlighted the importance of considering perceptions of risks, usefulness, ease of use, and attitudes toward technology as well as the behavioral factors upon adopting new technologies for health care education exemplified by ChatGPT. This can help AI developers, academics, and policy makers devise strategies to promote the effective and ethical use of ChatGPT and identify barriers to the adoption of this breakthrough revolutionary technology. By analyzing the acceptance and use of ChatGPT through a reliable and valid construct, evidence-based insights can inform decisions on the incorporation of this technology in health care education.