The recent COVID-19 pandemic has highlighted the importance and potential of digital health in optimizing the quality, efficiency, and safety of health care [1-4]. Despite this, the adoption of digital tools and technologies in this field has been slow [5,6], and their full implementation in clinical practice has yet to occur [7]. Research has pointed to several factors as potential barriers, including technology, infrastructure, and financial resources [8,9]. However, it is the lack of digital health literacy that most commonly obstructs the implementation of digital health services [6]. Health professionals have been identified as a key factor in the digital transformation of health care [10]. Accordingly, they must be equipped with digital health competences, ranging from basic skills (eg, computer literacy) to more complex ones such as the ability to teach patients how to use technology and digital data sources safely and appropriately. Beyond informing patients about the availability and potential benefits of these technologies, physicians guide them on integrating these tools into their health care routines, playing a pivotal role in this process. For instance, they can guide patients on using portals for test results or mobile apps for medication adherence. However, for in-depth training on specific technologies, other professionals such as nurses or technical support staff may be better suited [2,6,9,11].

In 2016, a survey of 200 health professionals conducted by the European Health Parliament’s Digital Skills for Health Professionals (COMPDIG-Salut) Committee found that, in most cases, health professionals felt that they lacked the appropriate skills to cope with the digital revolution in their professional practice [12,13]. Today, there is still a need for accessible, structured, and comprehensive education that will enable future health professionals to make the best use of technology and harness its full potential in terms of quality of care [5,14,15].

Health professionals need to develop digital health competences to keep up with new technologies and ensure that they can provide high-quality patient care [2,5,7,16,17]. To this end, we must first map the specific digital competences needed in health care (to provide the right kind of digital education) [18] and then create a model for assessing and accrediting such competences. While there is a growing number of individual digital health competence frameworks and reviews that focus on specific health care professions or settings [2], there is a lack of standardization in the definition of digital health competences themselves, including discrepancies and overlap among available frameworks and their approach to categorization [7]. This implies a need to continually update the competences required in this field as well as the methods used to assess them [7].

The Professional Dialogue Forum of Catalonia (northeastern Spain), one of the most advanced regions in Europe in the use of digital health technologies [19-21], highlighted “the need to improve information and communication technology (ICT) competences to advance in the use of ICT and the design of telehealth services” as one of the 17 primary current and future challenges facing health professionals [22]. The COMPDIG-Salut project was launched in response to the identified needs, focusing on three aims: (1) defining a specific digital competence framework for health professionals, (2) creating a specific assessment and accreditation model for health professionals, and (3) designing actions to train and qualify health professionals in digital competences. Having determined the current digital competence level among Catalan health professionals [23], it is time to work toward achieving the aims of the COMPDIG-Salut project [12,24].

This study had two objectives: (1) to create a specific map of digital competences for health professionals and (2) to define and test a digital competence assessment and accreditation model for health professionals.

The research presented in this paper is the result of collaboration between the TIC Salut Social Foundation (Information and Communication Technologies in Health and Social Care Foundation) and the Universitat Oberta de Catalunya in Spain. As this was an observational exploratory study focusing on the analysis of digital competences in health care, a mixed qualitative and quantitative methodology was used following an iterative approach, and questionnaires were designed for data collection.

When designing the study, we calculated the minimum sample size to ensure significant results with a 10% error rate, the maximum allowed in research studies [37]. On the basis of the results of an exploratory study [23] and the latest available report on the population of health professionals in Catalonia [22], the minimum sample size for the study was set at 168.

Descriptive statistics were performed for professionals who had correctly completed activities 1 and 2, with results presented as absolute and relative frequencies. Arithmetic means and SDs were used for comparative analysis of subgroups according to sociodemographic and professional characteristics, and P values were used for hypothesis testing.

The reliability of activity 2 was analyzed by measuring the consistency of its items. In addition, the level of discrimination of the items in relation to the advanced level was evaluated. The arithmetic mean and SD were used in descriptive statistics; P values, Bonferroni-adjusted P values, and Holm-adjusted P values were used in hypothesis testing and subgroup comparisons; and the greatest lower bound (GLB) was chosen to diagnose the test given the lack of homogeneity of the scoring scale, as was done in the exploratory study [23].

The study instrument was analyzed using the participants’ scores and the discrimination index (DI) [38]. The DI measures how well an item could discriminate between high-scoring participants (ie, those with strong digital competences) and low-scoring participants in activity 2. DI values between 0 and 0.2 indicate that the item is not discriminating, and negative values imply an inverse relationship between the score on that item and the total score.

Finally, for activity 3, the numerical variables were presented as arithmetic means and SDs, whereas the categorical variables were presented as absolute and relative frequencies.

All responses were analyzed using the R statistical software (version 4.2.0; R Foundation for Statistical Computing). Responses to the open-ended questions (questions 5 and 7) in the feedback questionnaire (activity 3) were analyzed using quantitative content analysis to group them into limited categories [39].

No ethics approval was required due to the type and nature of the study as the Catalan Department of Health is responsible for formulating the general criteria for health planning, setting the objectives, and the levels to be achieved in the topics that are included in the Health Plan for Catalonia [40]. All participants were informed about the study’s purposes and that their participation was voluntary. Data protection treatment was informed to the participant and before accessing the survey, participants had to provide acceptance.

Multimedia Appendix 7 shows the specific map of digital competences for health professionals validated by the 21 individuals (of different profiles) in the working group of challenge 4 of the Professional Dialogue Forum.

Digital health offers a valuable opportunity to make strides toward attaining the United Nations’ Sustainable Development Goal of universal health coverage by 2030 [42]. To that end, and given the gradual digitization of the health care sector, health professionals must have the appropriate competences regardless of their specific disciplines [2,5,16,17,43].

Furthermore, the constant evolution of new technologies requires the continuous updating of digital health competence. This entails reviewing both the relevant competences and the methods for properly assessing them [2]. Some discrepancies and overlap still exist among available frameworks, the methods used to conceptualize such frameworks, and the competences they include [2] and also among the health professionals at whom they are aimed [44,45]. The lack of a comprehensive framework applicable to all health professionals warrants the development of frameworks that include different health professionals.

Among the most recent frameworks is the Health Information Technology Competencies. Developed using an iterative method, it is the most complete framework aimed at a broad range of health professionals and medical specialties [30]. In Spain, and more specifically in the Basque Country (northern Spain), the Ikanos project was developed (2020) based on the European DigComp framework as a benchmark for the description of digital competences. Drawing on interviews with experts, the Ikanos framework focuses on profiles based initially on primary care professionals (medical and nursing staff) [29].

The COMPDIG-Salut project was launched for the purposes of designing a specific map of digital competences for health professionals and creating a specific digital competence assessment and accreditation model to enable health professionals to obtain a specific accreditation certificate. Taking an iterative approach, which included a review of the gray literature and consultation with local experts [2], we designed a map of digital competences, consisting of 10 competences under 4 competence areas: data access, management, and analysis; communication and collaboration; digital awareness; and professional development. The second and fourth of these areas have the highest number of competences (3 each). In short, the map includes competences related to essential skills for the digital management and analysis of health and social data and information and includes emerging technological innovation (ie, health apps, artificial intelligence, and autonomous decision support systems) [2,46]. Moreover, and given that digital health entails new methods of communication (teleconsultations and email), it is imperative for health professionals to know how to convey information in a precise, effective, and timely manner to patients, health professionals, and any collaborating parties [47-49]. Effective communication, collaboration, and teamwork are crucial in a health setting, so health professionals must also possess the ability to work effectively as members of interdisciplinary teams [2,50]. The creation and publication of health-related digital content is another basic competence that health professionals should have to ensure high-quality health care provision and help improve the communities they serve [31,51]. Finally, emphasis is placed on the importance of health professionals complying with the ethical principles and security criteria associated with the appropriate use of digital technologies [52] and also keeping abreast of the regulations on health and social data and information privacy, confidentiality, and protection [2,53,54]. However, we should bear in mind that the evolving nature of digital health and professional practice in the health field—which often outpaces research—requires constant, flexible development of professional competences [55]. Hence, some of the competences in the competence framework defined in the COMPDIG-Salut project may need to be adjusted to incorporate future digital trends, such as addressing challenges in the areas of cybersecurity and the use of artificial intelligence and robotics in professional practice. This implies that the methods for accurately assessing such competences may need to be updated.

Meanwhile, given the need to assess health professionals’ digital competence levels for the purposes of narrowing the digital gap and, by so doing, maintaining health service quality [11,45], a digital competence assessment model for health professionals was developed as part of the COMPDIG-Salut project. Drawing on the competence content defined in the project, the model was designed in accordance with ACTIC’s new scenario-based assessment model [23]. In this sense, the assessment model was adapted to the 4 distinct health sector profiles: professionals providing direct patient care; professionals providing indirect patient care; professionals providing innovation, research, or teaching services; and professionals who manage. This not only reflects the impact of the professional role on digital competence [56] but also, with the differentiation of the 4 profiles, facilitates the design of a specific training pathway for each role (Multimedia Appendix 14).

While the focus was on the type of health professional profile (P1, P2, P3, or P4), the most frequent professional categories into which participants fell were nurses, pharmacists, and physiotherapists. The internal consistency of the digital competence assessment model for health professionals in P1 was excellent overall, bearing in mind that it was a proposed assessment model (GLB=0.91). We found that 54% (15/28) of the questions discriminated between health professionals with an advanced level of digital competence and those with a lower level (DI≥0.20) and that 11% (3/28) of the questions were highly discriminating (DI>0.4). However, 7% (2/28) of the items had a negative DI, so these would need to be reviewed because a negative DI implies an inverse relationship between the level and the correct answer, meaning that the wording of the question may not have been sufficiently precise. However, as these values were close to 0, they could be regrouped with the nondifferentiating questions (DI=0-0.2). As the assessment model for the remaining profiles assessed the same competences as those in the P1 test, we expected the internal consistency and the degree of discrimination of the questions to be similar among the profiles. However, it was not possible to directly extrapolate the P1 results.

Most of the questions in the feedback questionnaire (activity 3) could not be robustly interpreted because they did not constitute a standard instrument that had been validated by another study. The values obtained in the Likert-type questions could not be robustly interpreted either because the values were not near the ends of the scale. Many health professionals felt that their roles and challenges were not adequately represented in the assessment model despite the professional profiles being well defined. This was especially the case among health professionals who made up the smallest groups (mostly nonmedical and nonnursing staff and those working in the private or pharmaceutical sectors). This result underscores the necessity of refining the assessment model to meet the diverse digital competence needs of health care professionals, emphasizing the potential requirement for more targeted measures tailored to specific groups, and acknowledging their potential cultural dependencies. Therefore, it would be necessary to either review the profile proposal for the competence assessment model or add scenarios and challenges to it for health professionals who were not reflected in the proposed situations. The sample size of some subgroups (physicians, health professionals with advanced digital competence, P2, P3, and P4) may have been too small to represent the population.

Although most study participants (110/122, 90.2%) reported an intermediate self-perceived digital competence level and only 9.8% (12/122) of participants considered themselves advanced, we found that the vast majority would not attain ACTIC 2 (intermediate) certification [34]. This situation was similar to the one found in an exploratory study conducted in 2021 [23]. Consistent with the low score for the ACTIC 2 certification (activity 1), the total mean score for activity 2, which assessed the defined competence framework, was significantly lower than the defined threshold. In fact, 73% (89/122) of the sample did not pass the tests. Participants with ACTIC 3 certification significantly passed activity 1 but not activity 2. In fact, none of the subgroups studied showed a significant increase in activity 2. This shows that the assessment model is a useful tool as an approach to assessing the competence framework for health professionals, and the results strengthen those obtained in the exploratory study [23]. This also shows that health professionals still need training, that such training goes beyond that required for the main tools currently used, and that merely being exposed to digital media as consumers is insufficient in terms of acquiring the necessary digital skills. More and more faculties of medicine are incorporating digital health training into their programs of study [57,58], and there is an ever-increasing number of initiatives to meet that need [2], such as the Digital Capability Framework developed by Health Education England [59] or the free introduction to eHealth web-based course offered by the EU*US eHealth Work project [60]. The designed map of digital competences could serve as the basis of the knowledge and skills for digital literacy specific to health professionals, to which other competences or areas could be added in accordance with the particularities of the various health professions and medical specialties. This map of digital competences could also inform and contribute to the development of future digital health training programs for health workers at different stages of their professional careers.

Our study provides a competence framework and a tool for assessing the digital competences of health professionals that is consistent with the certification available to citizens (ACTIC). It also yields results that strengthen those obtained in the previous exploratory study [23]. However, this study has several limitations. First, several weaknesses in the interpretation of findings from the review should be considered. While narrative reviews are often used in social science research for educational purposes, they may be biased and lack objectivity. That said, they can provide a unique perspective and identify knowledge gaps in the literature [61]. Although a considerable number of reference frameworks from gray literature were included, some may have been overlooked. Moreover, while performing thematic analysis, we found that some frameworks had either vague competence categories or overlaps among categories, which engendered differences of opinion during the classification process. However, the 4 reviewers drew on their experience to develop and clearly define the competence areas and assign the corresponding competences to them, first independently and then jointly through discussion-based agreement to reduce bias and classify the information in the most appropriate way possible. Although the reviewers tried to make the classification process as transparent and replicable as possible, it should be borne in mind that there could be other ways of interpreting and classifying the information and that the categorization might differ in the future with new advances in digital health. Second, while valid results were obtained for 122 participants, the minimum sample size set for the study was not reached (N=168). The platform used for the test (Moodle), the number of activities (3), and the time allotted (estimated at 1.5 hours) were almost certainly the reasons for poor participation in the study, especially given the high workload of the professional group in question. A study assessing both perceived and demonstrated eHealth literacy through a computer simulation of health-related internet tasks also revealed that evaluating demonstrated eHealth literacy via simulations is a challenging endeavor in terms of time [62]. When comparing the results of this study (using Moodle for the test) to those of the previous exploratory study (using Microsoft Forms for the test), it could be said that Moodle might have had an influence on a health professional’s decision not to take part in the study [23]. Third, the most highly represented health professionals in this study were those providing direct patient care (P1). Thus, many of the conclusions drawn from the study are limited to that profile. The poor participation of physicians (4/122, 3.3%) meant that they were underrepresented if compared to the 2017 professional population data for Catalonia [22]. As it fell to professional associations to inform their members about this study, the dissemination mechanisms they used are not precisely known. However, their impact was seemingly greater in the associations of nurses, pharmacists, and physiotherapists than in the associations of other health professions. Fourth, the absence of a detailed demographic analysis of the participant population limits our ability to provide a comprehensive understanding of sample characteristics. While demographic characterization could aid in contextualizing the results and their relationship to the simulation performance, such data were not collected in this study in accordance with the General Data Protection Regulation principle of “data minimization” [63]. Consequently, the lack of this analysis may restrict our ability to generalize findings to broader populations and fully comprehend the influence of demographic variables on study outcomes. Fifth, our findings enable us to estimate the prevalence of P1 as the most common profile; however, they do not provide a basis for drawing more specific conclusions. The 2022 report on health care professionals in Catalonia lacks profession categorization [22], and acquiring accurate and current data poses challenges attributable to factors such as professional mobility (eg, change of workplace, the coexistence of public and private sectors, and mobility between various institutions) and liberal professions. Subsequent research endeavors should prioritize the inclusion of all 4 profiles for a more comprehensive understanding. Sixth, the overrepresentation of health professionals with an intermediate self-perceived competence level compared to those considering themselves advanced also limited some of the study conclusions. We will probably need to review our definitions of the different self-perception levels because, perhaps due to the wording used, there was a greater perception of exigency than there really was (eg, advanced user: “I have attained the most advanced digital competences for transforming and innovating in today’s digital society.”). Moreover, the evaluation of digital health competences should take into account that, despite being notable, there is a moderate correlation between perceived and actual digital competences, as other studies looking at simulation scenarios in health care have shown [62,64]. Despite the considerable similarity between the latter simulation scenarios and those used in this study, previous research has focused on distinct demographic groups within the general population, such as individuals aged ≥50 years [62], or specific cohorts, such as patients with chronic illnesses [64], rather than health care professionals.

Assessing the digital competence level of health professionals based on a defined competence framework should enable such professionals to be trained and updated to meet real needs in their specific professional contexts and, consequently, take full advantage of the potential of digital technologies [65]. The information and data gathered, together with the results of an exploratory study on competence levels conducted in 2021 [23], should be taken as the starting point for promoting relevant strategic policies and actions to ensure that the right resources and conditions are in place for good professional performance [15,66-69]. Faced with the need to improve the digital competence of health professionals working in Catalonia, these results have informed the Health Plan for Catalonia 2021-2025 [40] and lay the foundations for designing and delivering specific training to first assess and then certify the digital competence of such professionals. Thus, the assessment model presented in this paper—designed in accordance with the competence content defined in the map of digital competences and based on scenarios—has the potential to be applied in diverse countries and languages with appropriate modifications to meet the specific needs and contexts of health care professionals. This might provide a preliminary perspective for assessing the competence levels and needs of health professionals in different health systems, although further evidence is needed to fully support this claim.