Training content was based on a validated laboratory measure of the strength of emotional connection in parent-child dyads [8,20] adapted for use in the clinical setting through an iterative process that began with a learning collaborative between Reach Out and Read and the Center for Early Relational Health at Columbia University, and also included pediatric providers (spanning pediatricians, family physicians, and nurse practitioners) and 2 parent leaders. The training module was hosted on Reach Out and Read’s Pathwright portal but was only available by special invitation. Training began with an experiential introduction to the “Lens of Emotional Connection” through four ~30-second videos of mother-infant dyads interacting with one another face-to-face, with varying levels of emotional connection across the videos. Participants were asked to use their “gut feeling” to rate the strength of emotional connection using visual analog scales from ‘not emotionally connected’ to “emotionally connected” (range 1‐100, but no numbers appearing for participants). Videos of these at-home interactions were recorded via Zoom as part of the longitudinal COMBO Initiative [21] and represent ethnically diverse mother-infant dyads at 4‐6 months postpartum. Participants were then provided with written and video educational content of the principles of emotional connection, followed by the same 4 plus 3 new mother-infant videos for practice rating the strength of emotional connection using the same visual analog scales. Participants were informed that the scale acts as an educational tool and that they are not expected to use such a scale in their clinical practice. Finally, participants were shown 3 videos of parent-infant well-child visits with Reach Out and Read pediatricians trained in the observation and promotion of emotional connection for a real-life demonstration of using the new “lens” in a clinical setting. The training module was estimated to take 45 minutes to complete.

To evaluate the training module, this project used a sequential mixed methods design [22] with a pre-post survey process and follow-up focus groups. Surveys permitted a breadth of understanding across a larger number of participants; focus groups provided explanatory information to further understand and explain survey data as well as to identify relevant factors that were not captured in surveys. This mixed methods study is reported in accordance with both GRAMMS (Good Reporting of a Mixed Methods Study) and STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) reporting guidelines to ensure proper reporting of both quantitative and qualitative aspects. The datasets generated and analyzed during this study (focus group interviews and survey data) are not publicly available, as the approved institutional review board (IRB) protocol did not allow for this. However, data can be made available by the corresponding author on reasonable request and pending appropriate IRB approval.

Pre-post surveys were developed across a series of recurring meetings held with an interdisciplinary team that included module developers, academic pediatricians, and social scientists with evaluation and measurement expertise. The pre- and postsurvey rationale was chosen to measure the effectiveness of the training in improving participants’ perceived confidence and knowledge in ERH. Additional postsurvey items assessed module experience. Survey development and review by the diverse members of the planning team, which included experts in ERH, pediatricians with experiential engagement with pediatrics training, and training intervention developers, focused on content and face validity to ensure relevance, clarity, and alignment with module objectives. Surveys are available in Multimedia Appendix 1.

The presurvey questions were administered before participants engaged with module training content and assessed age, self-identified race and ethnicity, and gender, training level, and questions about previous ERH and emotional connection experience, attitude, and knowledge. The postsurvey questions were administered after participants completed all module training content and included parallel items addressing attitude and knowledge related to ERH and emotional connection (repeated from the presurvey), as well as items addressing module experience and perspective on the implementation of emotional connection-related practices in pediatric primary care. Pre- and postsurveys were programmed and hosted on Columbia University’s Qualtrics and integrated directly into the Pathwright module experience. The combined length of the training module and pre-post surveys was ~45 minutes.

A total of 653 residents, advanced practice providers, and faculty at all 8 sites of the Carolinas Collaborative were recruited to participate in this study between June and October 2023 (Table 1). Inclusion criteria required participants to be either attending clinicians, resident clinicians, or faculty. Recruitment was the same across all sites, via email invitation with a link to the training. The module could be done on any computer and whenever they found or were allotted time. (Some clinical sites provided protected time for residents to complete the modules and surveys.) Participants were able to begin and finish at a later time as long as they used the same computer. It was made clear to participants that all aspects of this study were voluntary, and that participants could complete the training and choose to have their responses used in research or opt out of the use of their data for research purposes.

Survey data were not available for all participants who engaged with the module due to a combination of skipped survey items, some participants stopping before the postsurvey, and technical errors in the integration of data across Qualtrics and Pathwright. Sample sizes for available data for each survey item ranged between 44 and 75. Available data were analyzed using SAS software (version 9.4; SAS Institute, Inc.; copyright 2002‐2012) and used descriptive statistics and paired 2-tailed t tests to address change for individuals with both pre- and postdata. Additional ordered logit and ordinary least squares regression models examined the potential effect of varied participant demographic characteristics and prior related experience; however, minimal effect of background characteristics was detected, and these results are thus not reported here.

Individual Carolinas Collaborative sites coordinated with the social science research team at Duke University to schedule focus group opportunities with participants who completed the training between August and November 2023. Using a sequential mixed methods design [22], focus groups were constructed to provide additional context and explanation for survey-based quantitative results, in addition to providing added nuance more generally on experience with and effect of module engagement. Participants were recruited via email sent to the same distribution lists as during initial recruitment, and an incentive of a US $10 gift card was provided for participation in focus groups. As a result, 7 semistructured focus groups were conducted with 33 total participants across 4 Carolinas Collaborative sites. Focus groups were held virtually, lasted approximately 1 hour, and assessed initial knowledge and perspective on emotional connection, experience with the module, use, value of the module for their knowledge and practice, and opportunities for improvement; see Multimedia Appendix 2 for the focus group guide. Recruitment concluded after the interested participants identified by the end of the planned data collection period had been engaged. While the team had not definitively reached saturation across all possible points, results were revealing clear common themes regarding priority areas of experience and outcomes. Decisions against added waves of recruitment were based on project timeline considerations, likelihood of more limited participation with added recruitment efforts, and value in the existing scope data that had been collected (eg, clear common themes identifiable based on data already collected). Focus groups were recorded and transcribed. Data were analyzed using a rapid analysis framework [23], identifying key themes in responses to questions addressed. This used templated matrices structured based on primary areas of the focus group instrument (eg, aspects of module experience, distinct outcomes; see Multimedia Appendix 3 for matrix structure), review of transcripts and documentation of relevant content per item in each focus group, and cross-review among analysts (members of the social science team) to identify themes per topic. Qualitative and quantitative results were considered in tandem, with qualitative results providing an added explanatory lens on quantitative findings.

The survey portion of this study was approved by the AAP’s IRB (#22 ER 01), which served as the main approval body; the approval notice was shared with and reviewed by each constituent IRB within the Carolinas Collaborative (Atrium Health Levine Children’s Hospital, Atrium Health Wake Forest Baptist, Duke University, East Carolina University, Medical University of South Carolina, Prisma Health Midlands, Prisma Health Upstate, and the University of North Carolina), some of which did not require further review and some which considered the study exempt following site-specific application. The Columbia University IRB also separately approved the study (#AAAU5749). Participant consent was waived by both the AAP and Columbia University IRBs because the data collected were anonymous and could not be linked to any identifying information. In lieu of consent, all participants were provided with an informational sheet (electronically before the survey) that provided them with all the information about the study and offered them the opportunity to opt in or out of research participation. Participants could complete the training without participating in the research study. Participants did not receive compensation for their time completing the training (though, as noted above, a US $10 gift card was offered to the subset of participants who took part in the focus groups). The videos used in the training itself were collected in a previous study with participants who gave consent for their videos to be used for research and training purposes. The focus group portion of this study was approved by the Duke University IRB (#2023‐0300).

Of those invited, 207 (31.7%) clinicians participated in the module, and of these participants, 117 respondents completed at least 1 survey (56.5% response rate) with representation across 8 Carolinas Collaborative sites (Table 2). Participants were mostly under 30 years old (n=71, 60.7%), self-identified as women (n=92, 78.6%), and self-identified as White or Caucasian race and ethnicity (n=92, 78.6%). Most participants were residents (n=38, 32.5% post graduate year (PGY)-1; n=33, 28.2% PGY-2; n=23, 19.7% PGY-3/4). While response rates could not be quantified among faculty versus residents due to the inherent lack of demographic data for participants who did not complete at least 1 survey, participation rates relative to invitations were relatively similar for the 2 groups among the 115 participants who reported on their clinical role: n=19 of 149 (12.8%) for faculty and n=94 of 504 (18.7%) for residents. While the majority (n=108, 93.2%) reported having completed the Reach Out and Read Core Module, only 26.4% (n=31) reported having moderate or significant prior experience with emotional connection.

Survey data indicated that 64 out of 69 respondents (93%) reported that the module was a good way to learn about emotional connection (responses ranging between “somewhat agree” and “strongly agree”; Table 3). More than half of these respondents (n=40, 58%) reported they either “agree” or “strongly agree.” Similarly, 63 out of 69 respondents (91%) reported that the training provided valuable knowledge (responses ranging from “somewhat agree” to “strongly agree”; Table 3), with 39 (57%) reporting agreement to some degree and 24 (35%) reporting strong agreement. In addition, 60 out of 69 respondents (87%) reported satisfaction with the module length. Focus group data echoed a positive experience with the module content and format, with focus group data indicating that the videos embedded within the modules provided valuable examples of what to look for when assessing emotional connection:

A plurality of respondents reported that the training provided valuable knowledge, was a good way to learn about emotional connection, and presented strategies that could be implemented.

Suggested areas for improvement, reflected in both an open-ended survey question and in focus group data, frequently addressed aspects of content structure within the module. This included a desire for the incorporation of written summaries and/or reviews of the video content by experts either within or following module completion and clearer explanations of specific behaviors and examples of emotional connection within each video. Focus group data also indicated technical challenges with the log-in process, some challenges with the repetitive nature of the module videos, response fatigue, and a desire for added diversity in family structure within the videos (ie, the inclusion of more dyads beyond mother-baby):

Focus group data additionally explored the experience of how the module was presented to participants and when respondents were expected to complete the modules. Some respondents indicated that their programs set aside a specific time for module completion, while others reported completing it independently, with benefits for each format:

However, challenges were identified in both approaches: set-aside time may be rushed or pulled from busy schedules, but finding one’s own time could also be challenging. Distractions were experienced during module completion in both cases.

Regarding the module’s future use, 36 out of 44 respondents (82%) reported they would recommend this training to other clinicians. In addition, most respondents suggested this module be required (n=17 out of 44, 39%) or recommended (n=24 out of 44, 55%) as part of residency training. An additional 2 out of 44 respondents (5%) indicated it should be available but not promoted, and 1 respondent who selected an “other” response indicated this should not exist as a stand-alone activity but rather should be “incorporated into a larger/longer learning session where the nuances of such strategies can be explored through discussion”; this was echoed in focus group data, which indicated a desire for synchronous or live training or an opportunity for more interpersonal engagement through the training:

Additional survey data comparing responses on assessed outcomes before and after module completion (Table 4) indicated that the greatest changes were in familiarity with the concept of emotional connection (n=75, pre mean 54.20, SD 18.59; post mean 73.99, SD 14.73; Cohen d=1.14; P<.001) and confidence in observing the quality of the parent- or caregiver-infant relationship during well-child visits (n=75, pre mean 55.36, SD 18.49; post mean 74.20, SD 14.07; Cohen d=1.28; P<.001). In qualitative data from focus groups, respondents reported value in varied respects, including having a framework and model that provides guidance for the observation of emotional connection and that demonstrates the varied ways emotional connection can present and be observed:

Significant pre-to-post change was evidenced across all survey measures related to familiarity, confidence, and beliefs about ERH and emotional connection in well-child visits, and the expected estimated minimum time to observe the quality of the relationship between a parent-caregiver and infant during a well-child visit decreased.

Beliefs about ERH also exhibited a statistically significant change between the pre- and postsurvey. Specifically, participants exhibited a change in their beliefs that understanding ERH is essential to their work as a clinician (n=65, pre mean 80.41, SD 15.72; post mean 86, SD 14.20; Cohen d=0.49; P<.001), observing emotional connection in well-child visits will improve their practice as a clinician (n=46, pre mean 80.91, SD 15.73; post mean 84.15, SD 14.47; Cohen d=0.33; P=.03), and observing emotional connection in well-child visits will improve outcomes for patients and their families (n=47, pre mean 78.34, SD 16.01; post mean 82.68, SD 16.02; Cohen d=0.35; P=.02). As one focus group respondent elaborated:

Of note, the baseline (pre) data indicated considerably greater agreement (mean ranges 78.34‐80.91) with these assessed beliefs about emotional connection compared to assessed confidence in or familiarity with emotional connection (mean ranges 54.20‐55.36) prior to module engagement. This suggests the training is filling a learning gap that the respondents desire to gain expertise in.

In survey data, 66 out of 69 respondents (95%) reported that the strategies presented could be implemented in clinical practice (“somewhat agree” to “strongly agree” that they could be implemented) (Table 3). When asked to consider primary challenges to implementing the tools taught in the module in practice, the time required to assess emotional connection during a clinical encounter was the most noted challenge (30 out of 188 survey responses [25.6%] on a “select up to 3 response” item) (Table 5). As one focus group participant explained:

Respondents most often reported implementation challenges associated with the time required to assess emotional connection, knowing what to do if a problem was identified, and language and cultural challenges.

In addition, 19 out of 188 survey responses (16.2%) pointed to potential challenges with determining follow-up action steps (‘Knowing what to do if I identify a problem’), with focus group data echoing this:

Language and cultural differences between respondents and their patients and families were also cited as potential challenges (15 out of 188 survey responses [12.8%] and 14 out of 188 survey responses [12%], respectively), in addition to family willingness to engage in relevant discussion (13 out of 188 survey responses [11.1%]) (Table 5). Focus group data additionally raised the concern that well-child visits may not provide a good example of family dynamics at home:

ERH and emotional connection have been identified as foundational components of healthy child development, and yet there is a lack of scalable training resources to incorporate the observation of ERH into pediatric primary care practice [4,24-28]. Efforts to date have primarily focused on diagnosing and treating relational problems in children with relatively severe disturbances in behavior [29]. In addition to being narrowly targeted and costly, these efforts do not lend themselves to universal primary prevention [30]. Indeed, in a recent study exploring residency program directors’ views on the adequacy of current ERH training, a clear gap in this area of medical education was identified [31].

Here, we tested the feasibility of a brief self-paced, asynchronous, interactive ERH training module in teaching pediatricians the observation of emotional connection, an observable component of ERH. We found that the large majority of respondents reported that the information provided in this module gave them valuable knowledge on emotional connection, and that this module was a good way to learn about emotional connection. Despite social-emotional development and healthy relationships being domains addressed in the AAP Bright Futures guidelines and policy statements [5], these results clearly show that this is an area of educational growth needed for clinicians who were initially unfamiliar with these concepts. In addition, specific results provide key areas to target. For instance, pretraining scores for knowledge and confidence (familiarity with emotional connection and confidence in observing parent and caregiver–infant relationships during well-child visits) were lower than scores for beliefs about emotional connection (eg, its importance to clinical work, practice improvement, and patient outcomes). This may explain the larger effect sizes for knowledge and confidence, and it suggests the training addresses a key gap: residents value ERH and emotional connection but have less knowledge and confidence in observing them. This is also consistent with prior literature indicating knowledge and confidence to be more responsive to short trainings than beliefs and attitudes [32,33].

In a field as complex as medicine, evolving medical information necessitates a feasible and acceptable method for efficiently disseminating new knowledge. In recent years, asynchronous modules have been thought of as an efficient way to reach learners in different locations [34,35]; additionally, with time constraints on faculty and residents, modules developed by topic experts can be a valuable way to add to clinician education without coordinating timing for in-person learning. Access to modules can decrease the burden on individual programs or faculty to find ways to accomplish educational and curricular goals in both training and continuing medical education. Modules and other e-learning strategies have long been considered effective in accomplishing this goal [36,37] and were widely adopted as a key component of medical and graduate medical education during the COVID-19 pandemic [38,39]. In the domain of ERH, there is a clear benefit to these modules, which enable multiple learners across multiple programs to be trained in a similar manner, allowing for shared knowledge and a common language to discuss social-emotional health and development.

However, there is significant variability in the content and quality of medical education training modules. Intentional development of modules through interdisciplinary collaborations like the one presented here can potentially increase access to high-quality educational opportunities, especially for new or emerging topics. The results of this pilot training suggest that a~45-minute training module is a feasible and acceptable way to provide education about ERH and specifically emotional connection, and that providers found the training valuable to improve their knowledge base.

In addition to the knowledge that these modules provide and an acceptable means of delivering novel educational content, it is also important to look at the challenges participants face. Notably, we discovered challenges with identifying time to complete modules, and the benefits and drawbacks of asynchronous delivery are critical to consider. Additionally, concerns noted by participants about how to incorporate learned skills into already full well-child visits are pertinent to many topics as more components are introduced into the well-child visit. Given these challenges, it will be essential to develop trainings that address how to easily integrate the content into the workflow and how to make changes or adjustments to clinical practice. These benefits and challenges are generalizable to a wide range of topics and disciplines that are interested in expanding educational opportunities via asynchronous modules.

This study has several limitations, many of which can direct us to future opportunities for investigation. The first is that there may have been recruitment bias. Participants who engaged with the voluntary training module and study may have been interested in emotional connection or closely related topics that inspired them to participate. Response bias may also be a factor; specifically, social desirability in reporting knowledge or positive attitudes toward the subject or for residents who were asked to complete a module by their preceptors, regardless of the voluntary nature of the study [40]. Qualitative data provided some nuance on this possible bias. Although focus group participants provided an overall positive response to the module, they also displayed freedom to critique the training in this new feedback format, indicating that, indeed, the forced-choice survey data might be slightly inflated. Additionally, while there is value in the fact that these data represent 8 unique sites across the Carolinas Collaborative, they are all geographically located in the same area of the country, which limits generalizability. There was also a lack of diversity within the demographics of the participating clinicians. Additionally, adult learning styles were not considered for participants, and nuances in learning styles may make this training method more or less effective for individuals. Finally, this learning modality was not compared to a more traditional in-person experience, which was suggested by some participants as important and desired. We also do not have information from participants who started but failed to complete the modules. While we obtained some information from focus groups to understand improvements that could be made to the module, it would be important to better understand why some participants did not complete the training. Future studies will also need to evaluate if participation rates and satisfaction scores differ by sites that provide protected time for completion of the module versus those that do not.

Second, the study had a relatively low response rate. The timing of the study might have been a limiting factor in our successful recruitment of participants. Due to the timing of funding availability, modules were released at the beginning of June, and recruitment continued through October. Because the study primarily targeted residents and teaching faculty, timing may have contributed to the low response rate, as this is generally not an ideal time for change or new initiatives in academic centers. The voluntary nature of the training also likely contributed to the low response rate. This points to the necessity of embedding future iterations of such modules into already existing training infrastructure, such as the Reach Out and Read Core Curriculum, making it a mandatory component for certification.

In addition to these limitations, there are several areas that were not addressed by this study that call for future investigation. A concern mentioned by participants in both surveys and focus groups was that while there was thorough education about the importance of and strategies for identifying emotional connection, the modules did not address what to do when a lack of emotional connection was noted or an additional problem with social-emotional development was identified. Along this same theme, more investigation is needed to effectively address cultural and linguistic differences in parenting that could be perceived as “problems” without an appropriate appreciation for different cultural lenses to assess connection. Another future domain of study is how this education impacts clinicians’ daily practice. This could be accomplished by direct observed exams, which are the gold-standard in resident education to assess trainees’ competencies in various aspects of care [41,42]. Finally, we did not address the efficacy of the module, as defined in the ability to change how observers rate the video interactions compared to trained researchers.

Using the information learned here, we have created an updated iteration of this training module that is currently being piloted at Columbia University. This new version expands the training with a higher number of parent-child videos across a larger age range (0‐3 years), incorporates real-time feedback of participants’ ratings of emotional connection, and gives additional opportunities to score videos when the original score was out of range. The new module also includes several suggested actionable next steps if emotional connection is observed to be low, though participants are cautioned that light-touch intervention development targeting emotional connection is a current area under investigation.

ERH and emotional connection are foundational to healthy child development, and education about these topics delivered through asynchronous digital modules is highly valued by participants. While completion rates of voluntary training are generally low, including in this study, Reach Out and Read’s commitment to creation, validation, and eventual delivery of ERH training creates a scalable avenue for wide-scale implementation, given the organization’s presence in >6500 clinics across the 50 states. The data presented here inform next steps in the iterative design of medical education on ERH. Further research is needed to more fully understand how this training impacts clinician behaviors during clinical encounters, how families receiving care perceive these changes, the impact on child and family outcomes, and potential disparities in the perception of different cultural norms in parenting as demonstrative of strong or weak emotional connection, to avoid over- or underidentification of deficits in this area.