A Comprehensive COVID-19 Daily News and Medical Literature Briefing to Inform Health Care and Policy in New Mexico: Implementation Study

Introduction

On March 11, 2020, New Mexico Governor, Michelle Lujan Grisham, and the New Mexico Department of Health declared a public health emergency in response to the COVID-19 pandemic, after announcing 3 New Mexico residents tested presumptive positive for COVID-19 [1]. New Mexico Department of Health responded by creating the Medical Advisory Team, which brought together state officials, health care providers, and community members to compile and disseminate scientific findings, and create guidelines and recommendations to navigate the challenges of the pandemic. With the growing number of scientific publications and news reports and the potential for misinformation dissemination into the community, the Medical Advisory Team reached out to faculty at the University of New Mexico (UNM) to form a team to analyze and distribute reliable information to inform health care and public policy decisions for the state.

There was a high volume of both vital and inaccurate information available regarding COVID-19 [2,3]. To reduce information overload and misinformation, quality content had to be filtered and consolidated for state and health care leaders [4-6]. Researchers recommended the use of official public health organization websites as the most reliable source of information on COVID-19 preventative measures [7].

Prior to the COVID-19 pandemic, there was little guidance on methodology to generate comprehensive daily briefings. We aimed to provide guidance on how to create a daily briefing to address the high volume of information and misinformation. We surveyed the readers to determine if the briefing influenced their professional practice, if it was a main source of information, and if it helped combat misinformation. Readers also had the opportunity to share their thoughts on the briefings in free text.

During the first few months of the global pandemic, medical schools and various medical education governing bodies agreed that clinical medical education needed to be suspended due to high infectivity risk, limited COVID-19 testing supplies, and limited personal protective equipment (PPE). Studies have since assessed how psychologically and educationally detrimental it can be to study medicine in isolation. Participants in 1 study found an increase in depression, detachment from family and friends, and hopelessness, with a decrease in work performance and study time [8]. While virtual platforms are less ideal than in-person learning, learners found team-based projects to be more engaging [9,10]. Medical students were given the opportunity to take part in this project with the goal of increasing engagement during a difficult time of learning. We aimed to qualitatively determine if participation in the briefings showed net benefit to virtual medical education.

With a robust team of professionals, students, and volunteers, a comprehensive daily briefing was first disseminated on April 5, 2020, as the “UNM Global Health COVID-19 Intelligence Briefing.” Here, we describe the process of creating such briefings, the usefulness of these briefings to the community and its leaders, as well as their benefit to virtual medical education.

Methods

Briefing Process

A team of medical doctors, PhDs, journalists, graduate students, medical students, and researchers from both the United States and Kenya volunteered to participate in the composition of the daily briefings. Microsoft Teams served as the platform for automatic and manual article collection and triaging, composition of the briefing, as well as the administration of a qualitative survey. Microsoft Teams Flows were developed to automatically gather COVID-19–related papers and reports from various sources including Twitter (eg, BBC, JAMA, New Mexico Department of Health, New Mexico Office of the Governor, President of the United States), Google, LitCovid (National Institutes of Health), World Health Organization, and the Centers for Disease Control and Prevention while excluding duplicate URLs (Figure 1; Multimedia Appendix 1). Articles were categorized into “New Mexico Mainstream Media,” “Manual Requests,” “US Mainstream Media,” “International Media,” “Health Organization,” “Science and Medicine,” and “Literature” and were assigned a default “medium” priority tag through Microsoft Planner. Articles manually submitted by readers through a submission link from each briefing generated a Microsoft Teams Planner task and were reviewed and triaged accordingly. Triaging was completed by 3 administrators to ensure quality and consistency.

‎View this figure

Articles were manually triaged by administrators into “urgent,” “important,” or “low” priority groups. Scientific reports were triaged based on the veracity and potential impact on COVID-19–related health care or public policy response. Information on epidemiology, testing, public guidelines, medical practice guidelines, new therapies, vaccines, and pathogenesis were of primary focus. Evidentiary priority was given to systematic reviews of randomized controlled trials with meta-analyses and individual randomized controlled trials, followed by quasiexperimental, case control, or cohort studies. These were often labeled “urgent.” Studies with small sample sizes or problematic study designs were included or excluded at the triaging administrator’s judgment. Opinions pieces were largely excluded. News articles covering New Mexico, United States, and international pandemic responses and impacts were included based on relevance and content validity.

Articles labeled “important” were analyzed and summarized by a team member to include the type of study, pertinent results, other relevant information, the Digital Object Identifier, or hyperlink, and marked as “complete.” Headlines were carefully constructed to be as informative as possible rather than as “teasers.” Completed articles were autopopulated first into a Microsoft Excel spreadsheet, then relocated to a Microsoft Word document. Items typically triaged as “low” priority were commentaries, editorials, political messages, and non–evidence-based studies, and excluded from the briefing. The briefing was divided into the following sections: “Executive Summary;” “New Mexico Highlights;” “US Highlights;” “Economics, Workforce, Supply Chain, PPE Highlights;” “Epidemiology Highlights;” “Healthcare Policy Recommendations;” “Practice Guidelines;” “Testing;” “Drugs, Vaccines, Therapeutics, Clinical Trials;” and “Other Science.” Volunteers collaboratively edited the document with videoconference coordination each evening to ensure quality, clarity, accuracy, grammar, and proper citation. Reports with insufficient sample size, unclear conclusions, or poorly executed study designs were excluded. The final document included an “Executive Summary” with the general content, a disclaimer noting the inclusion of non–peer-reviewed content, and a list of all the participants of the day. The finalized briefing was emailed to the New Mexico Medical Advisory Team, sent to a listserv of subscribers, and posted on the publicly accessible UNM digital repository [11]. The UNM Department of Internal Medicine also incorporated the briefing into its daily department email. Listserv subscribers consisted of health care providers, researchers, and government employees, as well as UNM faculty, students, and readers not involved in health care or state policy. The UNM Digital Repository runs on BePress, which tracks global downloads of the briefings. The briefings have also been indexed by Google Scholar. There was no monetary gain associated with the creation of these briefings, with full access provided free of charge.

COVID-19 Global Health Briefing Survey

To assess the usefulness of the briefings to New Mexico policy makers, researchers, educators, physicians, and other health care professionals, an 18-question survey was created through Microsoft Forms, approved by the Institutional Review Board (20-263), and distributed to all briefing recipients. Survey participant demographics included academic degree(s) and place of occupation. We determined if participants provided direct clinical care to COVID-19–positive patients, if the briefing was their primary source of information regarding the pandemic, and how likely they were to share the briefing with colleagues.

To determine the briefing’s usefulness to the respondent, we assessed the following on a scale from strongly disagree to strongly agree:

1. The daily briefing has informed or changed my response to the COVID-19 crisis.
2. I have gained relevant knowledge I wouldn’t have otherwise because of this briefing.
3. This briefing has enabled me to combat misinformation.
4. This briefing has enabled me to clarify information I have seen elsewhere.

Lastly, the respondent could respond in free text how information from the briefing was applied professionally or personally and if they had comments or suggestions to improve the briefings.

Medical Education

Medical student education in the hospital setting was suspended, and students were not involved in direct patient care during the initial surge of COVID-19 around the globe. The UNM School of Medicine created a virtual COVID-19 course to allow for virtual didactics as well as asynchronous, self-directed learning. This course allowed students to be involved in various projects and, with curriculum committee approval, receive a 4th year elective credit for those efforts. A few students who were involved in the briefing proposed to the UNM School of Medicine curriculum committee that contribution to the briefings should count as course credit. The UNM School of Medicine approved the proposal.

Medical students were involved in manual submission of information, summarizing articles, editing the briefings every evening, and recruiting other students to join the project. The students also assisted in the assembly of the COVID-19 global health briefing survey. Lastly, the compilation and analysis of the data in this publication was entirely led by medical students who took part in various stages of the COVID-19 briefing development and dissemination.

Results

Briefing Process

Microsoft Teams Flows gathered an average of 456 articles daily, and 560 articles were manually submitted. A total of 39,715 articles were gathered throughout this project. Between April 5, 2020, to June 30, 2020, 58 UNM Global Health COVID-19 intelligence briefings were generated and published through the efforts of 68 individuals, including 19 faculty members, 31 medical students, 3 graduate students, 1 postdoctoral fellow, 3 staff members, and 11 contributors from outside of the UNM.

Each briefing was directly emailed to 176 members of the New Mexico Medical Advisory Team before the availability of the listserv on April 27, 2020. By June 30, 2020, the listserv had 400 subscribers. The UNM Department of Internal Medicine included the briefing in its daily newsletter sent to 525 members. By June 30, 2020, the briefing was sent to 1080 unique email addresses—693 within the UNM Health and Sciences Center and 387 outside of it.

Beginning April 24, 2020, all briefings were uploaded to the UNM Digital Repository for public access. The number of briefing downloads is plotted both daily and cumulatively in Figure 2. Between April 24, 2020, and July 5, 2020, there were 5047 downloads, with an average of 74 downloads per day. The highest number of downloads for an individual briefing was 260 (April 27, 2020). The average number of downloads was 85, the median 72, the maximum 260, and the minimum 4.

Throughout its entire duration, the briefings were downloaded in 116 countries (Figure 3). The 5 countries with the most downloads were the United States with 2164, Brazil with 235, India with 233, Canada with 224, and Germany with 224.

‎View this figure

‎View this figure

COVID-19 Global Health Briefing Survey

To evaluate the usefulness of the briefings, we sent an electronic survey to approximately 994 readers. A total of 111 individuals (approximately 11%) responded between May 8 and May 19, 2020. Among the 111 responders, 41 (37%) were physicians, 41 (37%) nonclinical academic faculty, 9 (8%) administrators, 8 (7%) academic staff, 5 (5%) government employees, 4 (4%) students, and 3 (3%) nurses. Of the 111 respondents, 30 (27%) were involved in providing direct clinical care to COVID-19 patients, 47 (42%) had MD or DO degrees, 34 (31%) had PhDs, and 9 (8%) had MPH degrees. The health care providers who responded (n=30) had an average of 19 years of clinical practice (95% CI 15-23).

A majority of respondents (105/111, 95%) agreed or strongly agreed that the briefing helped them gain relevant knowledge, 71% (79/111) changed their response to the pandemic, 79% (88/111) reported the briefings helped them combat public misinformation, and 89% (99/111) said it helped to clarify the information from other data sources (Figure 4). Moreover, 24% (27/111) of respondents cited the briefing as their primary source of information on the pandemic, and 73% (81/111) reported having shared the briefing with their colleagues. On a scale of 0 (“would never share”) to 10 (“will definitely share”), the respondents reported being very likely to continue to share the briefing with colleagues (mean score 8.8, 95% CI 8.5-9.1).

The briefings influenced New Mexico state government response to the pandemic. David Scrase, the New Mexico Cabinet Secretary for Health and Human Services, informed our team that the daily briefing influenced dozens of policy decisions including the following: (1) mandating universal mask use in New Mexico early in the pandemic; (2) expansion of remdesivir treatment; (3) caution about hydroxychloroquine treatment; (4) selecting R_effective as a key gating criterion for the state (COVID spread rate); (5) guiding the adequacy of the PPE supply chain (particularly overseas); and (6) recommending against the use of antibody testing as an adjunct to clinical (or patient) decision making (David R Scrase email communication, June 16, 2020).

The briefings also received positive feedback from other health and policy officials around the state, including the UNM associate dean of Continuous Professional Learning, the Presbyterian chief medical and transformation officer, the vice chair of veterans’ affairs, the vice chancellor for clinical affairs, lead of New Mexico Medical Advisory Team, and the president and chief executive officer of Christus St. Vincent Regional Medical Center (Table 1).

‎View this figure

Medical Education

The UNM Global Health COVID-19 Intelligence Report project enhanced medical student education. A total of 31 medical students contributed to the creation of the daily briefings. Participation in the project was approved for course credit by the UNM School of Medicine curriculum committee. The students were able to practice the analysis and quality assessment of scientific data, and they found participation in the briefings to be informative and rewarding (Textbox 1).

Discussion

Strengths and Principal Findings

The development of the briefing succeeded in informing the practice of physicians and other health care personnel, facilitating COVID-19 research efforts by faculty and staff in academia, influencing policy making by the government of New Mexico, and alerting the public about the most relevant developments as is evident in our survey results. Additionally, the briefings were accessed by approximately 3 times the number of people compared with those who were directly emailed and catered to an audience beyond the scientific and medical community as the categorization of articles with headers and summaries reported in everyday language.

This project also succeeded in allowing for the continuation of medical education during the initial surge of COVID-19 when medical students and research students were prohibited from their normal duties. The learners improved on their ability to analyze a scientific paper and helped them empathize with the public regarding misinformation. The students reported being part of a community project to be meaningful and that influencing the health of New Mexico reduced student anxiety of being away from direct patient care.

The utilization of a listserv and online archive helped expand the audience beyond medical advisory team and the state of New Mexico to the briefing's eventual global reach. Organizing interest through a listserv would help disseminate the material quicker and broader should another similar endeavor be attempted.

Due to both limited funding and volunteers, the effort was brought to a close as the final briefing was published on June 30, 2020. We were unable to continue the production of thorough, high-quality publications free of charge due to the inevitable loss of many of our volunteers. We were unable to replace team administrators and leadership to triage daily and ensure quality control. Medical students returned to clinical duties, and faculty were unable to simultaneously sustain this unfunded effort along with their teaching, research, and administrative responsibilities. Between April 9, 2020, to July 2, 2021, a similar COVID-19 briefing effort was created (“COVID-19 Literature Surveillance Team”) of medical students and faculty, who were able to continue their reports by changing their daily briefing to weekly [12]; however, this team ultimately discontinued publication [13]. Our COVID-19 briefing efforts were closed with a presentation that expressed gratitude to all who participated [14].

Limitations

Contributors and authors of the briefings had variable expertise in assessing scientific literature. Initially, all participants were allowed to triage content. Due to quality concerns, triaging was reduced to 3 administrators. It was not obvious how to standardize the judgment process that went into rapid decision-making regarding article inclusion. Those triaging needed to be aware of the prior body of work addressed, be able to analyze an article quickly, and be well versed in various research methodologies and their assessment. Standardized training for editors and authors was implemented to improve the consistency and quality of article summaries.

Initial article selection was less stringent and included more speculative reports. More selective criteria were implemented over time. The inclusion of articles of varying evidence, such as preprints, received criticism. Recent studies have shown that the standard in preprints for the life sciences is similar to that of peer-reviewed articles; therefore, they can be considered valid scientific contributions [15]. As a clarification, our later briefings included the source and type of study (preprint, meta-analysis, Reuters, etc) before each summary.

A major concern was the disagreement between global governing entities on treatment and the quickly evolving understanding of the virus. For example, the US Food and Drug Administration granted emergency use authorization for both chloroquine and hydroxychloroquine for certain hospitalized COVID-19 patients on March 28, 2020, whereas the European Union did not allow for its use. The emergency use authorization was revoked on June 15, 2020, due to side effects of the medications.

While our survey was disseminated to all subscribers of the briefing, persons who may have derived less value from the briefings or had more substantial criticism of the publication were not well represented among the respondents. While favorable impact ratings were provided by survey respondents, our free-text section (on how the respondent-applied information from the briefing influenced their active response to the pandemic) could have benefitted from more specific wording to solicit specific uses. The most substantive feedback came from a nonsurvey source, namely Secretary Scrase via email.

Comparison to Prior Work and Future Directions

Other programs have released daily briefings [16,17,18,19,20] in similar efforts to combat information overload and implement quality control during the COVID-19 pandemic. At the time of this publication, there has been no research that has compared briefing dissemination to other forms of information dissemination during a rapidly changing health crisis. The pandemic has shifted the scientific community toward a more robust and rapid data-sharing culture [21]. Future work could compare the efficacy of briefings compared to other measures to alter and improve policy and information dissemination.