BWS is a type of DCE based in economic theory. DCEs elicit respondent preferences for goods or services based on their stated intentions in hypothetical situations [17]. The term utility refers to the mathematical representation of preference, which, under microeconomic theory, assumes that decision makers will make choices that maximize the value of their utility function, subject to constrained resources [18].

There are 3 types of BWS surveys that differ in survey design complexity. More complex BWS survey types, such as profile or multiprofile case, allow for the comparison of factors with multiple attributes (eg, a medication with different prices, side effects, and modes of administration). The simplest version, the object case (also known as MaxDiff), was used for this study, which determines the relative value of a list of mutually exclusive objects [16]. An example of a BWS object case question is shown in Figure 1, where respondents are asked to identify the most preferred and least preferred item from a set of scenarios.

The first step in designing this BWS experiment was to determine the relevant factors to be included in the study. We initially used a comprehensive literature search to identify commonly cited barriers to wellness support and various help-seeking interventions [11,19,20]. We then worked with contributing authors ET and TMS, who serve as assistant program directors in the internal medicine residency program and psychiatry residency program, respectively, to elicit their experiences while working with resident physicians with regard to help-seeking interventions and frequently cited barriers to seeking care. AW, VR, TMS, and ET then worked together to optimize this list of factors such that the items were clearly defined, relevant, and did not have a singular factor that would not be universally selected as best or worst (item dominance), which are required for BWS study generation [17]. Item optimization was further refined in the pretesting stage, as described in the following section.

Inherent in BWS experimental design is the requirement to have systematic combinations of all surveyed items. This necessitates the need for a balanced design in frequency and orthogonality, where each surveyed factor needs to appear an equal number of times and equally with other factors, ensuring that each factor can be compared with all possible arrangements of other factors in the set. This was achieved using a balanced incomplete block design, a commonly used technique to design BWS sets and generated in JMP Pro 14 (SAS Institute Inc) [9,21]. Three BWS object case sets were created to assess preferences for each of the following factors: seeking support for work-related stress, barriers to using counseling, and barriers to using peer support groups. The three sets each tested seven factors, with every question in the set containing a subset of four factors, with a total of seven questions per set, which adhered to a balanced incomplete block design.

After the initial study design, AW pretested the survey with a subset of 9 volunteer psychiatry residents. Adhering to a previous framework on BWS instrument development, we focused pretesting on the comprehension of the surveyed factors, whether there were omitted factors, the clarity of the BWS format, and the study purpose [22].

Before study deployment, we worked with the Committee on Clinical Investigation for Beth Israel Deaconess Medical Center (BIDMC), the institutional review board for our institution, to receive an institutional review board exemption for this study. BIDMC is a teaching hospital in Boston, Massachusetts, with academic affiliations to Harvard Medical School. The internal medicine residency program is one of 13 Accreditation Council for Graduate Medical Education–accredited programs at the institution and comprises a total of 163 residents. It is a 3-year residency program that comprises resident physicians in their first postgraduate year of training (PGY-1) to their third postgraduate year of training (PGY-3). To be eligible for the survey, participants needed to be a resident physician between PGY-1 and PGY-3 years of training in the internal medicine program at BIDMC. Chief residents (who are in PGY-4 or PGY-5 of their training) were not invited, and the study did not include residents of other specialties.

In October 2020, residents in the BIDMC Internal Medicine residency were invited via email to participate in the study. This email contained a prospective agreement with consent and a link to the study, administered using Qualtrics Survey Software (SAP SE), emphasizing that participation in the study was voluntary and that data collected would not be individually identifiable. The demographic data collected included their age, postgraduate training year, and gender. These data were stored on Qualtrics Survey Software servers, which could only be accessed on an account that required two-factor encryption, for which only AW had access.

A total of 3 reminder emails were sent 1, 2, and 4 weeks after the study opened. After 6 weeks, the survey collection period ended. A US $5 Amazon gift card was given as compensation conditional to the completion of the survey.

A typical survey response rate for a web-based survey is 40%, which, using a margin of error calculation with our sample size, would have resulted in a margin of error of 9%. Given the desire to explore the novel nature of this study method and the possibility that it could result in highly discriminative utility values, we aimed for a 50% response rate, which would result in a margin of error of 8%. A BWS-specific power analysis was also conducted, which indicated that a 50% response rate (n=82) would have an 80% chance of detecting utility differences >0.118 at 95% CI; with a 40% response rate (n=65), we would be able to determine utility differences of 0.132 at 95% CI [23].

BWS data were analyzed using three techniques (Best-worst [B-W] scoring, multinomial logit [MNL], and latent class analysis [LCA]). For more details on multinomial logit and latent class analyses in this study, please refer to the Multimedia Appendices 1 and 2.

B-W scoring was conducted as described in previous BWS studies [13].

B-W scoring = ([number of times item selected best choice] – [number of times item selected worst]) / (number of times surveyed factor displayed in the total survey)

B-W scores have a range between −1.0 and +1.0, with scores closer to +1.0 having higher selected preference than lower-scoring options.

MNL and LCA modeling were performed using Lighthouse Studio 9.8 (Sawtooth Software). One-way analysis of variance with Tukey honest significant difference (HSD) posthoc test was performed using JMP Pro 14 to analyze the mean differences between factor preferences for survey respondents.

Rank ordering of factors was determined by sorting the raw utility values of the surveyed factors.

To our knowledge, this is the first known survey on resident burnout and wellness support that uses a BWS methodology. This framework allows resident program leadership to gather specific information on what residents value when it comes to seeking help and relevant barriers. Overall, surveyed residents most preferred to seek emotional support informally from their resident peers and with friends and family when presented with other options of formal mental health support or administrative support. When examining self-stated barriers to using therapy, residents most frequently cited pragmatic barriers of time and money rather than those related to mental health stigma.

Our study findings also reflect prior studies where time was the most frequently cited barrier to using mental health services among resident physicians [11,24]. Future efforts designed to increase use should thus consider how to best mitigate this barrier, such as dedicating specific time in a resident's schedule to allow for medical appointments or to assess the viability of using telehealth services for these purposes, which would allow for more flexibility on a resident's schedule [20,25]. Although the preference for informal support over formal counseling among physicians would need to be reestablished in the future with a larger sample size, a recent BWS study examining help-seeking preferences for mental health concerns among college students also similarly reflected a preference toward informal sources of support through friends and family over formal counseling [26]. As such, when considering the low overall utilization rate of mental health services among physicians, it may also be worth considering that preferences for informal support over formal counseling exist among the general population and that this may not represent a particularly unique issue among physicians, although this is certainly worth investigating in the future.

The BWS framework also allows for the use of latent class modeling, identifying segments of residents that share similar preferences. Although these findings are difficult to generalize based on our small study size, the identification of segments of residents that mostly prefer formal counseling, prefer to deal with problems on their own, or prefer informal supports represent potential phenotypes of residents that warrant future investigation, such as whether the frequency of these groupings is stable across differing institutions, specialties, and countries. Although future studies showing similar frequencies and phenotypes would aid in the generalizability of our study findings regarding physician resident attitudes on barriers to using mental health and seeking help, it cannot be expected that preferences will be identical, given the aforementioned variations in clinical settings and the variety of personality traits and personal values that physicians possess [27]. What could also be more useful for policy makers would be using repeated assessments at singular institutions of BWS-based studies across time to assess resident physician preferences regarding wellness initiatives longitudinally, which would also aid in determining the impact of potential interventions designed to encourage wellness initiative utilization.

The external validity of this study was limited by the singular specialty surveyed. Respondents only included 47% of the total internal medicine residency and may not reflect the aggregate preferences of all medical residents. There is a possibility that nonresponders could have had differing preferences from those that responded; given how a previous study examining physician response rates to web-based surveys cited lack of time as a primary reason for not responding to surveys, there is a possibility that the magnitude of time as a barrier to seeking help was understated [28]. As we did not survey other residents in other specialties, we cannot determine whether these preferences are generalizable to all residents or if they differ by specialty. However, we believe that the surveyed factors related to help-seeking barriers are universal to many residents; notably, most residents struggle with being able to take time to seek formal counseling, and issues of self-stigma and social stigma, when it comes to receiving mental health services, are not solely limited to our residency program. Furthermore, mental health interventions such as individual therapy and peer support are commonly offered wellness supports in many institutions. There is no reason to believe that these barriers are not encountered by other resident physicians; however, it would be interesting to repeat this survey in other specialties to observe the generalizability of our collected preferences. Although our study sample contained a higher proportion of female respondents, raising a concern that data would be skewed, we did not find meaningful differences in preference choices by gender, as shown in Table S5 in Multimedia Appendix 1.

To our knowledge, this is the first known study that uses a BWS approach to establish resident preferences for seeking support. [2]. In our study, understanding physician preferences for seeking help and stated barriers to seeking formal help was useful in generating potential reasons why these resources have been underutilized at our institution, in particular, a preference for informal help and strongly cited barriers related to time constraints.

By using a BWS framework for wellness initiative implementation, institutions can accurately assess the relevant barriers and demand for wellness services among their clinician population, for which these preferences will likely vary because of variations in practice settings, clinician specialties, and institutional culture. The design of BWS studies allows for repeated measurements of surveyed factors in a more efficient study design, allowing for more reliable measures of preferences of these factors on an individual respondent level in a survey that takes only a few minutes to complete. In contrast, traditional surveys using Likert scales cannot establish these preferences as they do not collect repeated measures of these data, do not force respondents to choose between surveyed factors, and are subject to scale-use bias, allowing policy makers to better understand how respondents value the surveyed factors.

This method may save money and time by reducing the likelihood of institutions implementing wellness initiatives that turn out to be underutilized because of unforeseen barriers and unique preferences for services. Physician residents are not a homogenous population where a singular intervention will be universally helpful to improve their wellness, as suggested by the lack of evidence of such a universal intervention and wide range of burnout prevalence among practice settings.

Although the design of BWS studies requires some understanding of the underlying discrete choice theory, several commonly used statistical platforms allow for the design and analysis of these studies (JMP, SAS, Qualtrics Survey Software, STATA, and R). Thus, the BWS framework provides an accessible opportunity to create personalized approaches to addressing clinician wellness.