The United States Medical Licensing Examination (USMLE) Step 1, arguably the most significant assessment in the USMLE examination series, changed from a 3-digit score to a pass/fail outcome in January 2022. Given the rapidly evolving body of literature on this subject, this paper aims to provide a comprehensive review of the historical context and impact of this change on various stakeholders involved in residency selection. For this, relevant keyword-based searches were performed in PubMed, Google Scholar, and Scopus to identify relevant literature. Given the unique history of USMLE Step 1 in the US residency selection process and the score’s correlation with future performance in board-certifying examinations in different specialties, this scoring change is predicted to significantly impact US Doctor of Medicine students, US Doctor of Osteopathic Medicine students, international medical graduates, and residency program directors, among others. The significance and the rationale of the pass/fail change along with the implications for both residency applicants and educators are also summarized in this paper. Although medical programs, academic institutions, and residency organizing bodies across the United States have swiftly stepped up to ensure a seamless transition and have attempted to ensure equity for all, the conversion process carries considerable uncertainty for residency applicants. For educators, the increasing number of applications conflicts with holistic application screening, leading to the expected greater use of objective measures, with USMLE Step 2 Clinical Knowledge likely becoming the preferred screening tool in lieu of Step 1.JMIR Med Educ 2023;9:e37069
The United States Medical Licensing Examination (USMLE) consists of 3 examinations (USMLE Step 1, Step 2, and Step 3) that medical students/graduates must pass before entering and completing postgraduate clinical residency training in the United States . The USMLE program is jointly administered by the National Board of Medical Examiners (NBME), the Educational Commission for Foreign Medical Graduates (ECFMG), and the Federation of State Medical Boards (FSMB) [ - ]. The USMLE Step 1 tests candidates’ knowledge of the preclinical basic sciences, namely, anatomy, biochemistry, immunology, microbiology, pathology, and pharmacology, while Steps 2 and 3 test candidates’ clinical knowledge. Typically, USMLE Steps 1 and 2 are completed by US students—both MD (Doctor of Medicine) and DO (Doctor of Osteopathic Medicine) candidates during medical school. USMLE Step 2 has historically been composed of 2 components: Step 2 CK (clinical knowledge) and Step 2 Clinical Skills. USMLE Step 3 is typically completed by these students just after medical school graduation or during residency.
For over 16 years, the USMLE Step 1, Step 2 CK, and Step 3 have been criterion-referenced, computer-based assessments. These exams historically provided a 3-digit score, similar to the Medical Council of Canada Qualifying Examination (MCCQE) Part I examination in Canada , the National Eligibility cum Entrance Test for Post-Graduation (NEET-PG) in India [ ], and the Comprehensive Osteopathic Medical Licensing Examination (COMLEX) of the United States, taken by students of DO schools alongside the USMLE, which all provide numeric scores and percentiles. However, these exams are different from USMLE’s counterparts in the United Kingdom, where the Professional and Linguistic Assessment Board 1 and 2 examinations function as pass/fail-only assessments. Meanwhile, the USMLE Step 2 Clinical Skills exam evaluated candidates through an in-person structured clinical assessment and provided only a pass/fail outcome. However, the latter, first introduced in 2004, was permanently suspended in 2020 due to COVID-19–related restrictions on testing [ ]. This change resulted in only 3 tests remaining for candidates aiming to join and complete a residency program in the United States, all providing 3-digit scores for candidates passing them.
In March 2019, the Invitational Conference on USMLE Scoring (InCUS) was held with delegates from 5 major bodies of medical education in the United States—Association of American Medical Colleges, American Medical Association, NBME, FSMB, and ECFMG—with the aim being to “facilitate broader system-wide changes to improve the transition from undergraduate medical education to graduate medical education” . The group, as a consensus, felt that the current system merited wide-spanning changes. In the following year, in 2020, FSMB and NBME announced that score reporting for USMLE Step 1 would change from a 3-digit numeric score to reporting a pass/fail outcome [ , ]. This change finally came into effect on January 26, 2022. Notably, NBME and ECFMG announced that all scores for USMLE Step 1 exams taken prior to the date of change will continue to be reported as the traditional 3-digit score, with no retroactive alteration of transcripts [ ]. In a parallel move, the National Board of Osteopathic Medical Examiners announced that COMLEX Level 1—the first of the 3 exams taken by DO candidates as a requirement for osteopathic medicine licensure, as well as medical school graduation, would also transition to a pass/fail reporting system from May 2022 [ ].
At the time of writing this paper, less than a year has passed since the scoring change came into effect. Importantly, candidates who had taken and obtained a score on USMLE Step 1 would not have their scores turned to pass/fail at any time in the future. In the US Residency Match Cycle of 2023, which is ongoing at the time of writing, there is a substantial, although unquantified, proportion of candidates with a pass/fail outcome, while several applicants have Step 1 scores. The vast majority of medical students receiving pass/fail reports will likely apply only in the Match Cycle of 2024 and beyond; therefore, definitive implications of this change remain to be seen.
Given the rapidly evolving body of literature on this subject, this paper aims to provide a comprehensive summary of the historical context of this change and the potential impact on various stakeholders involved in residency selection. This paper also aims to review the key studies that have emerged since the pass/fail change was announced to happen. For this, appropriate keyword-based searches were performed in PubMed, Google Scholar, and Scopus in order to identify relevant literature. Empirical data on the impact of this change can only be assessed from literature emerging after the conclusion of Match 2023 and potentially even Match 2024. However, some comprehension may be reached from reviewing the surveys and perspectives coauthored by applicants, program directors, leadership of professional organizations, etc, discussing the potential impact of the change.
Significance of the USMLE scores
The USMLE was originally intended only for licensure purposes . However, over the years, residency and fellowship programs increasingly co-opted USMLE scores for secondary uses, with these scores gradually becoming one of the most important factors influencing residency selection [ ]. According to a 2020 survey by the National Residency Matching Program, 90% of the program directors considered candidates’ USMLE Step 1 score while deciding whether to invite them for an interview, with 55% reporting that they had a target score for candidates, implying the use of Step 1 as a screening tool [ ]. The reliance on USMLE Step 1 scores for residency application considerations was particularly notable in competitive specialties. A case in point is a survey of over half of all neurosurgical residency program directors that found that 77% of them had always screened candidates using Step 1 scores [ ], and a score of >245 was the most significant predictor of success in the neurosurgery match (1990-2007) [ ]. Thus, aspirants for these specialties would find their specialty of choice out of reach if they had a low Step 1 score. In addition to residency selection, Step 1 scores were utilized for selection into honor societies and away rotations, which also influence, albeit to a lesser extent, the residency selection.
Performance in the Step 1 examination was also known to be widely correlated with performance on in-training exams taken during residency and with board certification passing rates, as demonstrated by a large amount of published literature across numerous specialties. For instance, Swanson et al  reported in 2009 that orthopedic surgery residents having low scores on Step 1 and Step 2 CK were at significantly higher risk of failing the Part I of the American Board of Orthopedic Surgery Certifying Examination. Similarly, in 2010, Dougherty et al [ ] reported that Step 1 scores correlated with American Board of Orthopedic Surgery Part 1 scores and commented that it may continue to be used in resident selection. Likewise, in a multicentric study, de Virgilio and colleagues [ ] reported that those general surgery residents who were potentially at risk of failing the American Board of Surgery qualifying and certifying examinations could be identified early if they had a low Step 1 score. Additionally, Step 1 and Step 2 CK scores were correlated with better performance in the American Board of Emergency Medicine certifying examination, as reported in a multicenter study by Harmouche et al [ ]. Further, in 2021, Filiberto et al [ ], through a single-institution study of interns in all specialties, determined that step scores were significantly associated with better evaluations of intern performance by program directors.
Rationale Behind the Scoring Change
The original purpose of the criterion-referenced examinations such as the USMLE, COMLEX, and MCCQE was not for sorting candidates for residency selection as done by the NEET-PG in India . Rather, these exams were intended to be an assessment of the candidate’s competence for practice [ , ]. Thus, the USMLE Step 1 was primarily intended to deliver a pass/fail standard, but its scores in effect gradually became the major attribute being utilized by stakeholders in residency selection for decades [ ]. Although the pass/fail standard (criterion-referencing) of the USMLE Step 1 was valid, reliable, and defensible, the same could not be said for its sorting function (norm-referencing). Thus, the primary rationale for the change was the attempt by licensing authorities to restore the USMLE Step 1 and COMLEX Level 1 to their original intended purpose [ ]. Additionally, the overreliance on Step 1 as a screening tool often led students to prioritize this exam over the in-house medical curriculum at their respective institutions, with students reportedly showing less commitment to competencies not deemed “high yield” on the Step 1 exam [ - ]. A reported mismatch between their in-house curriculum and Step 1 preparation existed, in effect, a parallel curriculum [ - ]. Furthermore, students belonging to disadvantaged and underrepresented groups in medicine have historically and consistently scored lower on standardized exams, including the USMLE Step 1, stemming from a multitude of socioeconomic factors. Step 1 scores were therefore correlated with racial and demographic disparities, disproportionately impacting underrepresented minority candidates [ , ]. Additionally, several medical educators argued that Step 1 scores could not assess other crucial, yet subjective, competencies such as interpersonal skills and professionalism [ ]. Thus, it was hoped that decreasing the reliance on Step 1 could help expand the holistic consideration of applicants from all backgrounds [ ]. Although these limitations have been long-standing, little change had taken place in several years; therefore, when this change was announced, it was met with much surprise and concern.
Impact of the USMLE Step 1 Scoring Change on Applicants
The impact of the USMLE Step 1 scoring change is likely to be enormous on all applicants, including US-MDs, US-DOs, and international medical graduates (IMGs), who may be either US citizen IMGs or non-US citizen IMGs, with the latter also known as foreign medical graduates. This impact was captured in several publications through surveys of residency program directors and applicants. However, these data should be interpreted with caution, as surveys are intrinsically limited by their response rates. If the response rate is 45%—the rate in the survey by Makhoul et al —the survey’s bias is estimated to be 55% [ ]. Response rates may also be related to representativeness, which further exacerbates this bias. Additional limitations include (1) a central tendency bias due to the use of a Likert scale [ ], (2) potential selection bias of those with stronger opinions regarding the change, and (3) a lack of subgroup analysis of responding programs due to anonymity in reporting. Additionally, there are studies such as those done on the otorhinolaryngology residency application process [ ], which have used different questionnaires; hence, findings from specialties may not be compared directly.
The major works that have been published on USMLE Step 1 scoring conversion are summarized in. Of note is the paper by Makhoul and colleagues [ ] in the New England Journal of Medicine, with similar specialty-specific papers derived from data collected by this research group also published and widely available. The authors conducted a seminal survey of over 2000 program directors from various specialties, with responses providing clues regarding the impact of the scoring change on applicants [ ]. Approximately 81% of the program directors felt that USMLE Step 2 CK would acquire more importance; therefore, it was perceived that the emphasis and anxiety had merely been shifted from Step 1 to Step 2 CK.
Exam-related anxiety is likely only to increase, as candidates now only have one chance to obtain a top score; this change has also removed the chance to demonstrate an improvement in scoring from Step 1 to Step 2 CK. A shift to a greater emphasis on performing well on Step 2 CK, which is taken later in medical school, has been hypothesized to adversely impact US-MD and US-DO performance in clinical rotations . Importantly, given that IMGs have historically relied on high Step 1 scores for demonstrating their competitiveness in the residency match, the potential impact of this change cannot be overstated.
|Authors, year||Stakeholders||Journal name||Title of work|
|Makhoul et al , 2020||Program directors of all specialties||New England Journal of Medicine||Objective measures needed—program directors’ perspectives on a pass/fail USMLEa Step 1|
|Mun et al , 2021||Program directors in internal medicine and orthopedics||BMC Medical Education||A comparison of orthopaedic surgery and internal medicine perceptions of USMLE Step 1 pass/fail scoring|
|Mun et al , 2021||Program directors in internal medicine||Medicine||Internal medicine residency program director perceptions of USMLE Step 1 pass/fail scoring: a cross-sectional survey|
|Ehrlich et al , 2021||US medical students||The American Surgeon||Implications of the United States Medical Licensing Examination Step 1 examination transition to pass/fail on medical students education and future career opportunities|
|Cangialosi et al , 2021||US medical students: perspective||Academic Medicine||Medical students’ reflections on the recent changes to the USMLE step exams|
|Gu et al , 2021||Program directors in orthopedics||Journal of the American Academy of Orthopedic Surgeons Global Research and Reviews||Effect of change in USMLE Step 1 grading on orthopaedic surgery applicants: a survey of orthopaedic surgery residency program directors|
|Asaad et al , 2021||Program directors in plastic surgery||Journal of Surgical Education||Applicant familiarity becomes the most important evaluation factor in USMLE Step I conversion to pass/fail: a survey of plastic surgery program directors|
|Lin et al , 2020||Program directors in plastic surgery||Plastic and Reconstructive Surgery Global Open||Implications of pass/fail Step 1 scoring: plastic surgery program director and applicant perspective|
|MacKinnon et al , 2021)||Program directors in radiology||American Radiology||Pass/fail USMLE Step 1 scoring—a radiology program director survey|
|Warren et al , 2021||Medical Twitter||Academic Medicine||#MedEd Twitter response to the USMLE Step 1 pass/fail score reporting announcement|
|Snyder et al , 2021||Residency applicants for neurosurgery||Journal of Neurosurgery||Applying to residency: survey of neurosurgical residency applicants on virtual recruitment during COVID-19|
|Romano et al , 2021||Neurosurgery program directors, program chairs, and program administrators||Journal of Neurosurgery||Optimizing the residency application process: insights from neurological surgery during the pandemic virtual application cycle|
|Mamidi et al , 2021||Program directors in otolaryngology||Annals of Otology, Rhinology, and Laryngology||Perceived impact of USMLE Step 1 score reporting to pass/fail on otolaryngology applicant selection|
|Chator et al , 2021||Program directors in physical medicine and rehabilitation||American Journal of Physical Medicine and Rehabilitation||Physical medicine and rehabilitation program directors’ perspectives on US Medical Licensing Examination Step 1 scoring changes|
|Glassman et al , 2021||Program directors in emergency medicine||The Western Journal of Emergency Medicine||Emergency medicine program directors’ perspectives on changes to Step 1 scoring: does it help or hurt applicants?|
|Patrinely et al , 2021||Program directors in dermatology||Cutis||USMLE Step 1 changes: dermatology program director perspectives and implications|
|Chisholm and Drolet , 2020||Program directors in urology||Urology||USMLE Step 1 scoring changes and the urology residency application process: program directors’ perspectives|
|Odei et al , 2020||Program directors in radiation oncology||Advances in Radiation Oncology||Potential implications of the new USMLE Step 1 pass/fail format for diversity within radiation oncology|
|Pontell et al , 2020||Program directors in general surgery, integrated vascular, integrated thoracic, and integrated plastic surgery||Journal of Surgical Education||The change of USMLE Step 1 to pass/fail: perspectives of the surgery program director|
|Erath et al , 2020||Program directors in anesthesia||Anesthesia and Analgesia||Program directors’ response to a pass/fail US Medical Licensing Examination Step 1|
|Huq et al , 2020||Program directors in neurosurgery||Journal of Neurosurgery||Perceived impact of USMLE Step 1 pass/fail scoring change on neurosurgery: program director survey|
|Ganesh Kumar et al , 2020||Program directors in neurosurgery||World Neurosurgery||Characterizing the effect of pass/fail US Medical Licensing Examination Step 1 scoring in neurosurgery: program directors’ perspectives|
|Manstein et al , 2021||Medical school deans||Plastic Surgery (Oakville, Ontario)||The upcoming pass/fail USMLE Step 1 score reporting: an impact assessment from medical school deans|
|Aziz et al , 2021||Program directors in general surgery||World Journal of Surgery||Selecting the next generation of surgeons: general surgery program directors and coordinators perspective on USMLE changes and holistic approach|
|Goshtasbi et al , 2021||Program directors in otolaryngology||Laryngoscope||The effects of pass/fail USMLE Step 1 scoring on the otolaryngology residency application process|
|Whaley et al , 2021||Pathology: perspective||Academic Pathology||Changes in USMLE Step 1 result reporting: a pass or fail for pathology programs?|
|Fiedler , 2021||Cardiothoracic surgery: perspective||Seminars in Thoracic and Cardiovascular Surgery||Commentary: USMLE Step 1 pass/fail = win/win for cardiothoracic surgery trainee selection|
|Rajesh et al , 2021||Residents (surgery): perspective||Journal of Surgical Education||Binary reporting of USMLE Step 1 scores: resident perspectives|
|Aggarwal , 2020||International medical graduates||Academic Radiology||USMLE Step 1 reported as pass/fail: did international medical graduates need a reform?|
|Wallach et al , 2020||Residents (internal medicine): perspective||Journal of Community Hospital Internal Medicine Perspectives||Internal medicine resident perspectives on scoring USMLE as pass/fail|
|Quesada et al , 2021||Otolaryngology residency applicants||OTO Open||Overemphasis of USMLE and its potential impact on diversity in otolaryngology|
|Ganesh Kumar et al , 2021||US medical students, residents||Journal of Graduate Medical Education||Comprehensive reform and greater equity in applying to residency-trainees\' mixed responses to a pass/fail USMLE Step 1|
|Choudhary et al , 2021||Program directors in internal medicine||Journal of General Internal Medicine||Impact of pass/fail USMLE Step 1 scoring on the internal medicine residency application process: a program director survey|
|Pascarella , 2020||Clerkship director (perspective)||JAMA Surgery||USMLE Step 1 scoring system change to pass/fail-perspective of a clerkship director|
|Girard et al , 2021||US medical students||Journal of Surgical Education||US medical student perspectives on the impact of a pass/fail USMLE Step 1|
|Belovich et al , 2021||International association of medical science educators||Medical Science Educator||USMLE Step 1 is going to pass/fail, now what do we do?|
|Boulet and Pinsky , 2020||International medical graduates||Academic Medicine||Reporting a pass/fail outcome for USMLE Step 1: consequences and challenges for international medical graduates|
aUSMLE: United States Medical Licensing Examination.
A focus on research productivity was already a prominent requirement for a successful match into competitive specialties . This may potentially further increase with the elimination of Step 1’s objective scoring. For IMGs in particular, this is anticipated to be a significant hurdle—medical student research opportunities remain abysmal in low- and lower-middle-income countries [ , ]. Even in institutions where research is encouraged, such as the authors’ medical schools, publishing is difficult with paywalls and publishing fees limiting integration into peer-reviewed indexed journals. In addition to research, an emphasis on letters of recommendation, Alpha Omega Alpha Honors Medical Society membership, and clerkship grades have been expected to become more pronounced in applications, particularly in competitive specialties. For example, according to a recent comparative study, orthopedics program directors were more likely to prioritize these factors when compared with internal medicine program directors [ ]. This represents another limitation for IMGs and students outside of institutions with faculty whose letters carry weight in decision-making processes.
Rotating at outside institutions and subsequently obtaining a letter of recommendation from the said institution’s program director was considered instrumental in receiving invitations to competitive specialties such as dermatology, neurosurgery, orthopedics, and plastic surgery. Concerningly, with the move to pass/fail reporting and completing away rotations, colloquially called “audition rotations,” may become important even for noncompetitive specialties . This may substantially increase the out-of-pocket costs for each medical student, further disadvantaging IMGs and financially less capable candidates [ ].
Approximately 57% of the program directors reported that they would consider medical school prestige while evaluating candidates . In the United States, Black medical schools and schools in Puerto Rico have historically produced the majority of African-American and Hispanic graduates; yet, these medical schools are rarely ranked highly [ ]. Socioeconomic status and race are linked [ ], and many of these disadvantaged students opt to attend more affordable institutions even if they are less prestigious. Thus, this scoring change could lead to a paradoxical worsening of the holistic review for these disadvantaged groups, leading to a further worsening of diversity across training programs [ ].
In addition, a survey of plastic surgery program directors reported that personal prior knowledge of the applicant was one of the most important factors in evaluation . This subjective metric of evaluation, often driven by multiple socioeconomic factors, may prove to be a less than ideal tool compared to objective measures, following the conversion of USMLE Step 1 to a pass/fail outcome. However, with the pressure to score well on standardized exams like USMLE Step 1 removed, or at the very least, delayed, to taking Step 2 CK, medical students may be able to pursue specialty interests via research early on, translating to better knowledge on clinical rotations and subsequent assessment metrics. They may be able to participate in more community activities and volunteering efforts. Additionally, it is possible that their mental health may improve, in the absence of a minimum score to aim for. Still, these perceived benefits should be contrasted with the aforementioned risks, as the net effect may still disadvantage underrepresented applicants as well as IMGs, particularly those aiming for competitive specialties [ ].
Through direct and indirect effects, the Step 1 pass/fail change may likely impact IMGs adversely, especially foreign medical graduates, and may decrease foreign medical graduate representation in US residency positions. IMGs fill a crucial gap in the US health care system, serving groups of all backgrounds and in underserved areas [, ]. IMGs constitute a significant proportion of the American physician workforce. In 2018, almost 25% of the residents and fellows were IMGs, even representing over 50% in some specialties [ ]. They have provided and will continue to provide significant contributions toward addressing the physician gap in the United States. In neurology, for example, the physician workforce gap is projected to increase by 18% by 2025 [ , , ]. Interestingly, after accounting for physician and practice characteristics, IMGs deliver medical care more often than US graduates for complex patients, with lower mortality rates for older Medicare patients, and reports indicate no differences in readmission rates while accounting for hospital indices, patient characteristics, and socioeconomic status [ ]. Given the high-quality care provided by IMGs and the dependence of the American health care system on IMG service for sustenance, the change of USMLE Step 1 to a pass/fail outcome has, thus yet, unclear but far-reaching consequences for IMGs and their matching into primary care specialties.
An important demographic to also consider includes DO candidates. Their match success rates, particularly in competitive specialties, have traditionally been far worse than their MD counterparts . A standardized DO candidate will write the COMLEX Levels 1, 2, and 3, typically taking USMLE Step 1 in tandem with COMLEX Level 1 for consideration in the residency match. In addition to the loss of the opportunity to becoming a more competitive applicant with a high USMLE Step 1 score, DO students may now need to prepare for USMLE Step 2 CK in tandem with COMLEX Level 2 following their clinical rotations. However, most osteopathic programs maintain a traditional curricular calendar with clinical rotations ending in June, thus leaving DO applicants without protected time to adequately prepare for USMLE Step 2 CK, COMLEX Level 2, and subinternships/away rotations, further exacerbating the residency match for osteopathic medical students [ , ].
Impact of the USMLE Step 1 Scoring Change on Educators
The impact of the Step 1 scoring change on educators, particularly program directors, will likely be multifaceted. Each year, candidacy to residency programs has steadily risen, with over 40,000 applicants in 2020 . Similarly, the number of applications submitted per applicant has increased, forcing program directors to use Step 1 scores as a screening tool. This is especially true for IMGs in internal medicine—the specialty taking the largest number of IMGs. In 2019, IMGs submitted an average of 98 applications [ ] compared to an average of 35 applications by US-MDs/DOs [ ], making Step 1 to be the one reliable metric for program directors to screen candidates. Considering this, only 15.3% of all program directors surveyed by Makhoul et al [ ] agreed with the USMLE Step 1 scoring change. In fact, the Association of Program Directors in Radiology announced their opposition to the USMLE Step 1 pass/fail format in August 2019 [ ]. Importantly, although the InCUS meeting was supposed to represent all stakeholders, it was reported that leaders from the Graduate Medical Education community felt underrepresented in this decision-making process [ ]. For educators, the increasing number of applications conflicts with the holistic application screening, leading to greater use of objective measures, with USMLE Step 2 CK likely becoming the preferred screening tool in lieu of Step 1 after the pass/fail change. Over 77% of the program directors indicated their belief that this change would make it more difficult to objectively compare candidates [ ]. In some specialties such as neurosurgery, Step 1 scores have been shown to correlate with neurosurgery board exam scores [ ], and similarly, in obstetrics and gynecology, USMLE performance was correlated with that of resident evaluation exams [ ]. In the otorhinolaryngology board exam [ ], underperforming (score<210) was linked to a higher chance of not passing board exams. Regardless of the debate surrounding their predictive utility [ ], underperforming in specialty boards incurs fines on programs; therefore, these potential correlations were valuable for program directors.
It remains to be seen how medical institutions will adapt their curricula to the USMLE Step 1 scoring change. US medical schools may change their calendar to allow students to take Step 2 CK earlier, with a clear advantage for candidates from programs with an accelerated preclinical curriculum. Some authors have pointed out that this change may allow medical schools more curricular flexibility and take courses on topics not related to Step 1 but those useful for medical practice . For many IMGs whose schools follow a 6-year schedule with inflexible preclinical curricula designed by national authorities in response to their national need, modifications in response to a US exam–related change are unlikely. One noteworthy concern for program directors is a decrease in the basic science knowledge, which forms the bulk of the Step 1 curriculum of medical graduates [ ]. For specialties like anesthesia [ ], which utilize conceptual frameworks heavily from basic sciences, this unintended consequence could have potential far-reaching, but currently little understood, impact.
After the scoring conversion, it is anticipated that program directors may now have to more closely look at Medical Student Performance Evaluations (MSPEs) or dean’s letters. Medical schools in the United States have continued to move from a ranked or scored evaluation to a pass/fail curriculum or similar broad categories [, ]. Although dean’s letters are often lengthy and time-consuming to evaluate, they offer detailed insight into a candidate’s suitability for a particular residency position. However, because the evaluation criteria for international medical schools vary widely, MSPEs of IMGs have historically carried a significant degree of heterogeneity, with their distinguishing capability often questionable.
Taken together, the conversion of USMLE Step 1 from a 3-digit numeric score to reporting a pass/fail outcome alone may leave program directors with a challenging task for adequate and holistic, yet time-bound, evaluation of applicants. Efforts are being made through the introduction of Preference Signaling and ERAS Supplemental Application in the residency match to provide for a more holistic review and to ensure a better match between programs and applicants. Odei and colleagues  suggested the consideration of 7 components for residency candidates: research achievements, academic scores, commitment to the field, demonstrated compassion, demonstrated leadership, interpersonal skills, and diversity of life experiences [ ]. Similarly, Makhoul et al [ ] suggested a composite score consisting of shelf exam results in the major clinical subjects as an objective measure [ ]—this may offset the bias toward Step 2 CK [ ].
Recommendations for Residency Applicants
To further break down the path to a competitive application to any residency program, at the beginning of their medical school career, often referred to as the preclinical or preclerkship years, junior doctors should seek mentorship and advice regarding various avenues available prior to residency application. Concurrently, they should seek shadowing and research opportunities with faculty members at their respective institutions, if possible, or at nearby medical programs if they do not have a home program [- ]. As with every field, attaining familiarity with faculty members in the desired discipline may facilitate opportunities for increased success, which may be reflected through research (published abstracts, peer-reviewed manuscripts, textbook chapters, etc), strong letters of recommendation, additional biomedical honors (eg, research paper prizes), time devoted to specialty (summer research, research electives, away rotations in the specialty, etc), and attendance at key networking events (conferences, continued medical education accredited events, grand rounds, etc). Utilizing these opportunities may help applicants aiming for competitive residency programs. Additionally, given the increasing conversion of standardized national and international examinations to pass/fail, medical students should ensure securing the highest marks in every facet of their application that still provides scores or grades, such as preclinical exams, clerkships, or subinternships, COMLEX Level 2, and USMLE Step 2 CK. Importantly, securing protected research time becomes paramount to differentiate one’s application for residency, and medical students, including IMGs, considering a competitive match ought to consider taking one or more years dedicated solely to increasing their research productivity [ ]. With regard to research productivity, in recent years, especially for competitive specialties, the average number of research experiences has increased, with some using the term “arms race” to describe this [ ]. With the Step 1 scoring change, such experiences may only acquire potentially heightened importance. This is especially true for medical students from institutions known to have prolific research output—programs may have heightened expectations [ , ]. Of note, taking time out of clinical occupation for research may necessitate a serious commitment to readjusting to the demands of a clinical medical curriculum to maintain high academic marks, and students must perform effective cost-benefit analyses before every decision. Still, the combination of a stellar academic record, outstanding letters of recommendation, effective networking, and demonstrated interest in research may be more than sufficient for obtaining a competitive residency position. We have summarized some key official resources that applicants may refer to in [ , , - ].
|National Residency Matching Program|
|Main residency match data and reports: 2022 ||A detailed report of characteristics of matched and unmatched applicants, allowing students to get a rough idea of what they need to do to enroll into their specialty of choice|
|Charting outcomes for the match: international medical graduates, 2020 and 2022 [, ]||Data specific to international medical graduates|
|Interactive charting outcomes for the match ||Granular database of individualized charting outcomes, which permits candidates to assess their chances overall by inputting their personal attributes|
|United States Medical Licensing Examination|
|Summary report and preliminary recommendations from the Invitational Conference on United States Medical Licensing Examination Scoring, March 11-12, 2019 ||A detailed assessment of the rationale and process behind the scoring change. The website also provides a list of references with a summary of the papers cited.|
|United States Medical Licensing Examination Step 1, frequently asked questions ||Frequently asked questions regarding the USMLEa|
aUSMLE: United States Medical Licensing Examination.
Given the unique history of USMLE Step 1 in the US residency selection process and the score’s correlation with future performance in specialty board–certifying examinations, this scoring change is predicted to significantly impact all stakeholders involved in residency selection. Empirical data on the impact of this change will likely only be available from the literature emerging after the conclusion of Match 2023 and potentially even Match 2024. However, some comprehension may be reached from reviewing the surveys and perspectives coauthored by applicants, program directors, leadership of professional organizations, among others. For aspiring physicians pursuing a US residency, considering the progressive conversion of both medical school and national examinations from a scored outcome to pass/fail, the focus should be made on building a holistic application for the specialty of choice. Candidates aiming to secure competitive residency positions may take additional steps, including, but not limited to, engaging in specialty-specific research opportunities, networking with candidates at every stage of their medical careers, and becoming involved in organized groups around the world.
AO conceptualized, drafted, edited, and revised the manuscript. VB conceptualized, drafted, edited, and revised the manuscript and corresponded with the journal. DD revised the manuscript. All authors approved this publication.
Conflicts of Interest
- United States Medical Licensing Examination. URL: https://web.archive.org/web/20220107142845/https://www.usmle.org/ [accessed 2022-10-10]
- Summary report and preliminary recommendations. Invitational Conference on USMLE Scoring (InCUS). URL: https://web.archive.org/web/20220923083920/https://www.usmle.org/sites/default/files/2021-08/incus_summary_report.pdf [accessed 2022-10-10]
- Change to pass/fail score reporting for Step 1. United States Medical Licensing Examination. URL: https://web.archive.org/web/20201112013817/https://www.usmle.org/incus/ [accessed 2022-10-10]
- Ganesh Kumar N, Makhoul AT, Pontell ME, Drolet BC. Characterizing the Effect of Pass/Fail U.S. Medical Licensing Examination Step 1 Scoring in Neurosurgery: Program Directors' Perspectives. World Neurosurg 2020 Oct;142:e440-e444. [CrossRef] [Medline]
- Medical Council of Canada qualifying examination part I. Medical Council of Canada. URL: https://mcc.ca/examinations/mccqe-part-i/ [accessed 2022-10-10]
- NEET-PG 2022 examination information bulletin. National Board of Examinations in Medical Sciences. URL: https://web.archive.org/web/20220115043739/https://nbe.edu.in/IB/Information%20Bulletin%20NEET-PG%202022%20-%20Final%20Version%20for%20NBEMS%20Website.pdf [accessed 2022-10-10]
- USMLE score reporting policy updates. Educational Commission for Foreign Medical Graduates. URL: https://web.archive.org/web/20200815200515/https://www.ecfmg.org/news/2020/07/22/usmle-score-reporting-policy-updates/ [accessed 2022-10-10]
- COMLEX-USA level 1 to eliminate numeric scores. National Board of Osteopathic Medical Examiners. 2020 Dec 17. URL: https://web.archive.org/web/20201217194619/https://www.nbome.org/news/comlex-usa-level-1-to-eliminate-numeric-scores/ [accessed 2022-10-10]
- Results and data, 2020 main residency match. National Resident Matching Program. URL: https://www.nrmp.org/wp-content/uploads/2021/12/MM_Results_and-Data_2020-1.pdf [accessed 2022-10-10]
- Huq S, Khalafallah AM, Botros D, Jimenez AE, Lam S, Huang J, et al. Perceived impact of USMLE Step 1 pass/fail scoring change on neurosurgery: program director survey. J Neurosurg 2020 Jun 19:1-8. [CrossRef]
- Bhandarkar AR, Graffeo CS, Johnson J. Stepping Up: How U.S. Neurosurgery Training Programs Can Innovatively Assess Resident Applicants in a Post-Step 1 World. World Neurosurg 2020 Oct;142:291-293 [FREE Full text] [CrossRef] [Medline]
- Swanson DB, Sawhill A, Holtzman KZ, Bucak SD, Morrison C, Hurwitz S, et al. Relationship Between Performance on Part I of the American Board of Orthopaedic Surgery Certifying Examination and Scores on USMLE Steps 1 and 2. Acad Med 2009;84(Supplement):S21-S24. [CrossRef]
- Dougherty P, Walter N, Schilling P, Najibi S, Herkowitz H. Do scores of the USMLE Step 1 and OITE correlate with the ABOS Part I certifying examination?: a multicenter study. Clin Orthop Relat Res 2010 Oct;468(10):2797-2802 [FREE Full text] [CrossRef] [Medline]
- de Virgilio C, Yaghoubian A, Kaji A, Collins JC, Deveney K, Dolich M, et al. Predicting performance on the American Board of Surgery qualifying and certifying examinations: a multi-institutional study. Arch Surg 2010 Sep;145(9):852-856. [CrossRef] [Medline]
- Harmouche E, Goyal N, Pinawin A, Nagarwala J, Bhat R. USMLE Scores Predict Success in ABEM Initial Certification: A Multicenter Study. West J Emerg Med 2017 Apr;18(3):544-549 [FREE Full text] [CrossRef] [Medline]
- Filiberto AC, Cooper LA, Loftus TJ, Samant SS, Sarosi GA, Tan SA. Objective predictors of intern performance. BMC Med Educ 2021 Jan 26;21(1):77 [FREE Full text] [CrossRef] [Medline]
- Green M, Angoff N, Encandela J. Test anxiety and United States Medical Licensing Examination scores. Clin Teach 2016 Apr;13(2):142-146. [CrossRef] [Medline]
- Dyrbye LN, Burke SE, Hardeman RR, Herrin J, Wittlin NM, Yeazel M, et al. Association of Clinical Specialty With Symptoms of Burnout and Career Choice Regret Among US Resident Physicians. JAMA 2018 Sep 18;320(11):1114-1130 [FREE Full text] [CrossRef] [Medline]
- Ishak W, Nikravesh R, Lederer S, Perry R, Ogunyemi D, Bernstein C. Burnout in medical students: a systematic review. Clin Teach 2013 Aug;10(4):242-245. [CrossRef] [Medline]
- Del Carmen MG, Herman J, Rao S, Hidrue MK, Ting D, Lehrhoff SR, et al. Trends and Factors Associated With Physician Burnout at a Multispecialty Academic Faculty Practice Organization. JAMA Netw Open 2019 Mar 01;2(3):e190554 [FREE Full text] [CrossRef] [Medline]
- Prober CG, Kolars JC, First LR, Melnick DE. A Plea to Reassess the Role of United States Medical Licensing Examination Step 1 Scores in Residency Selection. Acad Med 2016;91(1):12-15. [CrossRef]
- Kauffman CA, Derazin M, Asmar A, Kibble JD. Patterns of medical student engagement in a second-year pathophysiology course: relationship to USMLE Step 1 performance. Adv Physiol Educ 2019 Dec 01;43(4):512-518 [FREE Full text] [CrossRef] [Medline]
- Chen DR, Priest KC, Batten JN, Fragoso LE, Reinfeld BI, Laitman BM. Student Perspectives on the “Step 1 Climate” in Preclinical Medical Education. Acad Med 2019;94(3):302-304. [CrossRef]
- Cuddy MM, Swanson DB, Clauser BE. A Multilevel Analysis of Examinee Gender and USMLE Step 1 Performance. Acad Med 2008;83(Supplement):S58-S62. [CrossRef]
- Rubright JD, Jodoin M, Barone MA. Examining Demographics, Prior Academic Performance, and United States Medical Licensing Examination Scores. Acad Med 2019 Mar 27;94(3):364-370. [CrossRef] [Medline]
- McDade W, Vela MB, Sánchez JP. Anticipating the Impact of the USMLE Step 1 Pass/Fail Scoring Decision on Underrepresented-in-Medicine Students. Acad Med 2020;95(9):1318-1321. [CrossRef]
- Makhoul AT, Pontell ME, Ganesh Kumar N, Drolet BC. Objective Measures Needed — Program Directors’ Perspectives on a Pass/Fail USMLE Step 1. N Engl J Med 2020 Jun 18;382(25):2389-2392. [CrossRef]
- Fincham JE. Response rates and responsiveness for surveys, standards, and the Journal. Am J Pharm Educ 2008 Apr 15;72(2):43 [FREE Full text] [CrossRef] [Medline]
- MacKinnon GE, Payne S, Drolet BC, Motuzas C. Pass/Fail USMLE Step 1 Scoring-A Radiology Program Director Survey. Acad Radiol 2021 Nov;28(11):1622-1625 [FREE Full text] [CrossRef] [Medline]
- Goshtasbi K, Abouzari M, Tjoa T, Malekzadeh S, Bhandarkar ND. The Effects of Pass/Fail USMLE Step 1 Scoring on the Otolaryngology Residency Application Process. Laryngoscope 2021 Mar;131(3):E738-E743 [FREE Full text] [CrossRef] [Medline]
- Markham TH, de Haan JB, Guzman-Reyes S, Brollier LD, Campbell AN, Pivalizza EG. Potential Harm of Elimination of Score Reporting for the United States Medical Licensing Examination Step 1 Examination to Anesthesiology Residency Selection. Anesth Analg 2021;132(1):275-279. [CrossRef]
- Mun F, Scott AR, Cui D, Lehman EB, Jeong S, Chisty A, et al. Correction to: A comparison of orthopaedic surgery and internal medicine perceptions of USMLE Step 1 pass/fail scoring. BMC Med Educ 2021 Oct 27;21(1):543 [FREE Full text] [CrossRef] [Medline]
- Mun F, Scott AR, Cui D, Chisty A, Hennrikus WL, Hennrikus EF. Internal medicine residency program director perceptions of USMLE Step 1 pass/fail scoring: A cross-sectional survey. Medicine (Baltimore) 2021;100(15):e25284. [CrossRef]
- Ehrlich H, Sutherland M, McKenney M, Elkbuli A. Implications of the United States Medical Licensing Examination Step 1 Examination Transition to Pass/Fail on Medical Students Education and Future Career Opportunities. Am Surg 2021 Aug;87(8):1196-1202. [CrossRef] [Medline]
- Cangialosi PT, Chung BC, Thielhelm TP, Camarda ND, Eiger DS. Medical Students' Reflections on the Recent Changes to the USMLE Step Exams. Acad Med 2021;96(3):343-348. [CrossRef]
- Gu A, Farrar J, Fassihi SC, Stake S, Ramamurti P, Wei C, et al. Effect of Change in USMLE Step 1 Grading on Orthopaedic Surgery Applicants: A Survey of Orthopaedic Surgery Residency Program Directors. JAAOS Glob Res Rev 2021 May 4;5(5):e20.00216. [CrossRef]
- Asaad M, Drolet BC, Janis JE, Giatsidis G. Applicant Familiarity Becomes Most Important Evaluation Factor in USMLE Step I Conversion to Pass/Fail: A Survey of Plastic Surgery Program Directors. J Surg Educ 2021;78(5):1406-1412. [CrossRef] [Medline]
- Lin L, Makhoul A, Hackenberger P, Ganesh Kumar N, Schoenbrunner AR, Pontell ME, et al. Implications of Pass/Fail Step 1 Scoring: Plastic Surgery Program Director and Applicant Perspective. Plast Reconstr Surg Glob Open 2020 Dec 17;8(12):e3266 [FREE Full text] [CrossRef] [Medline]
- Warren CJ, Fano AN, Wisener J, Davis M, Behbahani S, Sadeghi-Nejad H. #MedEd Twitter Response to the USMLE Step 1 Pass/Fail Score Reporting Announcement. Acad Med 2021 Feb 01;96(2):162. [CrossRef]
- Snyder M, Reddy V, Iyer A, Ganju A, Selden NR, Johnson JN, Society of Neurological Surgeons and American Association of Neurological Surgeons Young Neurosurgeons Committee. Applying to residency: survey of neurosurgical residency applicants on virtual recruitment during COVID-19. J Neurosurg 2021 Nov 26:1-10. [CrossRef] [Medline]
- Romano R, Mukherjee D, Michael L, Huang J, Snyder MH, Reddy VP, Society of Neurological Surgeons. Optimizing the residency application process: insights from neurological surgery during the pandemic virtual application cycle. J Neurosurg 2022 Jan 21:1-9. [CrossRef] [Medline]
- Mamidi IS, Gu A, Mulcahy CF, Wei C, Zapanta PE. Perceived Impact of USMLE Step 1 Score Reporting to Pass/Fail on Otolaryngology Applicant Selection. Ann Otol Rhinol Laryngol 2022 May;131(5):506-511. [CrossRef] [Medline]
- Chator AA, Ganesh Kumar N, Drolet BC, Sullivan WJ, Kennedy DJ. Physical Medicine and Rehabilitation Program Directors’ Perspectives on US Medical Licensing Examination Step 1 Scoring Changes. Am J Phys Med Rehabil 2021 Jan 26;100(12):1202-1205. [CrossRef]
- Glassman G, Black J, McCoin N, Drolet B. Emergency Medicine Program Directors' Perspectives on Changes to Step 1 Scoring: Does It Help or Hurt Applicants? West J Emerg Med 2021 Dec 20;23(1):15-19 [FREE Full text] [CrossRef] [Medline]
- Patrinely JJ, Zakria D, Drolet B. USMLE Step 1 Changes: Dermatology Program Director Perspectives and Implications. Cutis 2021 Jun;107(6):293-294. [CrossRef] [Medline]
- Chisholm LP, Drolet BC. USMLE Step 1 Scoring Changes and the Urology Residency Application Process: Program Directors' Perspectives. Urology 2020 Nov;145:79-82. [CrossRef] [Medline]
- Odei B, Das P, Pinnix C, Raval R, Holliday EB. Potential Implications of the New USMLE Step 1 Pass/Fail Format for Diversity Within Radiation Oncology. Adv Radiat Oncol 2021;6(1):100524 [FREE Full text] [CrossRef] [Medline]
- Pontell ME, Makhoul AT, Ganesh Kumar N, Drolet BC. The Change of USMLE Step 1 to Pass/Fail: Perspectives of the Surgery Program Director. J Surg Educ 2021;78(1):91-98 [FREE Full text] [CrossRef] [Medline]
- Erath A, Makhoul A, Drolet B. Program Directors' Response to a Pass/Fail US Medical Licensing Examination Step 1. Anesth Analg 2020 Oct;131(4):e186-e187. [CrossRef] [Medline]
- Manstein SM, Laikhter E, Kazei DD, Comer CD, Shiah E, Lin SJ. The Upcoming Pass/Fail USMLE Step 1 Score Reporting: An Impact Assessment From Medical School Deans. Plast Surg (Oakv) 2021 Oct 20:229255032110348. [CrossRef]
- Aziz H, Khan S, Rocque B, Javed MU, Sullivan ME, Cooper JT. Selecting the Next Generation of Surgeons: General Surgery Program Directors and Coordinators Perspective on USMLE Changes and Holistic Approach. World J Surg 2021 Nov;45(11):3258-3265 [FREE Full text] [CrossRef] [Medline]
- Whaley RD, Booth AL, Mirza KM. Changes in USMLE Step 1 Result Reporting: A Pass or Fail for Pathology Programs? Acad Pathol 2021;8:2374289521998029 [FREE Full text] [CrossRef] [Medline]
- Fiedler AG. Commentary: USMLE Step 1 Pass/Fail = Win/Win for Cardiothoracic Surgery Trainee Selection. Semin Thorac Cardiovasc Surg 2021;33(3):832-833. [CrossRef] [Medline]
- Rajesh A, Asaad M, Sridhar M. Binary Reporting of USMLE Step 1 Scores: Resident Perspectives. J Surg Educ 2021;78(1):304-307. [CrossRef] [Medline]
- Aggarwal V. USMLE Step 1 Reported as Pass/Fail: "Did International Medical Graduates Need a Reform?". Acad Radiol 2020 Nov;27(11):1653-1654. [CrossRef] [Medline]
- Wallach SL, Williams C, Chow RT, Jadhav N, Kuehl S, Raj JM, et al. Internal medicine resident perspectives on scoring USMLE as pass/fail. J Community Hosp Intern Med Perspect 2020 Sep 03;10(5):381-385 [FREE Full text] [CrossRef] [Medline]
- Quesada PR, Solis RN, Ojeaga M, Yang NT, Taylor SL, Diaz RC. Overemphasis of USMLE and Its Potential Impact on Diversity in Otolaryngology. OTO Open 2021 Jul 20;5(3):2473974X2110314. [CrossRef]
- Ganesh Kumar N, Pontell M, Makhoul A, Drolet B. Comprehensive Reform and Greater Equity in Applying to Residency-Trainees' Mixed Responses to a Pass/Fail USMLE Step 1. J Grad Med Educ 2021 Oct;13(5):711-716 [FREE Full text] [CrossRef] [Medline]
- Choudhary A, Makhoul AT, Ganesh Kumar N, Drolet BC. Impact of Pass/Fail USMLE Step 1 Scoring on the Internal Medicine Residency Application Process: a Program Director Survey. J Gen Intern Med 2021 Aug;36(8):2509-2510 [FREE Full text] [CrossRef] [Medline]
- Pascarella L. USLME Step 1 Scoring System Change to Pass/Fail-Perspective of a Clerkship Director. JAMA Surg 2020 Dec 01;155(12):1096-1098. [CrossRef] [Medline]
- Girard AO, Qiu C, Lake IV, Chen J, Lopez CD, Yang R. US Medical Student Perspectives on the Impact of a Pass/Fail USMLE Step 1. J Surg Educ 2022;79(2):397-408. [CrossRef] [Medline]
- Belovich AN, Bahner I, Bonaminio G, Brenneman A, Brooks WS, Chinn C, et al. USMLE Step-1 is Going to Pass/Fail, Now What Do We Do? Med Sci Educ 2021 Aug;31(4):1551-1556 [FREE Full text] [CrossRef] [Medline]
- Boulet J, Pinsky W. Reporting a Pass/Fail Outcome for USMLE Step 1: Consequences and Challenges for International Medical Graduates. Acad Med 2020 Sep;95(9):1322-1324. [CrossRef] [Medline]
- Wadhwa H, Shah S, Shan J, Cheng J, Beniwal AS, Chen J, et al. The neurosurgery applicant's "arms race": analysis of medical student publication in the Neurosurgery Residency Match. J Neurosurg 2019 Nov 01;1:1-9. [CrossRef] [Medline]
- Siddaiah-Subramanya M, Singh H, Tiang KW. Research during medical school: is it particularly difficult in developing countries compared to developed countries? Adv Med Educ Pract 2017 Nov;8:771-776 [FREE Full text] [CrossRef] [Medline]
- Garg R, Goyal S, Singh K. Lack of Research Amongst Undergraduate Medical Students in India: It's time to Act and Act Now. Indian Pediatr 2017 May 15;54(5):357-360 [FREE Full text] [CrossRef] [Medline]
- Kremer TR, Kremer MJ, Kremer KP, Mihalic A. Predictors of getting a residency interview: Differences by medical specialty. Med Educ 2021 Feb;55(2):198-212. [CrossRef] [Medline]
- Pisano C, Riaz H. Denying International Medical Graduates Entry to the United States: A Loss at Both Ends. Am J Med 2017 Aug;130(8):878-879. [CrossRef] [Medline]
- Mullan F, Chen C, Petterson S, Kolsky G, Spagnola M. The social mission of medical education: ranking the schools. Ann Intern Med 2010 Jun 15;152(12):804-811 [FREE Full text] [CrossRef] [Medline]
- Williams DR, Priest N, Anderson NB. Understanding associations among race, socioeconomic status, and health: Patterns and prospects. Health Psychol 2016 Apr;35(4):407-411 [FREE Full text] [CrossRef] [Medline]
- Desai A, Hegde A, Das D. Change in Reporting of USMLE Step 1 Scores and Potential Implications for International Medical Graduates. JAMA 2020 May 26;323(20):2015-2016. [CrossRef] [Medline]
- Pinsky WW. The Importance of International Medical Graduates in the United States. Ann Intern Med 2017 Mar 07;166(11):840. [CrossRef]
- Milewicz D, Lorenz R, Dermody T, Brass LF, National Association of MD-PhD Programs Executive Committee. Rescuing the physician-scientist workforce: the time for action is now. J Clin Invest 2015 Oct 01;125(10):3742-3747 [FREE Full text] [CrossRef] [Medline]
- 2020 physician specialty data report executive summary. Association of American Medical Colleges. URL: https://web.archive.org/web/20221010163056/https://www.aamc.org/media/50476/download?attachment) [accessed 2022-10-10]
- Dall TM, Storm MV, Chakrabarti R, Drogan O, Keran CM, Donofrio PD, et al. Supply and demand analysis of the current and future US neurology workforce. Neurology 2013 Apr 17;81(5):470-478. [CrossRef]
- The role of international medical graduates in the US physician workforce. American College of Physicians. URL: https://web.archive.org/web/20221210203202/https://www.acponline.org/acp_policy/policies/role_international_medical_graduates_2008.pdf [accessed 2022-10-10]
- Mahajan A, London Z, Southerland AM, Khan J, Schuyler EA. Immigrant Neurologists in the United States. Neurology 2020 Nov 16;96(8):378-385. [CrossRef]
- Craig E, Brotzman E, Farthing B, Giesey R, Lloyd J. Poor match rates of osteopathic applicants into ACGME dermatology and other competitive specialties. J Osteopath Med 2021 Mar 01;121(3):281-286 [FREE Full text] [CrossRef] [Medline]
- Ahmed H, Carmody JB. Double Jeopardy. Academic Medicine 2020;95(5):666. [CrossRef]
- Table C-3: residency applicants to ACGME-accredited programs by specialty and medical school type, 2022-2023. Association of American Medical Colleges. URL: https://web.archive.org/web/20221210204323/https://www.aamc.org/media/6181/download [accessed 2022-10-10]
- Rozenshtein A, Mullins ME, Marx MV. The USMLE Step 1 Pass/Fail Reporting Proposal: The APDR Position. Acad Radiol 2019 Oct;26(10):1400-1402. [CrossRef] [Medline]
- Willett LL. The Impact of a Pass/Fail Step 1 — A Residency Program Director’s View. N Engl J Med 2020 Jun 18;382(25):2387-2389. [CrossRef]
- Nagasawa DT, Beckett JS, Lagman C, Chung LK, Schmidt B, Safaee M, et al. United States Medical Licensing Examination Step 1 Scores Directly Correlate with American Board of Neurological Surgery Scores: A Single-Institution Experience. World Neurosurg 2017 Feb;98:427-431. [CrossRef] [Medline]
- Tamakuwala S, Dean J, Kramer KJ, Shafi A, Ottum S, George J, et al. Potential Impact of Pass/Fail Scores on USMLE Step 1: Predictors of Excellence in Obstetrics and Gynecology Residency Training. J Med Educ Curric Dev 2021;8:23821205211037444 [FREE Full text] [CrossRef] [Medline]
- Grading systems used in medical school programs. Association of American Medical Colleges. URL: https://web.archive.org/web/20221210204619/https://www.aamc.org/data-reports/curriculum-reports/interactive-data/grading-systems-used-medical-school-programs [accessed 2022-10-10]
- Crane MA, Chang HA, Azamfirei R. Medical Education Takes a Step in the Right Direction: Where Does That Leave Students? JAMA 2020 May 26;323(20):2013-2014. [CrossRef] [Medline]
- Kortz MW, Shlobin NA, Radwanski RE, Mureb M, DiGiorgio AM. Virtual Neurosurgery Education for Medical Students without Home Residency Programs: A Survey of 2020 Virtual Neurosurgery Training Camp Attendees. World Neurosurg 2022 Jan;157:e148-e155. [CrossRef] [Medline]
- Kortz M, McCray E, Lillehei K, DiGiorgio A. Letter: A Novel Neurosurgery Virtual Interest Group for Disadvantaged Medical Students: Lessons Learned for the Postpandemic Era. Neurosurgery 2021 Sep 15;89(4):E253-E254. [CrossRef] [Medline]
- Barrie U, Detchou D. In Reply: A Novel Neurosurgery Virtual Interest Group for Disadvantaged Medical Students: Lessons Learned for the Postpandemic Era. Neurosurgery 2022 Jul 01;91(1):e38-e39. [CrossRef] [Medline]
- Ozair A, Bhat V, Raju B, Nanda A. Letter to the Editor Regarding "Characterizing the Effect of Pass/Fail U.S. Medical Licensing Examination Step 1 Scoring in Neurosurgery: Program Directors' Perspectives". World Neurosurg 2021 Jun;150:232-233. [CrossRef] [Medline]
- Wilson C, Brown N, Detchou D. Letter to the Editor. USMLE examination and implications of a recent change. J Neurosurg 2021 Sep 24;136(1):316-317. [CrossRef] [Medline]
- Main residency match data and reports. National Resident Matching Program. URL: https://web.archive.org/web/20221210205114/https://www.nrmp.org/match-data-analytics/residency-data-reports/ [accessed 2022-10-10]
- Charting outcomes in the match: international medical graduates. National Resident Matching Program. URL: https://www.nrmp.org/wp-content/uploads/2021/08/Charting-Outcomes-in-the-Match-2020_IMG_final.pdf [accessed 2022-10-10]
- Interactive charting outcomes in the match. National Resident Matching Program. URL: https://www.nrmp.org/match-data-analytics/interactive-tools/charting-outcomes/ [accessed 2022-10-10]
- United States medical licensing examination step 1 common questions. United States Medical Licensing Examination. URL: https://www.usmle.org/common-questions/step-1 [accessed 2022-10-10]
|CK: clinical knowledge|
|COMLEX: Comprehensive Osteopathic Medical Licensing Examination|
|DO: Doctor of Osteopathic Medicine|
|ECFMG: Educational Commission for Foreign Medical Graduates|
|FSMB: Federation of State Medical Boards|
|IMG: international medical graduate|
|InCUS: Invitational Conference on United States Medical Licensing Examination Scoring|
|MCCQE: Medical Council of Canada Qualifying Examination|
|MD: Doctor of Medicine|
|MSPE: Medical Student Performance Evaluation|
|NBME: National Board of Medical Examiners|
|NEET-PG: National Eligibility cum Entrance Test for Post-Graduation|
|USMLE: United States Medical Licensing Examination|
Edited by T Leung; submitted 05.02.22; peer-reviewed by M Sotiropoulos, E Langenau, D Jeffe; comments to author 06.07.22; revised version received 11.10.22; accepted 29.11.22; published 06.01.23Copyright
©Ahmad Ozair, Vivek Bhat, Donald K E Detchou. Originally published in JMIR Medical Education (https://mededu.jmir.org), 06.01.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on https://mededu.jmir.org/, as well as this copyright and license information must be included.