Understanding and Using the ACGME Resident Surveys to Improve Your Residency Program

Since 2004, the Accreditation Council for Graduate Medical Education (ACGME) has administered an annual survey of all residents and fellows to monitor ongoing program quality and to serve as a possible indicator of the need to assess programs between site visits. With the introduction of the Next Accreditation System (NAS)¹ in 2013, the annual survey took on even greater significance as the frequency of scheduled program site visits lengthened to every 10 years. While specific questions are only visible to the survey participants, general content areas include clinical experience/education, faculty teaching and supervision, evaluation, educational content, diversity and inclusion, program resources, patient safety and teamwork, professionalism, and overall satisfaction.² In family medicine, additional questions are directed at continuity, care across settings, and family-centered care.

Release of the secure survey link typically occurs in February of each year with completion required in a 5-week window by a minimum of 70% of residents in order to see the aggregate results. An exception has occurred in the 2021 survey as the result of the pandemic, permitting 8 weeks for response. The program is responsible for distribution of the secure link to all participants and for ensuring completion, as the ACGME does not remind residents. Programs are prohibited from directing residents in their responses. The survey is considered complete when all questions are answered and the resident selects the “submit” button. Program results aggregated by domain are subsequently published as a summary in the ACGME ADS system with national specialty comparison data. At no time are individual respondents identified, nor are the questions made available to the program director, faculty, or program coordinator.³

Historically, concerns about differing interpretations of questions have been reported, as well as inconsistency in the directionality of the frequency norms used.⁴ Specific areas of concern include the balance between service and education, in-house call vs night float, and competition from other learners.⁵ So how should program directors balance the importance of resident autonomy and confidentiality with maximizing accuracy in their responses?⁶ Because of the high-stakes nature of the survey, program directors should ensure residents understand the questions and their context.

Programs can consider offering an internal survey well in advance of the ACGME survey to identify areas that may need clarification. Many interns taking the survey may be unaware of specific resources or may not recall items discussed during orientation. Programs often conduct a residency meeting that includes faculty to review program resources and policies, especially those related to the clinical learning environment, duty hours, diversity and inclusion, and resident wellness. Discussing these resources and policies can help remind residents of the opportunities within the program and the institution to express and resolve concerns.⁷ Programs should also review the response options for questions related to frequency and occurrences to reinforce a shared understanding of their definitions.⁸

Aggregate results are normally provided to program directors in May or June. It is expected that these results be reviewed as part of the Annual Program Evaluation, addressing any areas that are “negative outliers” compared to national averages. Also, programs should review the trends over the past few years to determine if they are positive or going in a downward direction. This is a time to identify areas of improvement that can be part of the next academic year’s strategic plan.⁹ Be sure to address these areas in the Annual ACGME ADS as areas being worked on for improvement. Multiple areas of “negative outliers”, especially if part of an ongoing trend, may trigger an ACGME site visit earlier than planned.

The annual ACGME resident survey is an important tool in assessing the residents’ perspective of the program and one of the elements contributing to accreditation status. Preparation, clarification of program attributes and policies, and access to peer support during survey administration can result in valid, actionable data which can be used to stimulate program quality improvement and resident satisfaction.

• © 2021 Annals of Family Medicine, Inc.