THE NATIONAL GRADUATE SURVEY FOR FAMILY MEDICINE

The Association of Family Medicine Residency Directors (AFMRD) is excited to announce the rollout of a standardized national graduate survey. Beginning in 2016, the survey will be conducted through the American Board of Family Medicine (ABFM) Maintenance of Certification process. Understanding the scope of practice and success of family medicine residency graduates post-residency is a crucial step in improving residency education. The Accreditation Council of Graduate Medical Education (ACGME) Program Requirements for Graduate Medical Education in Family Medicine state that “Program graduates should be surveyed at least every 5 years, and the results should be used in the annual program evaluation” (V.C.6, Detail Requirement). Although some residencies collaborated on standardized surveys and data collection previously, no standardized national initiative was in place.

Recognizing an opportunity to both standardize the survey process and increase the response rate, the AFMRD approached the ABFM to explore opportunities to use ABFM-collected data to fulfill the graduate survey requirement. In turn, the ABFM offered to sponsor the development of a graduate survey to be administered through the Maintenance of Certification process, thus assuring a high response rate.

In cooperation with the ABFM, the AFMRD led a steering committee of representatives from AFMRD, ABFM, family medicine research organizations, the ACGME, and a new graduate. The steering committee identified the major stakeholders of a national graduate survey to be, in order of priority: (1) residencies for use in program evaluation and improvement and for milestone validation; (2) family medicine organizations regarding family medicine practice scope and characteristics, as well as the quality of and trends in medical education; and (3) the “public” for use in educational research and policy analysis. The steering committee recommended and then conducted a competitive request for proposal (RFP) process to select a survey development team, ultimately choosing a team from the University of Washington, headed by Dr Freddy Chen.

The survey development team completed a needs assessment and a literature search, reviewed previously developed graduate surveys, and conducted phone interviews of key stakeholders and recent graduates and roundtable discussions with program directors. In order to achieve a high survey completion rate, the goal was to limit the survey to questions that can be completed in fewer than 12 minutes. Two rounds of pilot testing have included input from stakeholders, including significant AFMRD input, to identify the most important questions to be included in the survey.

The survey is now being rolled out by the ABFM to all ABFM Diplomates 3 years after graduation. The ABFM will incorporate the survey into its Maintenance of Certification process, providing information to residency programs each year on the survey cohort. This meets the ACGME program requirement and provides valuable information to each program for continuous quality improvement of residency education. The ABFM will provide each program with its own confidential survey results, with individual responses de-identified. Individual program data will not be available to the ACGME for individual program accreditation. Broader graduate survey data will be available in aggregate form only, to assure the anonymity of information that may be sensitive for individual programs. A data set for research purposes will be available in de-identified form through the ABFM upon request. The ABFM and AFMRD are committed to the protection of individual data, while providing only aggregate data for national analysis.

The steering committee recommended using a 3-year post-graduation timeframe. Graduate scope of practice and success information is likely most useful to programs at that time, being neither too soon nor too long from the time of graduation. While the initial survey will include only residents 3 years after graduation, the ABFM intends to expand the survey so that each ABFM Diplomate completes a survey every 5 years, providing more robust information to programs.

Starting in 2016, the AFMRD plans to create an advisory group to work with the ABFM to annually monitor the performance of the survey, the usefulness of the data for residencies, and to determine if any changes are needed in the questions. The AFMRD urges all program directors to inform their residents and graduates of the importance of the national graduate survey and to encourage their participation; additionally the AFMRD encourages program directors to incorporate the results into their annual program evaluation.

• © 2015 Annals of Family Medicine, Inc.