THE RESIDENCY PERFORMANCE INDEX (RPI): AN AFMRD TOOL FOR FAMILY MEDICINE RESIDENCY PROGRAM ASSESSMENT

The Residency Performance Index (RPI) was developed by the Association of Family Medicine Programs (AFMRD) in 2012 to spur residency program quality improvement, using program metrics and benchmark criteria specific to family medicine training.

RPI provides a “dashboard” for program directors, using criteria believed to be critical to program quality and yet measureable and/or published. Using concepts borrowed from AAFP Residency Program Solutions’ Criteria for Excellence and TransforMED MHIQ, the dashboard uses the convention of red, yellow, and green to indicate achievement of targets representing the floor, status quo, and excellence. Like the dashboard of your car, the intent of the RPI is to monitor the important functions of the program and alert the driver (program director/program evaluation committee) if maintenance is required.

The development of RPI was well timed, considering the ACGME’s emphasis on conducting meaningful quality self-assessment and improvement. The RPI can summarize much of the data used internally by a program’s program evaluation committee to conduct its annual program evaluation. Consecutive annual RPI reports tracking progression from deficiency (red) to excellence (green) can be useful trending information for the 10-year self-study process.

RPI is a powerful tool that can easily organize and communicate meaningful data. It can provide faculty and leadership with an at-a-glance view of current status and future needs, and convey the complicated nature of residency training and accreditation. The visual presentation and comparison to aggregate data is appealing to data-minded individuals (DIOs, CMOs, etc) and is consistent with current business practices within and outside health care. Programs could, for example, use “red” items to advocate for corrective resources from their departments and systems, similar to the silver lining of RC citations, but with no accreditation repercussions.

The RPI is available at no cost to AFMRD program directors. Those who use the RPI tool, including AFMRD itself, have a professional obligation to use it for self-improvement purposes only. Publication or comparison of individual RPI data to that of other programs or data sets is strictly prohibited. The tool must never be used as an advertising/promotional tool. It is also not an accrediting tool (no accrediting bodies, including the RC-FM have access to the data). In a world obsessed with rankings, it should be noted that RPI does not produce or promote a ranking system of any kind.

The AFMRD owns all RPI data and survey results and uses data only in an anonymous, aggregate form for the purpose of advancing the mission of the AFMRD. Aggregate data can be used as a self-improvement tool for the discipline itself by identifying gaps and potential trends in family medicine training. Once such improvement areas are identified, national organizations such as the AFMRD can:

• Tailor national education offerings to meet identified training and faculty development needs

• Focus advocacy efforts with accrediting bodies, such as the RC-FM and ABFM

• Focus on areas nationally that fall into yellow or red zones of metrics

• Use data to bring context to discussions of training guidelines and best practices

To our knowledge, this is the first US specialty-based comprehensive quality improvement tool for residency programs. The larger GME community has taken notice. The RPI is featured in the December 2014 issue of the Journal of Graduate Medical Education.¹ The article outlines the development, implementation, benefits and current challenges of the tool.

The future direction of RPI will address its recognized limitations, which include:

• Single specialty study, which reduces generalizability

• Volunteer participants that introduce the potential for selection bias

• Concerns about data collection, terminology of data, and keeping pace with ACGME

• Redundant data entry and timing of data collection

• Metrics and red/yellow/green levels set by consensus, expert opinion (lack of evidence for metrics)

RPI has been well accepted and shows promise as a self-improvement tool for both individual residency programs as well as the discipline of family medicine itself. It has already been utilized by 122 out of 480 residency programs. In order to realize the full benefits of the tool and rectify its limitations, the family medicine residency training community must embrace the tool and commit to accurate data entry and a higher participation rate.

• © 2015 Annals of Family Medicine, Inc.