All Quality Metrics are Wrong; Some Quality Metrics Could Become Useful

• clinician experience
• dynamics of complex systems
• performance-based reimbursement
• primary care
• quality metrics

The article by Brulin and Teoh in this issue of Annals of Family Medicine paints a bleak picture of clinicians’ experiences of performance-based reimbursement, seeing it as leading to illegitimate tasks and moral distress, with no payoff in clinician-reported quality in primary care.¹ This will ring true to many family physicians. What is mindboggling is how we got here. The first large-scale evaluation of quality metrics and “pay-for-performance” in primary care was in the National Health Service (NHS) in England and showed resoundingly mediocre results.^2,3 Despite this, pay-for-performance quality metrics have proliferated and become big business with support from numerous American governments. The question feels rhetorical, but really: who could be against pay-for-quality? Despite the seemingly obvious logic, clinicians clearly feel otherwise.

Dichotomous thinking of pay-for-performance as innately good or bad is overly simplistic. We’d encourage people to think of pay-for-performance quality metrics as a set of tools to potentially incentivize better care. The rosiest view of pay-for-performance quality metrics is that they will lead to the Triple Aim (reduced cost, improved population health, and better patient experience) or, even better, the Quadruple Aim (adding clinician satisfaction).⁴ The unfortunate reality is that in their present form, evaluations in primary care have not identified any remarkable outcomes.^2,3 Even in more isolated systems with a much higher likelihood of having sizable impacts on care (eg, for end-stage renal disease), results were underwhelming.⁵ In hospital medicine, tying reimbursement to rates of readmission has been found to have, at best, small effect.⁶

The reality is that quality metrics and pay-for-performance are much costlier than most patients, clinicians, or even administrators would think. The direct costs associated with a single hospital reporting quality metrics (not inclusive of changes in care delivery, insurance company, or governmental costs) was estimated to be around $6.1 million a year (2022 US dollars).⁷ Interestingly, the most expensive metrics in this study were claims-based, similar to the majority of primary care quality metrics. In contrast, the direct costs of electronic medical record–based metrics were far lower. An underappreciated aspect of quality metrics is that most practices/health care systems will have numerous payers to report to, each with different incentives and metrics. This complexity will increase reporting costs and likely blunt any incentive effect the metrics could have had. Additionally, care for different patients should look different, but creating risk adjustment is a substantial problem, given high-quality adjustment is expensive, challenging, and would presumably need to be unique for many metrics.

The featured article also addresses under-appreciated indirect consequences of pay-for-performance quality metrics, which is that they skew clinicians’ values. This article makes a strong case that clinicians do not view quality metrics as reflecting the quality of care that they provide. After a decade in practice, it’s pretty clear that the amount of recommended care goes far beyond what patients desire and what the system is capable of delivering.⁸ Hence, one of primary care’s important values to patients and medical care systems is prioritizing among all the available care. Quality metrics warp this facet of primary care. As an example, it is demoralizing beyond the time commitment to have a best-practice advisory advocate for and promote a discussion of low effect size cancer screening with a patient during a visit for an acute worsening of a chronic condition. Adding this to the already overwhelming administrative burden faced by clinicians is akin to adding straw to a camel’s already broken back.

Despite the creation of bureaucratic organizations and numerous National Academy of Medicine documents, it is worth reconsidering the entire pay-for-quality scheme in primary care. Reconceptualizing around known dynamics of complex systems is probably a good place to start. First, at best, one can achieve 2 out of 3 of the ideal characteristics of high quality, low cost, and timely information. At present, we would venture that most primary care clinicians would rate quality metrics as achieving between 0 and 1 out of 3. Second, a much greater appreciation for Goodhart’s Law—when a metric becomes a target, it ceases to be a good metric—would do a lot to improve the system. Additionally, in the unpredictable complex systems in which we work, much greater focus should be placed on high-quality evaluation of metrics before and following implementation (more cluster randomized controlled trials). If and when pay-for-quality metrics are found to have small influences on care, become obsolete, or detract from care; they should be promptly deimplemented. It is also important to highlight that the alternative to decreasing the importance of pay-for-performance should not be an abandonment of quality reporting, but the removal (or re-working) of incentives. We’d recommend refocusing incentives to those that are impactful, time limited, low cost, and physician controlled. Otherwise, our fear is that the pay-for-performance models will continue to add administrative work to primary care clinicians that will further overburden the system with administrative care and unrecognized costs that further degrade primary care’s value to the medical care system.

• Received for publication February 6, 2025.
• Accepted for publication February 10, 2025.

• © 2025 Annals of Family Medicine, Inc.