Modifying the Measurement Paradigm or Questioning its Very Assumptions

“It is impossible to escape the impression that people commonly use false standards of measurement—that they seek power, success and wealth for themselves and admire them in others, and that they underestimate what is of true value in life.”

Sigmund Freud, Civilization and Its Discontents¹

“The things we cannot measure may be the very things that will sustain us.”

Rachel Naomi Remen, The Wonder and the Mystery²

Reading an Annals article seldom is complete without scrolling to the TRACK section at the end to submit or read comments. Further, the entirety of the recent discussion across multiple articles can be accessed by clicking the TRACK Discussion button. This online discussion challenges, amplifies, and expands the points made by article authors, bringing together new information, experiences, and viewpoints.

The particular online discussion I want to highlight relates to a series of articles in the May/June issue in which both frontline clinicians and high-level health services evaluators bring to light possible unintended consequences of the ascendant measurement paradigm.

The idea that people on the front lines need to be driven to achieve quality has achieved dominance in the public and policy consciousness. Measuring quality, and using the result of that measurement to hold vanguard practitioners’ toes to the fire, affects nearly all groups and professions, but is particularly prevalent in health care and public education. This measurement paradigm implicitly says:

We don’t trust those who are doing the day-to-day work in caring for our families and educating our children. Their actions and their movements toward change must be regularly assessed. The practitioners must be motivated, rewarded, or punished on the basis of these measures. It is worth considerable expense to do this measuring/rewarding/punishing, even if it draws resources and attention away from doing the actual work.

The measurement paradigm is motivated by some very real concerns about the quality of __________ (fill in the blank). The pervasive idea that “you can’t improve what you don’t measure” is as true as the truism that, “not everything that can be counted counts, and not everything that counts can be counted.”³

The drive toward measurement and central control is part of a larger information age movement from small-scale implicit human systems toward larger-scale systems that require more explicit organization. In this change from getting things done informally among people who know each other well to getting things done formally among people who often are less closely connected, the new systems are often clunky. In order to make them work, we need the nuanced approach of people on the front line, who can personalize, prioritize, and integrate the changes and systems that top-down rules are trying to accomplish.

Into this environment, articles in the last issue of Annals^4–8 reveal some of the trade-offs in the current application of the measurement paradigm, and begin to suggest ways to proceed. An interesting back-and-forth discussion arises between measurers at the National Committee for Quality Assurance who are “pushing hard to deliver on your expectations for a more engaging and productive recognition program,”⁹ and who “recommend other broader incentive programs than that of a transaction-based system of payment for individual pieces of information,”¹⁰ and practitioners who seek “some long-overdue changes in how we engage in, demonstrate and improve quality in practice and practice measurement.”¹¹

Here is one of the online comments from this discussion, in its entirety¹²:

Question: Why Did They Stop Building Pyramids?

Pyramids are visually, structurally, and metaphorically appealing. Even our dollar bill has a pyramid on its back. And front-line doctors have a pyramid on their backs... let’s euphemistically call it a pyramid of accountability.

At the top of the accountability pyramid are, of course, business executives. Just five of whom representing just five health care corporations (United, Cigna, Davita, Anthem, Tenet) made off with 70 million this year. Near the top tier are the so-called certifying organizations. Two well-publicized examples just extracted about 100 million between them. The professional medical organizations and government-affiliated regulators, too numerous to count, also mine their share.¹³ Next the measurement industry. According to a recent Institute of Medicine (IOM) report the estimated 850 integrated health systems in the US spend about 3.5–12 million EACH for measurement.¹⁴ Finally, at the base, are physicians. Amongst them are the small-time doctors like Antonucci and Ho who ante up tens of thousands in time and money just to stay in the game though no one seems to care if they do. Little wonder that at a recent dinner party an executive admitted that she and some of her colleagues sometimes felt guilty about what was happening in health care... but how could they be expected to change when they were making so much money?

So what is to be done?

Why not make measures of accountability simpler? That is the strong message from the IOM. As an example, Ho and colleagues¹⁵ have shown how the waste inherent in vendor administration of multi-item “patient- experience” measures could be eliminated by a single measure obtained at the point of service. That change might save 300 million dollars.

But just think of all the jobs lost!

Others suggest that physicians at the base of the pyramid should unionize.¹⁶

That’s a possibility... but it didn’t work for the ancient Egyptians or Mayans.

We see on the economic landscape pyramids and pyramid schemes everywhere. Pyramids must be good business...

Question: Why did they stop building pyramids?

Answer: Because the money ran out.

I encourage you to view and contribute to the entire online discussion at http://www.AnnFamMed.org.

• © 2015 Annals of Family Medicine, Inc.