Clinical Concerns About Clinical Performance Measurement
========================================================

* Rachel M. Werner
* David A. Asch

## Abstract

Performance measurement has become one of the foundations of current efforts to improve health care quality and has successfully increased compliance with practice guidelines in many settings. Despite the successes of performance measurement, many physicians remain apprehensive about its use because performance measurement “gets in the way of” delivering good care. There are several reasons clinicians might feel this way. First, performance measurement is increasingly being extended to areas that have only a small clinical benefit and thus risk diverting attention from other more important but unmeasured aspects of care. Second, most performance measurement systems provide no priority for following guidelines likely to yield a large clinical benefit compared with guidelines likely to yield at best a small clinical benefit. Third, performance measures focus physicians’ attention narrowly on compliance with those measures rather than more broadly on the needs of the individual patient. Because performance measures are evaluated at the level of the indicator, they may crowd out quality at the level of the patient that is equally important but that cannot be easily measured. Performance measures play an important role in improving health care quality and will undoubtedly continue to do so; however, they are only one part of the solution to improving health care quality. Good performance is not necessarily good care, and pressure to improve performance can come at the sacrifice of good care. In its current state, performance measurement is better suited to improving measured care than improving the care of individual patients.

*   Health care quality, access, and evaluation
*   quality improvement
*   primary health care
*   patient-centered care

## INTRODUCTION

Performance measurement has become one of the foundations of current efforts to improve health care quality. The concept underlying this approach is that clinical performance will improve by setting standards for clinical processes or outcomes and measuring performance against those standards. The idea is borrowed from industrial approaches to quality improvement and has great intuitive appeal because it has the potential to link practice with evidence and standardize care.

There is compelling evidence that performance measurement leads to performance improvement. For example, in 1995 the US Department of Veterans Affairs began measuring and providing feedback on performance along selected clinical indicators in screening, prevention, and treatment of common disorders.1 Pneumococcal and influenza vaccination rates more than doubled as a result, and substantial improvements were made in rates of appropriate diabetes management.2 As the successes of performance measurement have become clear, performance measures have proliferated and now cover a progressively larger portion of clinical care in many health care settings. Indeed, the potential for quality improvement is seen to be so great that measured performance is increasingly being used in the United States and elsewhere, not only to provide quality-of-care information to physicians and the public, but also to determine physicians’ compensation.3,4

Despite the successes of performance measurement, some physicians continue to express apprehension about these measures. Some concern is expected from any group being assessed, and many of these concerns have previously been well articulated.5–9 For example, there are areas of clinical care that lack evidence-based measures.9 When evidence is available, individual physicians often do not have adequate numbers of patients with those particular conditions to assess their performance accurately,10 and performance measurement may cause physicians to aim for target rates and discount patient preferences and clinical judgment.6

Even with increasing attention to the important limitations of performance measurement, clinicians continue to express concern that performance measurement gets in the way of delivering good care. We initially found these protests paradoxical—after all, how can a system of performance measurement reduce quality of care? After many conversations with physicians, we noted that clinicians believe performance measurement hinders health care delivery by focusing physicians’ attention narrowly on compliance with performance measures rather than on the needs of the individual patient. If an individual patient has needs that lie outside the measured conditions, performance measurement may poorly measure quality of care and may give physicians incentive to attend to measured medical problems rather than unmeasured ones.

Our objective in this essay is to help articulate the point of view that performance measurement—although well-meaning and an appropriate part of an overall program to improve quality—may inadvertently reduce some aspects of health care quality. We synthesized stories related to us by other physicians or from our own experiences with performance measurement. These stories do not represent a full or balanced view of experiences with performance measurement. Instead, they are meant to illustrate several related problems with these approaches, so that future performance measurement is better able to improve health care quality.

### Performance Measures Are Designed to Improve Compliance With Practice Guidelines

Mr Edwards, a 55-year-old man with a history of hypertension and diabetes, has no complaints, and his blood pressure and diabetes are well controlled on his current medications. As his physician, Dr Smith, is completing the visit, the software in Mr Edwards’ electronic medical record reminds her that Mr Edwards’ low-density lipoprotein cholesterol is higher than recommended for a patient with diabetes. Based on this reminder, Dr Smith recommends starting a medication to lower Mr Edwards’ cholesterol levels.

Patients often do not receive recommended care.11 Performance indicators are designed to act as a safety net for care that is missed—either immediately, in the form of reminders, or later in the form of measurement and feedback. Indeed, performance measurement has been shown to improve compliance with standards of care and disease-specific health outcomes, and in some cases they may improve overall survival.2,12,13 These successes, however, have also led to the expansion of performance measurement into areas where clinical benefit may be limited.

### Some Performance Measures Have Limited Clinical Benefit

Dr Green believes that adhering to practice guidelines will improve the quality of care he provides to his patients. He therefore routinely examines the feet of each of his diabetic patients, screening for patients who are at high risk for foot complications from loss of sensation or circulation or who have unrecognized visible lesions. After several months he has not yet found a patient with otherwise unrecognized foot ulcers or infections despite the time he has put into his screening efforts.

The idea behind examining diabetic patients’ feet is a good one: foot ulcerations cause substantial morbidity and lower-extremity amputations might be prevented through early identification of ulcers and their risk factors. In general, screening makes sense if it selects for high-risk patients who would not have been otherwise identified, and if intervening early improves outcomes. For some medical conditions, such as hypercholesterolemia, screening and treating patients improves outcomes; however, routine examination of diabetic patients’ feet may not. Foot examinations for ulcers may not reduce the incidence of foot complications.14,15 Larger studies might show a benefit to these practices, but any benefits are likely to be small.

Numerous interventions for which performance is measured have small magnitudes of effect, such as screening for alcohol abuse and depression, or counseling for obesity, nutrition, and exercise. The problem is not that these issues are unimportant or that screening and counseling in these areas are not beneficial to patients. The problem is that these interventions have small effects or that routine screening rarely yields positive cases.16

Small benefits are still benefits; however, these small benefits come at a price. Performance measures of interventions with small benefits are given the same weight as performance measures of interventions with large benefits. In addition, the small overall benefit of some measured interventions must be balanced against the potential for clinical reminders to divert attention from other more important but unmeasured aspects of care. In a busy patient visit, time taken to comply with these performance measures competes with time for other activities that may provide more value.

### Equal Priority Is Given to All Measures

Mr Walker, a 61-year-old man with mild hypercholesterolemia and diabetes, is new to Dr Jones’ practice. Dr Jones is careful to attend to recommended performance indicators, screening for depression, alcohol, tobacco, and drug use; counseling Mr Walker about pneumococcal vaccine, exercise, and diet; and performing the required screening and testing related to Mr Walker’s hypercholesterolemia and diabetes. She also recommends colorectal cancer screening, but Mr Walker is not interested. Dr Jones believes that if she had more time to counsel Mr Walker about the importance of colorectal screening, he might have agreed to it.

Dr Jones’ general diligence leads to some frustration. Had she concentrated her efforts on a few higher-benefit conditions, such as colorectal screening, she might have provided more valuable care to this patient. Most performance measurement systems provide no priority for following guidelines likely to yield a large clinical benefit compared with guidelines likely to yield, at best, a small clinical benefit. Priority is increasingly important as performance measures proliferate to cover more aspects of care but have smaller effects on patient outcomes.

### Measured Activities May Crowd Out Unmeasured Activities

Mr Frank has not visited his regular doctor, Dr Brown, in several years. Recently Mr Frank has noticed that he is getting pains in his legs when he walks up hills. Dr Brown structures the short visit with Mr Frank around his health maintenance, much of which has been missed since he was last in. Dr Brown runs through a lengthy list of performance measures that are undocumented in Mr Frank’s medical record and completes the necessary documentation. Before Mr Frank can communicate his complaint of leg cramping, the allotted time for the appointment is over and Dr Brown has left the examination room. Although Dr Brown asked Mr Frank to make a routine follow-up appointment with her on the way out, Mr Frank does not return to Dr Brown’s office.

There is little doubt that Mr Frank received poor-quality care. He did not have his concerns addressed and was dissatisfied with the encounter. Yet a review of Dr Brown’s performance measures may lead to a different conclusion. In a limited amount of time, Dr Brown managed to comply with a long list of health care indicators, performing screening and counseling designed to improve patients’ health and reduce adverse outcomes. She did not, however, address Mr Frank’s issues.

This example may seem exaggerated, but the underlying pressures are real. Performance measurement shifts the priority within a clinical encounter from the patient’s expressed needs to the physician’s needs. Even well-meaning and evidence-based performance measures can encourage this shift in priorities, as physicians devote their limited available time first to those activities that are measured.

Good performance measures encourage physicians to adhere to guidelines for which a strong evidence base establishes that improving practice will improve outcomes. These performance measures also take time away from other activities that may deserve higher priority for individual patients even though they remain unmeasured. Indeed, one limitation of performance measurement is that it measures quality of care at the level of the indicator, not at the level of the patient. The indicators for Mr Frank’s visit suggest that it was a productive and high-quality visit. At a patient level, Mr Frank’s dissatisfaction with the visit makes it clear that the indicators can be misleading in their report of quality. Because performance measures are evaluated at the level of the indicator, they may crowd out the delivery of high-quality care in areas that are not being measured. These pressures will increase as performance measurement is associated with stronger incentives, such as pay-for-performance compensation.

## IMPROVING PERFORMANCE MEASURES

Performance measurement undoubtedly improves care in some clinical areas and at a time when nearly one half of all Americans do not receive recommended care,11 we believe that performance measurement is part of the solution. As this approach expands to include measures in a larger set of clinical areas, however, increasingly it will be extended to areas where there is limited clinical value, and current problems with performance measurement will increase.

There are ways to improve performance measurement. First, to be a good performance measure, the indicated condition must be widespread, events related to the condition must be common, and there must be a well-established evidence base suggesting that intervention in the indicated condition improves outcomes sufficiently to justify the overall effort. The number of clinical situations that satisfy these conditions is likely to be limited.

Second, each measured indicator should be weighted by its clinical value. Many physicians have incentives to comply equally with measures that provide a small patient benefit and measures that substantially improve health. As performance measures proliferate, setting priorities becomes even more important. One way to address this issue is with a weighting system. An example of a weighting system is the United Kingdom’s Quality and Outcomes Framework, which uses a point system that translates points into physician payment. A complex method of allocation makes points available based on the prevalence of the conditions covered by the Quality and Outcomes Framework and the workload required to provide care to the relevant standard.3 A weighting system should also reward physicians for giving relative priority to performance measures with a greater expected benefit for the patient. Although these weighting systems increase the complexity of performance measurement and implementation, they may provide the clinical face validity to help physicians accept what performance measurement can offer and help physicians give priority to areas most clinically beneficial.

Even so, limiting performance measures to good measures and implementing weighting systems will not overcome the fundamental difference between measuring and improving quality at the level of the indicator and measuring and improving quality at the level of the patient. Existing performance measurement systems are associated with improvements in quality along those measured indicators.2,12 Nevertheless, there is little consensus that these programs increase the quality of care more generally at the level of the patient.17 It is plausible that shifting clinical concentration to measured areas of care will crowd out activity in unmeasured areas that could have provided important clinical benefits, such as, for example, when Dr Brown does not attend to Mr Frank’s leg cramps.

It is possible that the care of Mr Frank’s leg cramps represents a clinical situation which ought to receive measurement priority. One solution is to include performance measures of patients’ global rating of their health care, such as patient satisfaction. These measures emphasize patients’ overall quality of care across diseases rather than disease-specific components of quality. Measures of interpersonal care have been extensively tested and validated18 and will soon be added to the list of measures the Centers for Medicare and Medicaid Services uses to assess quality of care at hospitals.19 Although patients’ ratings of their care certainly add an important dimension to performance measurement, such measures are not associated with technical quality of care,20 and patients may give more weight to interpersonal aspects of care than to technical quality of care when choosing clinicians.21 Clearly, satisfying patients’ immediate perceived needs is important, but so is meeting standards of technical quality of patient care11 and patient safety.22 The latter two are not easily evaluated by patients even in the setting of performance measurement, and the right balance between elements of care that are perceptible or imperceptible by patients will never be settled.

Another way to give measurement priority to overall quality of care is to change measurement methods. Measures might be more useful if they assess performance more broadly rather than assess discrete disease-specific elements of care delivery. Structured implicit review, a process whereby an expert reviewer judges the quality of care for an individual patient after reviewing the entire medical record,23 is one way to achieve broader performance assessment. A broader assessment would attempt to judge the quality of care at the level of an individual patient case. Such processes are labor intensive and costly, but they are reliable,24 have face validity, and can capture elements of care that are often the most important to patients.

Another solution is to narrow the role of performance measurement in quality improvement and maintain it within the broader set of activities that aim to improve health care. The complexity of health care makes it implausible that performance indicators could be developed for everything, or even that performance measurement alone can improve quality broadly. Rather, performance measurement should be seen as only one aspect of quality improvement,25 and perhaps a narrow one.

Performance measurement and pay for performance are well-intentioned and almost surely part of the right answer toward improving health care. They are easily oversold, however, and alone they risk directing our attention to what is measurable rather than what is important. Clinical performance is different from clinical care. Good performance is not necessarily good care, and pressure to improve performance can come at the sacrifice of good care. In its current state, performance measurement is better suited to improving measured care than improving the care of individual patients. So long as our focus remains on patients, our quality improvement efforts ought to be directed there.

## Acknowledgments

The authors thank Kenneth Frank, Alan Garber, Ross Gombiner, Joshua P. Metlay, and Sankey V. Williams for comments on earlier drafts of this manuscript.

## Footnotes

*   *Conflicts of interest: none reported* 

*   Received for publication June 1, 2006.
*   Revision received July 12, 2006.
*   Accepted for publication July 24, 2006.


*   © 2007 Annals of Family Medicine, Inc.

## REFERENCES

1.  Kizer KW. The “new VA”: a national laboratory for health care quality management. Am J Med Qual. 1999;14(1):3–20.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3Bham0iO3M6NToicmVzaWQiO3M6NjoiMTQvMS8zIjtzOjQ6ImF0b20iO3M6MjI6Ii9hbm5hbHNmbS81LzIvMTU5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

2.  Jha AK, Perlin JB, Kizer KW, Dudley RA. Effect of the transformation of the Veterans Affairs health care system on the quality of care. N Engl J Med. 2003;348(22):2218–2227.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMsa021899&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12773650&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000183134700009&link_type=ISI) 

3.  Roland M. Linking physicians’ pay to the quality of care—a major experiment in the United Kingdom. N Engl J Med. 2004;351(14):1448–1454.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMhpr041294&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15459308&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000224153900016&link_type=ISI) 

4.  Centers for Medicare and Medicaid. Medicare begins performance-based payments for physicians groups: new demonstration program tests 3 nancial incentives for improved quality and coordination in large group practices [press release]. Available at: [http://www.cms.hhs.gov/media/press/release.asp?Counter=1341](http://www.cms.hhs.gov/media/press/release.asp?Counter=1341). Accessed: 7 March 2005.
    
    

5.  Casalino LP. The unintended consequences of measuring quality on the quality of medical care. N Engl J Med. 1999;341(15):1147–1150.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJM199910073411511&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10511617&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000083087200011&link_type=ISI) 

6.  Walter LC, Davidowitz NP, Heineken PA, Covinsky KE. Pitfalls of converting practice guidelines into quality measures: lessons learned from a VA performance measure. JAMA. 2004;291(20):2466–2470.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.291.20.2466&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15161897&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000221599700029&link_type=ISI) 

7.  Epstein AM, Lee TH, Hamel MB. Paying physicians for high-quality care. N Engl J Med. 2004;350(4):406–410.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMsb035374&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=14736934&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000188254800015&link_type=ISI) 

8.  Lee TH, Meyer GS, Brennan TA. A middle ground on public accountability. N Engl J Med. 2004;350(23):2409–2412.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMsb041193&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15175444&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000221754200014&link_type=ISI) 

9.  Landon BE, Normand SL, Blumenthal D, Daley J. Physician clinical performance assessment: prospects and barriers. JAMA. 2003;290(9):1183–1189.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.290.9.1183&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12953001&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000185090400014&link_type=ISI) 

10. Hofer TP, Hayward RA, Greenfield S, Wagner EH, Kaplan SH, Manning WG. The unreliability of individual physician “report cards” for assessing the costs and quality of care of a chronic disease. JAMA. 1999;281(22):2098–2105.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.281.22.2098&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10367820&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000080668000032&link_type=ISI) 

11. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635–2645.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMsa022615&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12826639&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000183726400008&link_type=ISI) 

12. Asch SM, McGlynn EA, Hogan MM, et al. Comparison of quality of care for patients in the Veterans Health Administration and patients in a national sample. Ann Intern Med. 2004;141(12):938–945.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.7326/0003-4819-141-12-200412210-00010&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15611491&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000225923000005&link_type=ISI) 

13. Higashi T, Shekelle PG, Adams JL, et al. Quality of care is associated with survival in vulnerable older patients. Ann Intern Med. 2005;143(4):274–281.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.7326/0003-4819-143-4-200508160-00008&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16103471&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000231237100004&link_type=ISI) 

14. Mayfield JA, Reiber GE, Nelson RG, Greene T. Do foot examinations reduce the risk of diabetic amputation. J Fam Pract. 2000;49(6):499–504.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10923547&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000087652400001&link_type=ISI) 

15. Singh N, Armstrong DG, Lipsky BA. Preventing foot ulcers in patients with diabetes. JAMA. 2005;293(2):217–228.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.293.2.217&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15644549&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000226171000038&link_type=ISI) 

16. Desai MM, Rosenheck RA, Craig TJ. Screening for alcohol use disorders among medical outpatients: the in3 uence of individual and facility characteristics. Am J Psychiatry. 2005;162(8):1521–1526.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1176/appi.ajp.162.8.1521&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16055775&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 

17. Williams SV. Improving patient care can set your brain on 3 re. Ann Intern Med. 2005;143(4):305–306.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16103476&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 

18. Agency For Healthcare Research and Quality. CAHPS Surveys and Tools. Available at: [http://www.cahps-sun.org/home/index.asp](http://www.cahps-sun.org/home/index.asp). Accessed: 19 June 2006.
    
    

19. Centers for Medicare and Medicaid. Hospital Compare. Available at: [http://www.hospitalcompare.hhs.gov/](http://www.hospitalcompare.hhs.gov/). Accessed: 19 April 2005.
    
    

20. Chang JT, Hays RD, Shekelle PG, et al. Patients’ global ratings of their health care are not associated with the technical quality of their care. Ann Intern Med. 2006;144(9):665–672.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.7326/0003-4819-144-9-200605020-00010&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16670136&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000237293700006&link_type=ISI) 

21. Dafny LS, Dranove D. Do report cards tell consumers anything they don’t already know? The case of Medicare HMOs. NBER Working Paper Series #11420. 2005.
    
    

22. Kohn LT, Corrigan JM, Donaldson MS. T*o Err is Human: Building a Safer Health System.* Washington, DC: National Academies Press; 1999.
    
    

23. Kahn KL, Rubenstein LV, Sherwood MJ, Book RJ. *Structured Implicit Review for Physician Measurement of Quality of Care; Development of the Form and Guidelines for its Use.* Santa Monica, Calif: RAND Corp; 1989.
    
    

24. Hofer TP, Asch SM, Hayward RA, et al. Pro3 ling quality of care: is there a role for peer review? *BMC Health Serv Res*. 2004;4(9). Available at: [http://www.biomedcentral.com/1472-6963/4/9](http://www.biomedcentral.com/1472-6963/4/9). Accessed: 8 March 2005.
    
    

25. Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med. 1989;320(1):53–56.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJM198901053200110&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=2909878&link_type=MED&atom=%2Fannalsfm%2F5%2F2%2F159.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1989R576400010&link_type=ISI)