Review ArticleSystematic Review

Pay-for-Performance in the United Kingdom: Impact of the Quality and Outcomes Framework—A Systematic Review

Stephen J. Gillam, A. Niroshan Siriwardena and Nicholas Steel
The Annals of Family Medicine September 2012, 10 (5) 461-468; DOI: https://doi.org/10.1370/afm.1377

Abstract

PURPOSE Primary care practices in the United Kingdom have received substantial financial rewards for achieving standards set out in the Quality and Outcomes Framework since April 2004. This article reviews the growing evidence for the impact of the framework on the quality of primary medical care.

METHODS Five hundred seventy-five articles were identified by searching the MEDLINE, EMBASE, and PsycINFO databases, and from the reference lists of published reviews and articles. One hundred twenty-four relevant articles were assessed using a modified Downs and Black rating scale for 110 observational studies and a Critical Appraisal Skills Programme rating scale for 14 qualitative studies. Ninety-four studies were included in the review.

RESULTS Quality of care for incentivized conditions during the first year of the framework improved at a faster rate than the preintervention trend and subsequently returned to prior rates of improvement. There were modest cost-effective reductions in mortality and hospital admissions in some domains. Differences in performance narrowed in deprived areas compared with nondeprived areas. Achievement for conditions outside the framework was lower initially and has worsened in relative terms since inception. Some doctors reported improved data recording and teamwork, and nurses enhanced specialist skills. Both groups believed that the person-centeredness of consultations and continuity were negatively affected. Patients’ satisfaction with continuity declined, with little change in other domains of patient experience.

CONCLUSIONS Observed improvements in quality of care for chronic diseases in the framework were modest, and the impact on costs, professional behavior, and patient experience remains uncertain. Further research is needed into how to improve quality across different domains, while minimizing costs and any unintended adverse effects of payment for performance schemes. Health care organizations should remain cautious about the benefits of similar schemes.

INTRODUCTION

Strong primary care is widely accepted to be a prerequisite for effective, efficient, equitable health systems and to lead to better population health.1 Introduced in 2004, the UK Quality and Outcomes Framework (QOF) is arguably the most comprehensive national primary care pay-for-performance (P4P) scheme in the world.2 The QOF is more than a payment scheme; it is a complex intervention comprising a number of elements (Table 1), including financial incentives and information technology (computerized prompts and decision support), designed to promote structured and team-based care with the aim of achieving evidence-based quality targets.3

View this table:
Table 1

Domains of the Quality and Outcomes Framework

It was one component in the reorganization of primary care resulting from a new General Medical Services contract for general practitioners that led to a practice-based, rather than practitioner-based, contract and investment to reward quality of care through both fixed and performance-related funding streams. The financial incentives are substantial, with a maximum of 1,000 points available to practices, and an average payment per practice in 2011–2012 of £130 (US $204) for each point achieved.4 More than one-half of these points are allocated to clinical indicators, which currently cover 20 chronic conditions.5

In 2009–2010, practices in England achieved an average of 937 points, with a range in each of the 152 primary care trusts from 878 to 972 points.6

Since its introduction, the effects of the QOF on quality of care have been the subject of considerable debate, which is now being informed by an accumulating body of research. As national governments seek to improve the quality of health systems in the face of financial stringency, searching analysis of this evidence is timely. The successes and failures of the QOF as a national centralized system may be predictive of the effects of P4P in the United States.7

P4P schemes have been extensively reviewed.7–14 A recent Cochrane review found that, whereas they improved patients’ well-being, the effects of financial incentive schemes on the quality of primary health care were “modest and variable.”15 Previous reviews have focused on a single dimension of care or have had strict inclusion criteria with few articles retrieved or have not been conducted systematically. The huge variation in P4P schemes in different countries has made it difficult for reviewers to draw generalizable conclusions, whereas the uniform design of the QOF lends itself to close scrutiny. The research evidence about the QOF has grown rapidly and merits systematic review. We sought to examine the impact of the QOF on the quality of UK primary medical care, using broad inclusion criteria.

METHODS

We searched MEDLINE, EMBASE, and PsycINFO databases to identify all publications that sought to evaluate the QOF. The following search terms were used as free text in the title, abstract, or key words: (quality outcomes framework) OR (QOF) OR (pay for performance) OR (P4P) OR (pay-for-performance) AND (England) OR (Scotland) OR (Wales) OR (UK) OR (United Kingdom) OR (Great Britain). We limited the search to publications from January 1, 2004, to July 31, 2011, and limited the search to publications in the English language. We retrieved 575 references from MEDLINE (n = 348), EMBASE (n = 294), and PsycINFO (n = 55). We further searched by hand the reference lists of these articles for additional relevant studies. After eliminating duplicates and screening abstracts for relevance, 305 articles were excluded. The remaining 124 were read and rated by 2 authors (S.G. and N.S.) using a modified Downs and Black rating scale for observational studies (n = 110) and a Critical Appraisal Skills Programme rating scale for qualitative studies (n = 14). Thirty articles were excluded because they did not meet quality scores (less than 5 of 7 for observational or less than 7 of 10 for qualitative studies), did not evaluate the effect of the QOF, were a repeat publication, or were a review of previous research. Discrepancies in coding and areas of disagreement were resolved through discussion and adjudication by the third author (A.N.S.). The flow of information through the review is presented as a PRISMA flowchart (Figure 1).16

Figure 1
Figure 1

Flow of information through the review.

a Of the 575 articles, 348 were from MEDLINE, 294 were from EMBASE, and 55 were from PsycINFO, including duplicates.

Several typologies have been used to define quality in primary care,17–20 and the Institute of Medicine’s 6 dimensions of quality have been widely used: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity.20 We used these dimensions to categorize the included studies.

RESULTS

Our search and review retained 94 articles, which we have categorized into 4 areas: effectiveness, efficiency, equity, and patient experience. These correspond to dimensions of the Institute of Medicine’s definition,20 with the exception of safety and timeliness, for which we did not find any relevant studies. Several studies have been considered in more than one of these dimensions. Most of the qualitative studies examined professionals’ experience and team working and were considered separately as a fifth area.

Effectiveness

Supplemental Table 1 (47 studies) displays descriptions of the studies on the impact of the QOF on effective care (available at http://annfammed.org/content/10/5/461/suppl/DC1). The QOF has helped consolidate evidence-based methods for improving care by increasing the use of computers, decision support, clinician prompts, patient reminders, and recalls.21 It has resulted in better recorded care, enhanced processes, and improved intermediate outcomes for most conditions, notably diabetes. These improvements decreased after the first year of the QOF, however, and subsequent increases have followed secular trends. For example, Doran et al showed that achievement rates improved for most indicators in the preincentive period. There were significant increases in the rate of improvement in the first year of the incentive scheme (2004–2005) for 22 of the 23 incentivized indicators. These reached a plateau after 2004–2005, but quality of care in 2006–2007 remained higher than predicted by preincentive trends for 14 incentivized indicators. There was no overall effect on the rate of improvement for nonincentivized indicators in the first year of the scheme, but by 2006–2007 achievement rates were significantly below those predicted by preincentive trends.22 These improvements have been accompanied by rising prescription rates in associated drug categories.23 Performance improvements for those conditions that were not included in the QOF were significantly lower than for incentivized indicators, and these differences increased over time.22,24

Overall, modest population mortality reductions have been estimated.25 Fleetcroft et al modeled a potential saving of 11 lives per 100,000 people per year aggregated across all clinical indicators and domains in the first year of the contract, with no further gain in the second year as performance for a typical practice already exceeded the target payment levels.26

Efficiency

Supplemental Table 2 (5 studies) displays descriptions of the studies on the impact of the QOF on efficiency and costs (available at http://annfammed.org/content/10/5/461/suppl/DC1). There is limited evidence that increasing the quality of ambulatory care may reduce admission rates and hence costs for some conditions.25 For example, epilepsy care as incentivized by the QOF was associated with fewer epilepsy-related emergency admissions.27

There have been few attempts to model the cost effectiveness of QOF attainments.28,29 Walker et al could assess only a minority of indicators, and concluded that QOF incentive payments were cost-effective, even with only modest improvements in care, although they took no account of the costs of administering the QOF scheme. They found average indicator payments ranged from £0.63 to £40.61 per patient, and the percentage of eligible patients treated ranged from 63% to 90%. The improvements in performance required for QOF payments to be cost-effective varied by indicator from less than 1% to 20%. There was no relationship between the size of payments in a clinical domain and the likely resulting health gain.30,31

Equity

Supplemental Table 3 (25 studies) displays descriptions of the studies on the impact of the QOF on equity (available at http://annfammed.org/content/10/5/461/suppl/DC1). The QOF was not specifically designed to reduce health inequalities resulting from socioeconomic disadvantage. Nevertheless, inequalities in processes of care comparing the most and least deprived areas have narrowed. For example, Doran et found that the gap in median achievement comparing practices from the most deprived and least deprived quintiles narrowed from 4% to 0.8% between 2004 and 2007.32 In contrast, there have been variable effects on inequalities in care for long-term conditions based on age, sex, and ethnicity. Differences in care according to age for cardiovascular disease and diabetes narrowed after the introduction of the QOF as a result of greater improvement in those patients (usually older groups) receiving worse care. Disparities between men (who more often received better care) and women for cardiovascular disease and diabetes persisted or increased. Similarly, ethnic variations in quality of care have been reduced. For example, Millet et al found that improvements in blood pressure control were greater in the black group than the white, with disparities evident at baseline being attenuated (black 54.8% vs white 58.3% reaching target in 2005). Lower recording of blood pressure in the south Asian group, evident in 2003, was attenuated in 2005.33 The QOF has encouraged greater consistency of care irrespective of deprivation, but the practitioners’ option to exclude (exception report) hard-to-reach patients from the population used to determine payment may limit its impact on health inequalities.34

Patient Experience

Supplemental Table 4 (7 studies) displays descriptions of the studies on the impact of the QOF on patient experience (available at http://annfammed.org/content/10/5/461/suppl/DC1). There were no significant changes in quality of care reported by patients between 2003 and 2007 for communication, nursing care, coordination, or overall satisfaction.35

Continuity of care worsened for patients with chronic disease, and only access to urgent appointments improved significantly but modestly for these patients but not for adult patients more generally; overall, patients reported seeing their usual physician less often and gave lower satisfaction ratings for continuity of care.36 The few detailed ethnographic studies suggest that some practice teams have changed their consultations and clinical care in ways that may result in patients receiving a more biomedical type of care.37 There are also health professionals who acknowledge that an emphasis on protocol-driven care (“box-ticking”) may have distracted them from patient-led consultations and listening to patients’ concerns.38

Professionals and Team Working

Supplemental Table 5 (6 studies) displays descriptions of the studies on the impact of the QOF on professionals and team working (available at http://annfammed.org/content/10/5/461/suppl/DC1). Interviews with doctors and nurses suggest that the QOF has had positive effects on practice organization. For example, on team working and the diversification of nursing roles, both groups acknowledge that an enhanced role for nurses in managing long-term conditions could result in potential deskilling of doctors. They regret a decline in personal continuity of care between doctors and patients.39,40 The QOF appears to have introduced new hierarchies within practice teams and led to greater stratification of medical roles.41 Some team members resent not benefiting financially from payments, and there are concerns about an ongoing culture of performance monitoring in the United Kingdom.42 Quality of care may have become too narrowly focused on QOF domains and targets, with less regard to other areas for practice development, innovation, and quality improvement.43

DISCUSSION

There are inevitably conflicting findings from this large and diverse body of research, but some consistent themes have emerged. The implementation of the QOF has helped consolidate evidence-based methods.44 It has been associated with an increased rate of improvement of quality of care during the first year of implementation, returning to preintervention rates of improvement in subsequent years. There have been modest reductions in mortality and hospital admissions in some areas, and where they have been assessed, these modest improvements appear cost-effective. The QOF has led to narrowing of differences in performance in deprived areas compared with areas not deprived.45 It has strengthened team working.

The effect of the QOF in unincentivized areas has been disappointing. Prescription rates for antidepressants, statins, and other drugs have increased, but this increase is not clearly attributable to the QOF.46–48 The costs of administering the scheme are substantial, and some staff are concerned that primary care has become more biomedical in focus and less patient centered.

The QOF has strengthened team working and promoted a diversity of new roles, especially for nurses. Indeed, the QOF may have diminished the workload of general practitioners, enabled them to concentrate on more complex care, and led to teams in which work and knowledge is more distributed among its members.

The QOF has been described as scientific bureaucratic medicine, where indicators and guidelines are perceived as threatening professionalism in various ways.49 For better or worse, the QOF can be seen to have reduced clinical autonomy and provided performance data that can be used to compare clinicians nationally.

Remarkably little is known of what patients make of these changes, although anecdotal reports point to unintended consequences detracting from patient-centered care.50 The fear expressed by some that adherence to single disease-based guidelines might override respect for patient autonomy, lead clinicians to ignore comorbidities, promote a mechanistic approach to chronic disease management, or reduce clinical practice to a series of dichotomized decisions at the expense of personal aspects of care,51 has not been borne out by the research to date.

Strengths and Limitations

The great majority of studies used statistical analyses of trends or before and after comparisons, there being no possibility of controlled trials. The influence of many other regulatory, workforce-related, and educational changes on the quality of general practice is hard to disentangle. The development of evidence-based medicine, guidelines, and the introduction of other contractual incentives predated the QOF.

Some of the most intriguing findings, particularly concerning the culture of care and professional and patient experience, derived from the small number of qualitative studies. The incorporation of qualitative research into the conventional processes of systematic review presents epistemological and methodological challenges that are unresolved.52 Strengths of this review are that it has been conducted systematically, it includes more studies on the QOF than any previous review, and it considers a broad range of outcomes.

To what extent the apparent improvements in quality of care are the result of improved data entry remains unclear, but some of the documented improvement is likely to be due to recording of care previously delivered.53 Several factors impair the QOF’s impact at the population level. Setting targets below 100% and the process of exception reporting reduces the public health effectiveness of population targets by shifting the focus of the practice away from patients who are harder to reach.53 More fundamentally, payment for adhering to guidelines cannot be assumed to improve health status, regardless of whether it improves performance: improved processes (eg, treating hypertension) may not always translate to improved outcomes (eg, stroke prevention); and other powerful confounding influences affect outcomes, such as differential access to care, nonmodifiable risk factors (genetics, familial), or patterns of comorbidity. Process measures are often preferred for incentive schemes, as they are under the control of the health system and can be more efficient.7,54 The QOF’s evidence base will only ever be partial because its indicators by their very nature will focus on measuring the measurable.

Implications for Policy

The lessons that policy makers draw must, of course, take account of the different historical and organizational contexts in which their health system operates. A sensible verdict regarding the QOF’s effectiveness must balance a nuanced assessment of health and other gains against its costs, many of which are hard to describe, let alone quantify.

The system of P4P and how it is designed will affect how it professionals feel and behave. Family doctors in the United Kingdom, despite feeling that the QOF has changed the nature of the consultation, appear less negative toward P4P than doctors working in California, where lack of new funding, rewards directed at organizations rather than individual physicians, lack of identification with or ownership of measures, and thresholds rather than incremental levels of improvement have led to resentment and greater evidence of dysfunctional or coercive behavior toward patients regarded as noncompliant.55 It may be that the nature of medical practice is too complex to be improved by simple financial incentives.56

The limited evidence for cost-effectiveness and opportunity cost of the scheme is a central critique for the QOF’s detractors. If £1 billion a year of additional funding to general practice has yielded only modest improvements in measured quality of care, might greater benefits have been achieved if this investment had funded an alternative approach to quality improvement? The opportunity costs of the QOF are to a great extent unknown and unknowable, but the imperfect evidence available suggests that the same benefits could be maintained at reduced cost, particularly if systems are designed to involve clinicians and align with their values.57,58 More sophisticated modeling is required.

Developing the QOF

Although some have argued for discarding the QOF, it seems wiser to concentrate on addressing weaknesses rather than throwing away the gains. There is no reason why both technical aspects of quality and personal care cannot improve together.59 The involvement of the National Institute for Health and Clinical Excellence has greatly strengthened the QOF’s scientific underpinnings.60 There will always be a fine judgment about timing, level of evidence required, and whether to accept a consensus rather than evidence-based indicator. An argument for greater consistency of care should not prevail where evidence is lacking. The evidence base for existing indicators needs to be under constant review. Some indicators for which performance has reached a ceiling may need to be retired,61 although performance may not be maintained,62 and new indicators should be introduced after piloting.63

Gaming is known to occur in many systems that are driven by P4P; however, there has been little evidence of gaming in the QOF despite, or perhaps because of, a rigorous system of checks at various levels.64 On the contrary, practices are exceeding the upper payment thresholds and levels of exception reporting continue to fall year on year.65 Nevertheless, vigilant monitoring systems are needed. The balance of fixed vs performance-related funding should be reviewed. There is merit in linking the size of financial rewards to the public health impact of attaining individual indicators.66

In view of our findings that observed improvements in care from a very large payment-for-performance scheme in the United Kingdom were modest, with uncertainty about possible adverse effects, we recommend that policy makers continue to exercise caution about implementing similar schemes. Consideration should be given to improving different dimensions of quality, including user experience and equity. Costs should be monitored and balanced against benefits. Wherever possible, schemes should be designed in collaboration with health service researchers to evaluate the benefits of minor differences in system design. Payment for performance is still an imperfect approach to improving primary care, and should be considered as only one option alongside alternative quality improvement methods.

Future Research

The conclusions of this review are based on the available observational evidence, with all its limitations. They raise many questions about the design and implementation of payment for performance in health care. There is a clear need for more experimental research in many areas, and we suggest 5 high-priority areas. First, does the size of incentive payment affect achievement? Psychological research has surprisingly found that large incentives for tasks requiring greater levels of cognition may lead to lower levels of achievement, yet no research has been done on this finding in health care.56

Second, how can the patient-user experience be better assessed and more directly linked to the payment of financial incentives? Third, do incentives lead to a trade-off between technical and patient-centered dimensions of quality, or can they produce improvements across different dimensions of quality? Fourth, what effect do incentives have on such harder-to-measure outcomes as the interpersonal aspects of care and care for underserved populations? Fifth, what is the optimum time for a quality indicator to be included in a payment scheme before being reviewed or replaced by a different incentive?

Footnotes

  • Conflicts of interest: Drs Gillam and Steel are members of the Primary Care Quality and Outcomes Framework Advisory Committee, National Institute for Health and Clinical Excellence.

  • Received for publication December 17, 2010.
  • Revision received December 16, 2011.
  • Accepted for publication January 6, 2012.

References

  1. ↵
    1. Starfield B,
    2. Shi L,
    3. Macinko J
    . Contribution of primary care to health systems and health. Milbank Q. 2005;83(3):457–502.
    OpenUrlCrossRefPubMed
  2. ↵
    1. Roland M
    . Linking physicians’ pay to the quality of care—a major experiment in the United kingdom. N Engl J Med. 2004;351(14): 1448–1454.
    OpenUrlCrossRefPubMed
  3. ↵
    1. Siriwardena AN
    . Should the quality and outcomes framework be abolished? No. BMJ. 2010;340:c2794.
    OpenUrlFREE Full Text
  4. ↵
  5. ↵
    NHS Employers. Quality and Outcomes Framework guidance for GMS contract 2011/12. Delivering investment in general practice. NHS Employers. April 2011. http://www.nhsemployers.org/Aboutus/Publications/Documents/QOF_guidance_GMS_contract_2011_12.pdf.
  6. ↵
    The NHS Information Centre, Prescribing and Primary Care Services. Quality and Outcomes Framework Achievement Data 2009–2010. The NHS Information Centre for Health and Social Care; 2010. http://www.ic.nhs.uk/statistics-and-data-collections/audits-and-performance/the-quality-and-outcomes-framework.
  7. ↵
    1. Van Herck P,
    2. De Smedt D,
    3. Annemans L,
    4. Remmen R,
    5. Rosenthal MB,
    6. Sermeus W
    . Systematic review: Effects, design choices, and context of pay-for-performance in health care. BMC Health Serv Res. 2010;10:247.
    OpenUrlCrossRefPubMed
    1. Annemans L,
    2. Boeckxstaens P,
    3. Borgermans L,
    4. et al
    . Voordelen, nadelen en haalbaarheid van het invoeren van ‘Pay for Quality’ programma’s in België [Advantages, disadvantages and feasibility of the introduction of ‘Pay for Quality’ programmes in Belgium]. KCE report 118A & supplement 118S ed. 2009. Brussels, Federaal Kenniscentrum voor de Gezondheidszorg (Belgian Health Care Knowledge Centre). https://kce.fgov.be/sites/default/files/page_documents/d20091027353_1.pdf.
    1. Christianson JB,
    2. Leatherman S,
    3. Sutherland K
    . Financial Incentives, Healthcare Providers and Quality Improvements. A Review of the Evidence. London, The Health Foundation; 2007.
    1. Greene SE,
    2. Nash DB
    . Pay for performance: an overview of the literature. Am J Med Qual. 2009;24(2):140–163.
    OpenUrlFREE Full Text
    1. Steel N,
    2. Willems S
    . Research learning from the UK Quality and Outcomes Framework: a review of existing research. Qual Prim Care. 2010;18(2):117–125.
    OpenUrlPubMed
    1. Petersen LA,
    2. Woodard LD,
    3. Urech T,
    4. Daw C,
    5. Sookanan S
    . Does pay-for-performance improve the quality of health care? Ann Intern Med. 2006;145(4):265–272.
    OpenUrlCrossRefPubMed
    1. Town R,
    2. Kane R,
    3. Johnson P,
    4. Butler M
    . Economic incentives and physicians’ delivery of preventive care: a systematic review. Am J Prev Med. 2005;28(2):234–240.
    OpenUrlCrossRefPubMed
  14. ↵
    1. Armour BS,
    2. Pitts MM,
    3. Maclean R,
    4. et al
    . The effect of explicit financial incentives on physician behavior. Arch Intern Med. 2001;161(10): 1261–1266.
    OpenUrlCrossRefPubMed
  15. ↵
    1. Scott A,
    2. Sivey P,
    3. Ait Ouakrim D,
    4. et al
    . The effect of financial incentives on the quality of health care provided by primary care physicians. Cochrane Database Syst Rev. 2011;9:CD008451.
  16. ↵
    1. Moher D,
    2. Liberati A,
    3. Tetzlaff J,
    4. Altman DG
    PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.
    OpenUrlCrossRefPubMed
  17. ↵
    1. Donabedian A
    . Evaluating the quality of medical care. Milbank Mem Fund Q. 1966:44(3):Suppl:166–206.
    OpenUrlCrossRefPubMed
    1. Maxwell RJ
    . Quality assessment in health. Br Med J (Clin Res Ed). 1984;288(6428):1470–1472.
    OpenUrlFREE Full Text
    1. Campbell SM,
    2. Roland MO,
    3. Buetow SA
    . Defining quality of care. Soc Sci Med. 2000;51(11):1611–1625.
    OpenUrlCrossRefPubMed
  20. ↵
    Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.
  21. ↵
    1. Gillam S,
    2. Siriwardena AN
    1. Gillam S,
    2. Siriwardena AN
    . The quality and outcomes framework: triumph of evidence or tragedy for personal care? In: Gillam S, Siriwardena AN, eds. The Quality and Outcomes Framework: QOF Transforming General Practice. Oxford: Radcliffe Publishing Ltd; 2010: 156–166.
  22. ↵
    1. Doran T,
    2. Kontopantelis E,
    3. Valderas JM,
    4. et al
    . Effect of financial incentives on incentivised and non-incentivised clinical activities: longitudinal analysis of data from the UK Quality and Outcomes Framework. BMJ. 2011;342:d3590.
    OpenUrlAbstract/FREE Full Text
  23. ↵
    1. MacBride-Stewart SP,
    2. Elton R,
    3. Walley T
    . Do quality incentives change prescribing patterns in primary care? An observational study in Scotland. Fam Pract. 2008;25(1):27–32.
    OpenUrlAbstract/FREE Full Text
  24. ↵
    1. Gillam S,
    2. Siriwardena AN
    1. Steel N,
    2. Willems S
    . Learning from the QOF: a review of existing research. In: Gillam S, Siriwardena AN, eds. The Quality and Outcomes Framework: QOF Transforming General Practice. Oxford: Radcliffe Publishing Ltd; 2010;31–69.
  25. ↵
    1. Martin S,
    2. Smith PC,
    3. Dusheiko M,
    4. Gravelle H
    . Do Improvements in Primary Care Reduce Secondary Care Costs? London: The Health Foundation; 2010.
  26. ↵
    1. Fleetcroft R,
    2. Parekh-Bhurke S,
    3. Howe A,
    4. Cookson R,
    5. Swift L,
    6. Steel N
    . The UK pay-for-performance programme in primary care: estimation of population mortality reduction. Br J Gen Pract. 2010;60(578): e345–e352.
    OpenUrlAbstract/FREE Full Text
  27. ↵
    1. Shohet C,
    2. Yelloly J,
    3. Bingham P,
    4. Lyratzopoulos G
    . The association between the quality of epilepsy management in primary care, general practice population deprivation status and epilepsy-related emergency hospitalisations. Seizure. 2007;16(4):351–355.
    OpenUrlCrossRefPubMed
  28. ↵
    1. Fleetcroft R,
    2. Parekh-Bhurke S,
    3. Howe A,
    4. Cookson R,
    5. Swift L,
    6. Steel N
    . The UK pay-for-performance programme in primary care: estimation of population mortality reduction. Br J Gen Pract. 2010;60(578): e345–e352.
    OpenUrlAbstract/FREE Full Text
  29. ↵
    1. Walker S,
    2. Mason AR,
    3. Claxton K,
    4. et al
    . Value for money and the Quality and Outcomes Framework in primary care in the UK NHS. Br J Gen Pract. 2010;60(574):e213–e220.
    OpenUrlAbstract/FREE Full Text
  30. ↵
    1. Fleetcroft R,
    2. Parekh-Bhurke S,
    3. Howe A,
    4. Cookson R,
    5. Swift L,
    6. Steel N
    . The UK pay-for-performance programme in primary care: estimation of population mortality reduction. Br J Gen Pract. 2010;60(578): e345–e352.
    OpenUrlAbstract/FREE Full Text
  31. ↵
    1. Walker S,
    2. Mason AR,
    3. Claxton K,
    4. et al
    . Value for money and the Quality and Outcomes Framework in primary care in the UK NHS. Br J Gen Pract. 2010;60(574):e213–e220.
    OpenUrlAbstract/FREE Full Text
  32. ↵
    1. Doran T,
    2. Fullwood C,
    3. Kontopantelis E,
    4. Reeves D
    . Effect of financial incentives on inequalities in the delivery of primary clinical care in England: analysis of clinical activity indicators for the quality and outcomes framework. Lancet. 2008;372(9640):728–736.
    OpenUrlCrossRefPubMed
  33. ↵
    1. Millett C,
    2. Gray J,
    3. Wall M,
    4. Majeed A
    . Ethnic disparities in coronary heart disease management and pay for performance in the UK. J Gen Intern Med. 2009;24(1):8–13.
    OpenUrlCrossRefPubMed
  34. ↵
    1. Fleetcroft R,
    2. Steel N,
    3. Cookson R,
    4. Howe A
    . “Mind the gap!” Evaluation of the performance gap attributable to exception reporting and target thresholds in the new GMS contract: National database analysis. BMC Health Serv Res”. 2008;8:131.
    OpenUrlCrossRefPubMed
  35. ↵
    1. Campbell SM,
    2. Kontopantelis E,
    3. Reeves D,
    4. et al
    . Changes in patient experiences of primary care during health service reforms in England between 2003 and 2007. Ann Fam Med. 2010;8(6):499–506.
    OpenUrlAbstract/FREE Full Text
  36. ↵
    1. Campbell SM,
    2. Kontopantelis E,
    3. Reeves D,
    4. et al
    . Changes in patient experiences of primary care during health service reforms in England between 2003 and 2007. Ann Fam Med. 2010;8(6):499–506.
    OpenUrlAbstract/FREE Full Text
  37. ↵
    1. Grant S,
    2. Huby G,
    3. Watkins F,
    4. et al
    . The impact of pay-for-performance on professional boundaries in UK general practice: an ethnographic study. Sociol Health Illn. 2009;31(2):229–245.
    OpenUrlCrossRefPubMed
  38. ↵
    1. Maisey S,
    2. Steel N,
    3. Marsh R,
    4. Gillam S,
    5. Fleetcroft R,
    6. Howe A
    . Effects of payment for performance in primary care: qualitative interview study. J Health Serv Res Policy. 2008;13(3):133–139.
    OpenUrlAbstract/FREE Full Text
  39. ↵
    1. Campbell SM,
    2. McDonald R,
    3. Lester H
    . The experience of pay for performance in English family practice: a qualitative study. Ann Fam Med. 2008;6(3):228–234.
    OpenUrlAbstract/FREE Full Text
  40. ↵
    1. Maisey S,
    2. Steel N,
    3. Marsh R,
    4. Gillam S,
    5. Fleetcroft R,
    6. Howe A
    . Effects of payment for performance in primary care: qualitative interview study. J Health Serv Res Policy. 2008;13(3):133–139.
    OpenUrlAbstract/FREE Full Text
  41. ↵
    1. Checkland K,
    2. Harrison S
    . The impact of the Quality and Outcomes Framework on practice organisation and service delivery: summary of evidence from two qualitative studies. Qual Prim Care. 2010;18 (2):139–146.
    OpenUrlPubMed
  42. ↵
    1. Campbell SM,
    2. McDonald R,
    3. Lester H
    . The experience of pay for performance in English family practice: a qualitative study. Ann Fam Med. 2008;6(3):228–234.
    OpenUrlAbstract/FREE Full Text
  43. ↵
    1. Apekey TA,
    2. McSorley G,
    3. Tilling M,
    4. Siriwardena AN
    . Room for improvement? Leadership, innovation culture and uptake of quality improvement methods in general practice. J Eval Clin Pract. 2011;17 (2):311–318.
    OpenUrlCrossRefPubMed
  44. ↵
    1. Gillam S,
    2. Siriwardena AN
    . The Quality and Outcomes Framework: QOF Transforming General Practice. Oxford: Radcliffe Publishing Ltd; 2010.
  45. ↵
    1. Dixon A,
    2. Khachatryan A
    . A review of the public health impact of the Quality and Outcomes Framework. Qual Prim Care. 2010;18(2): 133–138.
    OpenUrlPubMed
  46. ↵
    1. Morrison J,
    2. Anderson MJ,
    3. Sutton M,
    4. et al
    . Factors influencing variation in prescribing of antidepressants by general practices in Scotland. Br J Gen Pract. 2009;59(559):e25–e31.
    OpenUrlAbstract/FREE Full Text
    1. Alabbadi I,
    2. Crealey G,
    3. Turner K,
    4. et al
    . Statin prescribing in Northern Ireland and England pre and post introduction of the quality and outcomes framework. Pharm World Sci. 2010;32(1):43–51.
    OpenUrlCrossRefPubMed
  48. ↵
    1. Cupples ME,
    2. Byrne MC,
    3. Smith SM,
    4. Leathem CS,
    5. Murphy AW
    . Secondary prevention of cardiovascular disease in different primary healthcare systems with and without pay-for-performance. Heart. 2008;94(12):1594–1600.
    OpenUrlAbstract/FREE Full Text
  49. ↵
    1. Checkland K,
    2. Harrison S,
    3. McDonald R,
    4. Grant S,
    5. Campbell S,
    6. Guthrie B
    . Biomedicine, holism and general medical practice: responses to the 2004 general practitioner contract. Sociol Health Illn. 2008;30 (5):788–803.
    OpenUrlCrossRefPubMed
  50. ↵
    1. Gillam S,
    2. Siriwardena AN
    1. Wilkie P
    . Does the patient always benefit? In: Gillam S, Siriwardena AN, eds. The Quality and Outcomes Framework: QOF Transforming General Practice. Oxford: Radcliffe Publishing Ltd; 2010:128–136.
  51. ↵
    1. Gillam S,
    2. Siriwardena AN
    1. Starfield B,
    2. Mangin D
    . An international perspective on the basis forf payment-for-performance. In: Gillam S, Siriwardena AN (eds). The Quality and Outcomes Framework: QOF Transforming General Practice. Oxford: Radcliffe Publishing Ltd; 2010:147–155.
  52. ↵
    1. Dixon-Woods M,
    2. Agarwal S,
    3. Jones D,
    4. Young B,
    5. Sutton A
    . Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 2005;10(1):45–53.
    OpenUrlAbstract/FREE Full Text
  53. ↵
    1. Gillam S,
    2. Siriwardena AN
    1. Kordowicz M,
    2. Ashworth M
    . ‘Smoke and mirrors’? Informatics opportunities and challenges. In: Gillam S, Siriwardena AN, eds. The Quality and Outcomes Framework: QOF Transforming General Practice. Oxford: Radcliffe Publishing Ltd; 2010:82–92.
  54. ↵
    1. Mant J,
    2. Hicks N
    . Detecting differences in quality of care: the sensitivity of measures of process and outcome in treating acute myocardial infarction. BMJ. 1995;311(7008):793–796.
    OpenUrlFREE Full Text
  55. ↵
    1. McDonald R,
    2. Roland M
    . Pay for performance in primary care in England and California: comparison of unintended consequences. Ann Fam Med. 2009;7(2):121–127.
    OpenUrlAbstract/FREE Full Text
  56. ↵
    1. Ariely D,
    2. Gneezy U,
    3. Loewenstein G,
    4. Mazar N
    . Large stakes and big mistakes. Rev Econ Stud. 2009;76(2):451–469.
    OpenUrlAbstract/FREE Full Text
  57. ↵
    1. McDonald R,
    2. Roland M
    . Pay for performance in primary care in England and California: comparison of unintended consequences. Ann Fam Med. 2009;7(2):121–127.
    OpenUrlAbstract/FREE Full Text
  58. ↵
    1. Marshall M,
    2. Harrison S
    . It’s about more than money: financial incentives and internal motivation. Qual Saf Health Care. 2005;14(1):4–5.
    OpenUrlFREE Full Text
  59. ↵
    1. Shekelle P
    . New contract for general practitioners. BMJ. 2003;326 (7387):457–458.
    OpenUrlFREE Full Text
  60. ↵
    1. Lester H,
    2. Roland M
    . Future of quality measurement. BMJ. 2007;335 (7630):1130–1131.
    OpenUrlFREE Full Text
  61. ↵
    1. Reeves D,
    2. Doran T,
    3. Valderas JM,
    4. et al
    . How to identify when a performance indicator has run its course. BMJ. 2010:340: c1717.
    OpenUrlFREE Full Text
  62. ↵
    1. Lester H,
    2. Schmittdiel J,
    3. Selby J,
    4. et al
    . The impact of removing financial incentives from clinical quality indicators: longitudinal analysis of four Kaiser Permanente indicators. BMJ. 2010:340: c1898.
    OpenUrlAbstract/FREE Full Text
  63. ↵
    1. Gillam S,
    2. Siriwardena AN
    1. Campbell S,
    2. Lester H
    . Developing indicators and the concept of QOF ability. In: Gillam S, Siriwardena AN, eds. The Quality and Outcomes Framework: QOF Transforming General Practice. Oxford: Radcliffe Publishing Ltd; 2010:16–30.
  64. ↵
    1. Gravelle H,
    2. Sutton M,
    3. Ma A
    . Doctor Behaviour Under a Pay-for-performance Contract: Evidence From the Quality and Outcomes Framework. CHE Research Paper 28. 2007. York: Centre for Health Economics; 2010.
  65. ↵
    1. Doran T,
    2. Fullwood C,
    3. Reeves D,
    4. Gravelle H,
    5. Roland M
    . Exclusion of patients from pay-for-performance targets by English physicians. N Engl J Med. 2008;359(3):274–284.
    OpenUrlCrossRefPubMed
  66. ↵
    1. Fleetcroft R,
    2. Parekh-Bhurke S,
    3. Howe A,
    4. Cookson R,
    5. Swift L,
    6. Steel N
    . The UK pay-for-performance programme in primary care: estimation of population mortality reduction. Br J Gen Pract. 2010;60(578): e345–e352.
    OpenUrlAbstract/FREE Full Text
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Get Permissions

Jump to section

Related Articles

Cited By...

More in this TOC Section

Similar Articles

Subjects