Abstract
PURPOSE Major primary care reforms have been introduced in recent years in the United Kingdom, including financial incentives to improve clinical quality and provide more rapid access to care. Little is known about the impact of these changes on patient experience. We examine patient reports of quality of care between 2003 and 2007, including random samples of patients on practice lists and patients with long-term conditions.
METHODS We conducted a cross-sectional design study of family practices in which questionnaires were sent to serial samples of patients in 42 representative general practices in England. Questionnaires sent to samples of patients with chronic disease (asthma, angina, and diabetes) and random samples of adult patients (excluding patients who reported any long-term condition) in 2003, 2005, and 2007 addressed issues of access, communication, continuity of care, coordination, nursing care, and overall satisfaction.
RESULTS There were no significant changes in quality of care reported by either group of patients between 2003 and 2007 for communication, nursing care, coordination, and overall satisfaction. Some aspects of access improved significantly for patients with chronic disease, but not for the random samples of patients. Patients in both samples reported seeing their usual physician less often and gave lower satisfaction ratings for continuity of care. Most scores were significantly higher for the chronic illness samples than for the random samples of patients in 2003, even after adjusting for age.
CONCLUSIONS There was a modest improvement in access to care for patients with chronic illness, but all patients now find it somewhat harder to obtain continuity of care. This outcome may be related to the incentives to provide rapid appointments or to the increased number of specialized clinics in primary care. The possibility of unintended effects needs to be considered when introducing pay for performance schemes.
INTRODUCTION
In the United Kingdom, a number of major reforms have been introduced in family practice in recent years. For example, the National Health Service (NHS) Plan in 2000 set access targets stipulating that people should be able to make an appointment with a physician at their own practice within 48 hours,1 and practices could volunteer to take part in schemes where these targets were incentivized.2 In 2004 the UK government introduced the Quality and Outcomes Framework (QOF), a pay-for-performance scheme, as part of new contractual arrangements for general practitioners.3 The scheme covered 3 key areas of quality: chronic disease management, practice organization, and patient experience. The 2004 General Medical Services Contract changed out-of-hours arrangements, with most general practitioners and practices opting out of 24-hour care responsibility. Practices were also encouraged to keep their practices open longer to provide extended appointments as part of the new contract. The most frequent changes reported in response to the new contract were offering more open or advanced-access slots4 and increasing staffing levels with nursing staff absorbing a higher proportion of the clinical workload.5
There is evidence that clinical quality was improving markedly before the introduction of the new contract and QOF in 20046 and has continued to improve subsequently.7,8 There are data on changes in patient-reported quality of care spanning the introduction of these changes in the organization and delivery of health care. We have reported previously that there were no significant changes in registered patients’ reports of access to care or interpersonal aspects of care, but that there had been a reduction in ratings of continuity of care in patients sampled from the general population.8 Indeed, a recent report by the National Audit Office reported that the current satisfaction of patients with family practice care remains in line with rates of satisfaction before the new arrangements.9 Previous reports, however, did not include other domains of patient experience, such as coordination or care or nursing care in the general population. More importantly, there have been no previous longitudinal reports of quality of care for patients with long-term medical problems.
Interpersonal or relational continuity of care and coordination of care are of importance to and a priority for patients generally,10,11 but they are particularly valued by vulnerable patients and those with long-term conditions.12,13 These patient groups are targeted by the clinical indicators in the QOF,15 and there has been concern that the clinical focus and targets of the pay-for-performance scheme might lead practitioners to neglect other aspects of care, such as access to care and continuity of care at the patient level, and communication skills of physicians.3,16–17 This article focuses on the reported experiences of patients with long-term conditions between 2003 and 2007, in the years after the introduction of QOF. We also compare these experiences with general population samples of registered patients across the same period.
METHODS
Participants
We collected data in a cohort sample of 42 family practices nationally representative for practice size, deprivation, and QOF achievement scores (Table 1⇓). Patients with confirmed diagnoses of chronic illness (coronary heart disease, diabetes, or asthma) were randomly sampled in each practice as part of separate clinical audits, which were conducted between February and August 2003, 2005, and 2007 respectively.7–8,18 Up to 12 patients with each condition were included per practice. In each year, all patients included in the audits were mailed a copy of the questionnaire, with 1 reminder in October-November 2003, 2005, and 2007, respectively.
In addition, in each practice a random sample of 200 adult patients (aged 18 years and older) were mailed a copy of the questionnaire, with 1 reminder, during May-June 2003, May-June 2005, and May-June 2007, respectively.
Survey Instrument
We used a version of the General Practice Assessment Questionnaire (GPAQ) (http://www.gpaq.info) for the patient evaluations in each period (2003, 2005, 2007) that contained the same items used in a previous quality assessment.19 All 33 items in the questionnaire reported in this article, however, were the same as those in published versions of the questionnaire.20,21 Items within GPAQ are used to create scale scores for patient assessments of (1) coordination of care within and outside practices, (2) the communication skills of the patient’s usual physician, (3) nursing care, and (4) overall satisfaction. In addition, individual items relate to continuity of care with a usual physician and access, assessed by patient reports about being able to get an appointment with any physician within 48 hours and a particular physician within 48 hours, as well as a question about being able to get an urgent appointment on the same day. We asked patients to rate these issues from “very poor”’ to “excellent.” The scales and items used are shown in the Appendix⇓.
Scores ranged from 0 to 100 for each of the GPAQ scales and individual rating items for the samples of patients with chronic illness and for the random samples of patients in 2003, 2005, and 2007, respectively (http://www.npcrdc.ac.uk/OtherPubs/GPAQ%20manual%20V2%201.doc). Dichotomous items were not rescaled. Patients in the random samples who self-reported a limited long-standing illness were excluded from the analyses of the random samples.
Data Analyses
Patient experience data were collected as part of a larger repeated cross-sectional design study of quality of care that began in 1998.19 When it became clear that a pay-for-performance scheme was going to be implemented in 2004, we designed an interrupted time series study whereby data on quality of care was collected at 2 points before (1998 and 2003) and 2 points after (2005 and 2007) its introduction. GPAQ data for a random sample of patients was collected at participating practices from 1998, but only from 2003 for specific chronic illness samples. To compare between samples, we used the data for both groups from 2003 onward.
In these analyses we do not attempt to quantify the effect of the introduction of the pay-for-performance scheme on patient experience. As such, our approach is different from our previous analyses.7,8 Our focus in this report is on changes in patient experience across the entire period from 2003 to 2007, both between and within the 2 samples. To focus on the long-term trend, free from any short-term effects immediately after the introduction of pay-for performance, our main analysis uses only the data from 2003 and 2007. For completeness, however, we present some descriptive statistics on the outcome scores for 2005.
We used 2-level multivariate regressions to analyze the scores for each outcome, with patients nested within practices, so we could control for individual patient demographics in the analysis. For scale scores and ratings, we applied linear regressions for dichotomous items logistic regressions. In all cases a higher score represented higher patient-reported quality of care.
A single regression analysis was used for each outcome. We combined time-point (2003 or 2007) and sample (chronic or random) into a single 4-level categorical variable and used this variable together with patient characteristics (sex, age, ethnicity, accommodation [owner-occupier vs rented], and employment status) as regression factors. Postestimation comparisons were then used for the following: (1) test the mean change from 2003 to 2007 within the chronic sample; (2) test the corresponding change within the random sample; (3) compare the 2 samples on mean score in 2003; and (4) compare the samples on change in mean score from 2003 to 2007. The third test examined whether patients in the chronic and random samples differed in their ratings of care at the start of the period (2003), whereas the fourth test examined whether they subsequently differed in the degree to which their views changed during the 4-year period. An α error of 5% was used throughout, and all analyses were conducted using Stata, version 10.22 We used the xtmixed and xtmelogit regression commands in Stata, treating practice as a random effect and fitting the models using maximum likelihood. We present results in the form of coefficients (for linear regressions) and odds ratios (for logistic regressions).
Scores on the outcome scales tended to have skewed distributions with heterogeneity of variance across practices. We therefore assessed the sensitivity of the results to these violations of parametric regression assumptions by repeating the analyses using a bootstrap method with 1,000 bootstrap samples. The bootstrap inference method makes no distributional assumptions. None of the results that were significant in the primary analysis ceased be significant in the sensitivity analysis.
The study had full ethical committee and research governance approval.
RESULTS
Response rates for the patient survey are shown in Table 2⇑. Mean practice level GPAQ scale and item scores are shown in Table 3⇑ for both the chronic illness samples and the random samples of patients in 2003, 2005, and 2007 respectively. Table 4⇑ displays the results from the regression analyses.
Patients in the Chronic Illness Samples
Mean ratings of quality by patients with chronic illness were similar in all 3 time periods for the GPAQ scale scores for communication, coordination, nursing, and overall satisfaction (Table 3⇑). This finding was borne out by the regression analysis results (Table 4⇑), which indicated no significant change between 2003 and 2007 on any of these measures (P >.05). There was also no significant change on any of the GPAQ speed-of-access items (P >.05), including ability to see a particular physician or any physician within 48 hours. There were, however, negative changes in mean scores from 2003 to 2007 with regard to the 2 continuity-of-care items: how often patients reported being able to see their usual physician (decrease of 6.9%; 95% confidence interval [CI], −8.4 to −4.4) and satisfaction with this experience (decrease of 4.8%; 95% CI, −6.2 to −3.4). Significant differences between 2003 and 2007 scores were observed for 3 of the 5 GPAQ access items relating to urgent appointments: ability to book an urgent appointment with any doctor (improvement with odds ratio [OR] = 1.24; 95% CI, 1.06 to 1.45), satisfaction with this experience, and satisfaction with the ability to book an urgent appointment with a particular doctor (increase of 2.6%; 95% CI, 0.9 to 4.2). Differences in the speed-of-access items are not observable in Table 3⇑, however, as they only emerge after controlling for patient-level characteristics.
Patients Randomly Sampled From Registered Lists
Results for patients randomly sampled from practice lists matched those for patients with chronic illness in all aspects except the speed-of-access items: no significant changes over time were observed in any of these items. The only significant changes from 2003 to 2007 were in regard to continuity of care, with a reduction in how often patients reported being able to see their usual physician (decrease of 6.7%; 95% CI, −8.8 to −4.6) and also in the associated satisfaction ratings (decrease of 4.2%; 95% CI, −6.1 to −2.2).
Comparison Between Chronic Illness and Random Samples
For all GPAQ scale scores in each time period, mean scale scores were higher for the chronic illness samples than the random samples (Table 3⇑). Table 4⇑ shows that patients in the chronic illness samples had significantly higher adjusted ratings than patients in the random samples for all GPAQ scale scores, except the nursing scale and for most items relating to access and continuity of care in 2003 (P <.01 in all cases). The exceptions were the ability to get an appointment within 48 hours with a particular physician (OR = 0.90; 95% CI, 0.74 to 1.09) or any physician (OR = 0.90; 95% CI, 0.74 to 1.09) and obtaining a same-day urgent appointment (OR = 0.94; 95% CI, 0.73 to 1.22).
The degree of change between 2003 and 2007 did not differ between the groups with respect to any outcome, except that ratings of how quickly an appointment could be obtained with a particular physician increased in the random sample, relative to chronic illness patients (3.4%; 95% CI, 0.2 to 6.2).
The confidence intervals around all the estimated effects were narrow, in all cases no more than plus or minus 3 points on a scale of 0 to 100.
DISCUSSION
We found no significant changes in quality of care reported by either group of patients between 2003 and 2007 for communication, nursing care, coordination, and overall satisfaction. While some aspects of urgent access improved significantly for patients with chronic disease, they did not for the random samples of patients. Patients in both samples reported seeing their usual physician less often and gave lower satisfaction ratings for continuity of care.
Strengths and Weaknesses of the Study
In this study we collected data from both random samples of patients and patients with chronic illness in the same nationally representative sample of family practices18 using a validated instrument—the General Practice Assessment Questionnaire.21 The value of these repeated cross-sectional design data sets is that they provide the ability to track trends. Research into the impact of the QOF has tended to focus on clinical aspects of care7,8 and cross-sectional associations between target achievement and, for example, practice characteristics,15 deprivation data,23 or population characteristics.24 In this study we used patient level to evaluate both patient assessments and trends in patient assessments of care for patients with chronic illness and from the general population across 5 years spanning the introduction of a range of healthy service reforms affecting UK primary care, including the QOF pay-for-performance scheme. The error intervals around the estimated effects were all very narrow; however, response rates—especially for the population samples—were low, but on a par with previous studies.25,26 Any systematic source of response bias is likely to be the same for all the samples included, although differential nonresponse bias might have affected the comparison between the chronic illness and random samples.
Implications for Practice and Policy
The responsiveness of health systems to the needs of the population is an important part of the evaluation of health care27 and is best measured through patients’ evaluations of the care they have received.28 By unpacking the patients’ experiences for those with and without long-term conditions, this study corroborates the finding of the recent National Audit office report9 that there have been few changes in patient satisfaction in the United Kingdom during the 2000s in relation to overall satisfaction, nursing care, and coordination of care.
Our findings suggest that patients in the United Kingdom are significantly less likely to report being able to make an appointment with their usual physician, and they report lower ratings of continuity of care in 2007 compared with 2003. This finding was observed in patients with chronic illness and in population samples of patients. It may not be surprising that continuity has decreased when initiatives to improve access to physicians have been prioritized.29
Patients’ reports differed across the 2 groups of patients. For the chronic illness sample there was improvement on some of the access items, especially satisfaction with the ability to book an appointment with either a particular physician or any physician within 48 hours. For the random sample groups of patients, however, there was no significant improvement with time. Policy initiatives privileging access to primary care in the United Kingdom—including incentivizing the availability of appointments within 48 hours—have therefore had only a modest impact on patients with chronic illness (the predominant clinical focus of QOF) but not for random samples of patients. The net no mean effect may have resulted from some patients finding it easier and some harder to gain access. Others, however, have argued that access initiatives in the United Kingdom have failed because of uncertainty of purpose and subsequent diversity of implementation.2 The focus on access has been reinforced by the introduction of 2 new indicators in to the QOF based on patient reports of their ability to see a physician within 48 hours and the ability to book appointments in advance.30
The introduction of quality improvement strategies can lead also to unintended consequences, such as the neglect of other measurable items of nonincentivized care.31 We found that patients report no overall change in patient assessments in their experience of communication with their usual physician. We did find a reduction in continuity of care in the period spanning the introduction of pay-for-performance as assessed by patients, as predicted previously,17 as well as during initiatives designed to improve access to care. Concern has been expressed also about the effect of the QOF on the way physicians and patients relate to each other within consultations.32,33
In this study we identified some loss of continuity of care for both groups of patients. Individual patient’s preferences vary according to the reason for making an appointment, and patients do not necessarily want fast access at all costs.14,25,34–36 Indeed, interpersonal continuity is sometimes more important to patients than fast access, particularly for people with long-term conditions.25,37
There may be evidence of a possible perverse incentive created by the 24-hour and 48-hour access targets; alternatively, our findings might reflect the employment of more part-time staff or a greater use of chronic disease management clinics. The percentage of part-time physicians in this sample of 42 representative practices, however, increased during the study period, from 24% in 2003 to 26% in 2005 and 26% in 2007.38 Moreover, the number of practices providing appointments within clinics for asthma, diabetes, and heart disease showed only minor changes between 2003 and 2007 (61% vs 69%, 68% vs 70%, and 46% vs 41% in 2003 and 2007 for asthma, diabetes, and heart disease, respectively.)
These findings, especially our observation that continuity of care may be compromised by initiatives oriented toward enhanced access, have resonance for and inform current discussion in the United States about the patient-centered medical home (PCMH) model.39,40 Although the PCMH lacks a shared vocabulary,41 primary care in the United Kingdom meets several principles of the United States PCMH. For example, patients have an ongoing relationship with their family physician, a personal physician trained to provide first-contact, continuous, and comprehensive care with a whole-person orientation who leads a team of individuals at the practice level that collectively take responsibility for both ongoing care of the patient and the coordination through the whole system. Our findings provide evidence that in the development of this model, one relevant feature, enhanced access, may need to be balanced against the perhaps more fundamental feature of continuity.29,42 This observation is consistent with previous research conducted in the United States.43 The paucity of empirical data on the issue makes more research necessary to elucidate whether it would be actually possible to devise PCMH models that could include both enhanced access and continuity of care.
Acknowledgments
The authors would like to thank the staff of all participating practices, as well as Nan Bailey, Michele Bohan, Nicholas Burr, and Angela Swallow, who contributed to data collection.
Footnotes
-
Conflicts of interest: Dr Roland reports serving as an academic advisor to the government and British Medical Association negotiating teams during the development of the UK pay-for-performance scheme during 2001 and 2002. We report no other potential conflict of interest relevant to this article.
-
Funding support: This work was done at the National Primary Care Research and Development Centre, which receives funding from the UK Department of Health. The views expressed are those of the authors and not necessarily those of the UK Department of Health.
- Received for publication June 29, 2009.
- Revision received March 1, 2010.
- Accepted for publication April 1, 2010.
- © 2010 Annals of Family Medicine, Inc.