Relationship Between Clinical Quality and Patient Experience: Analysis of Data From the English Quality and Outcomes Framework and the National GP Patient Survey

INTRODUCTION

Providing high-quality clinical care and providing a good patient experience are priorities for most health care systems.¹ Patient experience is a valuable outcome in its own right, beyond any relationship with clinical outcomes. Nevertheless, there are a number of possible causal pathways that might link patient experience and clinical or organizational quality which are relevant to the assessment and management of care. These pathways include patients being more satisfied with care of better technical quality, poor access in which patients fail to attend scheduled appointments and therefore receive a poorer standard of care, poor communication leading to reduced compliance with treatment, and poor continuity of care reducing compliance with appointments and treatment. In addition, it is possible that increased attention paid to clinical aspects of care, for example as a result of pay-for-performance schemes such as the Quality and Outcomes Framework (QOF),² could inadvertently lead to a deterioration in patient-centered care and hence to patient-reported experience.

Although there is a considerable literature dedicated to the measurement of clinical quality and patient experience, respectively, there has been little research exploring the relationship between these 2 key domains of quality. Studies in primary care settings have reported findings ranging from no association^3–5 to modest associations^6,7 between measures of patient experience and clinical quality, whereas studies conducted in secondary care generally report positive associations with correlations, some modest (eg, 0.19),⁸ and some substantial (eg, 0.63).⁹ Methodological issues may contribute to the wide range of results reported in these studies, including sample size, choice of indicators used to define clinical quality and patient experience, and whether the analyses were conducted at individual, practice, or health plan level. In particular, analyses conducted with small samples may underestimate correlations in the absence of shrinkage estimation, aggregation, or compositing (combining a number of indicators).

In this study, we set out to investigate the relationship between 2 established measures of quality in UK primary care: the QOF, a major pay-for-performance scheme that focuses largely on clinical aspects of care, and the General Practice Patient Survey (GPPS), which measures patient experience of family practices through a large cross-sectional population survey.^10,11 For the purpose of this report, we use the term clinical quality to denote technical aspects of medical care.

METHODS

We explored associations of clinical and patient experience dimensions of quality at the level of the primary care practice.

RESULTS

A total of 7,759 practices in England (93%) had data that could be included in the analyses. For individual clinical domains in the QOF, the number of included practices ranged from 1,312 (16%) for the cancer domain to 7,758 (93%) for the smoking cessation domain. Table 1 shows the medians and interquartile ranges for the shrunken estimates of practice performance on the GPPS questions and QOF clinical quality summary score. Median practice performance on these measures varies from 13.2% to 94.6%, with most showing an appreciable range of practice performance.

Table 2 shows the results of the rank correlation analysis (column 2). Although most of the correlations are statistically significant, the strength of the associations is generally weak, with the highest being 0.18 and more than one-half at 0.11 or less. Correlations with clinical quality were highest for patient assessments of access (telephone access 0.16, availability of urgent appointments 0.15, ability to book ahead 0.18, ability to see preferred doctor 0.17) and overall satisfaction (0.15). Correlations for individual clinical conditions are shown in Supplemental Table 1, available at http://annfammed.org/content/11/5/467/suppl/DC1. In the sensitivity analysis (not shown) using unshrunken clinical quality scores, most correlation coefficients in the Supplemental Table changed by less than 0.002, with only 8 changing by more than 0.02.

The results of the regression analysis modeling the association between the QOF clinical summary score and the various GPPS items are also shown in Table 2 (columns 3 and 4). All associations were statistically significant, but the size of the associations is small, with standardized regression coefficients ranging from 0.05 to 0.17. In the sensitivity analysis (not shown) using unshrunken QOF proportions, standardized regression coefficients are smaller in all cases (as expected), by up to 0.03. Adjustment for practice-level case mix leads to a small decrease in the standardized regression coefficients for most GPPS items.

Table 3 shows the proportion of variance in the QOF clinical summary score associated with different groups of GPPS questions. Access has the strongest association with QOF scores, followed by overall satisfaction and continuity of care, care planning, communication, and confidence and trust in the doctor. We can also consider the proportion of variance uniquely associated with each GPPS domain, ie, the variance explained above and beyond other factors. When considered, access and care planning are each uniquely associated with more than 3 times as much variance in QOF scores as any other GPPS domain (not shown in table).

Supplemental Table 1 shows a breakdown of correlations by type of quality indicator (recording/review, tests/specialist referral, treatment, intermediate outcome). In general, the correlations were highest for associations between GPPS scores and intermediate clinical outcomes. Intermediate outcome indicators denote percentages of patients achieving, for example, a recommended cholesterol, hemoglobin A_1C, or blood pressure measurement. The associations with GPPS scores were again small, with no correlation coefficients above 0.18.

DISCUSSION

Our results show that across general practices in England there are statistically significant but weak associations between patient-reported experience of care as measured by the GPPS and the technical quality of care as measured by the QOF. These positive associations persist after adjustment for population-level factors (Table 2).

Among the different areas covered by the GPPS, questions about access (ability to get through on the telephone and to make appointments) have the strongest correlations with the clinical scores, followed by questions addressing continuity of care, overall satisfaction, and then care planning. Dimensions of patient experience are themselves interrelated, however, and it is notable that considerably more variation in QOF scores are associated with access and care planning than with any of the other quality dimensions. These results could suggest a more direct causal link between improved access and improved care planning on clinical quality than for other patient experience dimensions. For example, by enhancing access, practices may increase patient encounters and create added opportunities to address aspects of care identified in the QOF. In concordance with these findings, previous research has reported positive correlations between patient-reported access and process measures of clinical quality in family practice.⁶ Furthermore, a previous analysis in England reported positive associations between patient ratings of access drawn from a precursor of the GPPS and clinical quality scores from QOF scores; it also reported that improved access scores were associated with reduced emergency admissions.¹⁸

Associations of overall patient satisfaction with clinical quality were positive in our study, unlike findings from smaller US primary care studies,^3,4 which found no relation between these 2 domains. The lack of a strong association between patients’ overall satisfaction and any technical quality measures may not be surprising, as we know from qualitative research that patients’ assessment of technical quality is heavily colored by communication with the physician.¹⁹ This association is consistent with evidence that patient reports of overall satisfaction correlate closely to communication ratings.²⁰ As such, our results offer little evidence that improved technical care would lead to improved patient satisfaction. Correlations between patient satisfaction and clinical summary scores are consistently positive, however, and this finding challenges the suggestion that, by encouraging doctors to concentrate on technical aspects of care, incentive schemes such as the QOF will lead to deterioration in the doctor-patient relationship.

Our findings are consistent with a situation in which physicians who are committed to high quality in one dimension are somewhat more likely to be committed to high quality in the other, but with the dimensions being sufficiently distinct that improving one has little, if any, direct effect on improving the other. Alternatively, it may be that the modest positive association between satisfaction and clinical quality reflects 2 separate causal links: (1) a strong contribution of access, care planning, and other specific dimensions of patient experience to overall patient satisfaction, and (2) a modest contribution of access and care planning to clinical quality.

Our findings are relevant more widely to health systems investing in health care monitoring and improvement initiatives. The lack of negative correlations in our study tells us that focus on one domain of health care quality need not necessarily be at the expense of another. At the same time, the small magnitude of the associations cautions against relying on a single measure and emphasizes the need to maintain a multidimensional approach by constructing quality indicators across all domains. Such an approach promotes a quality measurement model where such patient priorities as access and continuity are integrated alongside clinical and organizational effectiveness.

Most family practice research on the relationship between quality of care and patient experience has relied on process measures of technical quality. The QOF contains a combination of process and outcome measures. When we split our clinical quality data according to the type of indicators, correlations between outcome measures and patient survey data were higher, compared with correlations between process measures and patient survey data. Overall, satisfaction and continuity (seeing preferred doctor) were patient survey domains that showed the highest correlations with outcome measures of clinical quality. Sequist et al analyzed the relationship at the patient and practice level on US data drawn from the Health Plan Employer Data and Information Set (HEDIS), which considers process and outcome measures of clinical quality separately. In contrast with our results, the study found no positive correlations between patient experience and clinical outcome measures.⁷

The QOF and GPPS are established measures of family practice quality in England. Both are the product of rigorous research and subject to regular revision. Limitations intrinsic to both sources of data may affect our findings, however. Although concerns relating to both a low response rate (38%) and reliability have been raised with respect to the GPPS, there is little evidence that low response rates have introduced bias,¹¹ and research shows that most survey questions used in this study meet stringent guidelines for reliability.²¹ The QOF has been criticized for failing to capture important elements of family practice.²² In fact, many important aspects of clinical care are not measured, either because they are not amenable to reliable and widespread measurement, or because they are yet to be incorporated into the QOF. Even though the QOF remains an imperfect measure of technical quality of care, it is the most reliable and comprehensive measure available.

Our data only allowed for practice-level analysis. We examined to what extent practice scores for patients’ experience correlated with practice scores for clinical quality. Even though this ecological association is of inherent interest, it does not allow us to draw conclusions about patient-level associations. Studies using the individual as the unit of analysis would enable us to further elucidate the relationship between patient experience and quality of technical care.

Our findings support the hypothesis that although there are positive associations between clinical quality of care and measures of patient experience, these 2 domains of care quality remain predominantly distinct, with statistically significant but very low correlations. The strongest correlations are between clinical quality and access. Clinical and patient experience domains of quality need to be considered separately when assessing the overall performance of a family practice.