Abstract
PURPOSE We investigated 3 approaches for implementing the Chronic Care Model to improve diabetes care: (1) practice facilitation over 6 months using a reflective adaptive process (RAP) approach; (2) practice facilitation for up to 18 months using a continuous quality improvement (CQI) approach; and (3) providing self-directed (SD) practices with model information and resources, without facilitation.
METHODS We conducted a cluster-randomized trial, called Enhancing Practice, Improving Care (EPIC), that compared these approaches among 40 small to midsized primary care practices. At baseline and 9 months and 18 months after enrollment, we assessed practice diabetes quality measures from chart audits and Practice Culture Assessment scores from clinician and staff surveys.
RESULTS Although measures of the quality of diabetes care improved in all 3 groups (all P <.05), improvement was greater in CQI practices compared with both SD practices (P <.0001) and RAP practices (P <.0001); additionally, improvement was greater in SD practices compared with RAP practices (P <.05). In RAP practices, Change Culture scores showed a trend toward improvement at 9 months (P = .07) but decreased below baseline at 18 months (P <.05), while Work Culture scores decreased from 9 to 18 months (P <.05). Both scores were stable over time in SD and CQI practices.
CONCLUSIONS Traditional CQI interventions are effective at improving measures of the quality of diabetes care, but may not improve practice change and work culture. Short-term practice facilitation based on RAP principles produced less improvement in quality measures than CQI or SD interventions and also did not produce sustained improvements in practice culture.
- primary health care
- family medicine
- patient-centered medical home
- quality improvement
- practice facilitation
- diabetes mellitus
- practice-based research
INTRODUCTION
To meet the challenges of a reformed health care system, primary care must adopt substantially new models such as the Patient-Centered Medical Home (PCMH) and integrate their work within accountable care organizations.1–4 The PCMH has emerged as a cornerstone of primary care redesign with its strong appeal of uniting 4 compelling areas of health care reform: (1) the well-demonstrated value of primary care based on 4 core attributes,5–6 (2) proactive, population-based approaches to chronic care, (3) consumerism and patient-centered care, and (4) new health information technology. Much of the redesign effort has focused on implementing the Chronic Care Model,6,7 which has been associated with better health outcomes for patients with chronic conditions and, specifically, type 2 diabetes8,9; however, data regarding adoption of this model’s principles into primary care practices have been disappointing.10,11 Primary care practices have few mechanisms for incorporating new programs, which can slow adoption of innovations and cause disruptions when innovations are finally implemented.12–16 With the central importance of primary care in health care redesign models such as the PCMH and accountable care organizations, effective strategies for enhancing primary care practice improvement and transformation are critically important.
Practice facilitation has emerged as a key method for assisting practices in implementing organizational changes.17–21 Facilitators can assist practices in implementing evidence-based programs, tailoring programs to individual practice situations, improving incorporation into operations, and increasing sustainability. Emerging evidence indicates that practice facilitation can be successful in improving preventive care18–20 and implementing aspects of the PCMH.21–23
Different approaches to practice facilitation have been described, but studies to determine optimal characteristics of facilitation are lacking. A recent meta-analysis found a relationship between the intensity of facilitation and effect size, but no relationship between the duration of facilitation and effect size.20 Aspects of facilitation are generally shared across approaches, including the formation of improvement teams consisting of diverse practice members who meet regularly to plan and implement changes24; however, the strategies and theoretical framework of the change process have been quite different across various facilitation programs. The model used most widely has been based on continuous quality improvement (CQI), using a series of incremental plan-do-study-act cycles focused on quality measures to implement practice improvements.25–27
Others have adopted more systemic approaches to change based on complexity science, including the reflective adaptive process (RAP).28–30 Organizational culture has been shown to be associated with successful innovation and quality of care in medical practices.31–33 RAP focuses on enhancing practice capacity to make change by improving a practice’s change culture, communication, problem solving, and organizational learning to lay the groundwork for an ongoing process of change and improvement.
We undertook a cluster-randomized controlled trial, Enhancing Practice, Improving Care (EPIC), to compare the effectiveness of 3 approaches for implementing and sustaining Chronic Care Model–based systems to improve diabetes care: (1) practice facilitation using a RAP approach to stimulate reflective conversations and improve the practice’s capacity to manage change, applying the change process to diabetes care; (2) practice facilitation using a CQI approach to implement quality improvement for diabetes to improve diabetes care; and (3) providing self-directed (SD) practices with information and resources about the Chronic Care Model and quality improvement to improve diabetes care, but without facilitation. Our hypotheses were that (1) the RAP approach would improve practice change culture to a greater and more sustained degree than the CQI or self-directed approaches; (2) the CQI approach would lead more quickly to improvement in diabetes performance measures than the RAP or SD approaches; and (3) the RAP approach would produce more sustained improvements in performance measures through positive impact on practice change and work cultures.
METHODS
Study Design and Recruitment
The setting for the study was small to midsized community health centers and independent mixed-payer primary care practices in Colorado. Practices were recruited through multiple contact methods targeting interested primary care clinicians, especially those in the State Networks of Colorado Ambulatory Practices and Partners (SNOCAP), a collaborative of several practice-based research networks. A total of 44 practices were recruited, and 40 were enrolled in the study in 3 waves. Practices were stratified by location (urban vs rural), practice size, and practice type (community health center vs other) and randomized so that the distribution of practice characteristics would be similar across groups. None of the practices were participating in any financial incentive programs or other diabetes improvement initiatives during the study period.
The methods used to collect outcomes data included patient chart audits and patient, clinician, and staff surveys at baseline and at 9 and 18 months after practice enrollment. The study was approved by the University of Colorado Institutional Review Board and registered with ClinicalTrials.gov (Protocol Registration Receipt NCT00414986). We report data from chart audits and clinician and staff surveys here; additional data from surveys will be reported elsewhere.
Practice Interventions
Practices in the RAP group received practice facilitation using the RAP change model based on complexity theory.28–30 The RAP approach focused on changing organizational functioning to improve diabetes care. This approach assumed that improving organizational capacity to make and sustain change is primary in achieving practice improvements and implementation of changes. Practice facilitators received specific training in the use of change management strategies and participated in regular debriefing sessions with an experienced facilitation supervisory team to ensure fidelity to the intervention process. Practice facilitators in this group performed a multimethod practice assessment, including assessment of practice communication, change and work culture, and level of implementation of the Chronic Care Model, and provided feedback to the practice. They then assisted the practices in developing improvement teams to implement the Chronic Care Model for diabetes, but without a specified change package. This approach allowed each practice to set its own priorities, pace, and targets of change. Theoretically, this form of facilitation should be necessary for only a relatively short period of time, with the practice improvement team progressively assuming responsibility for the ongoing improvement efforts after the initial facilitation. The facilitation intervention for this group lasted 6 months, with facilitators meeting with the improvement teams an average of 7.4 times (range, 4–11 times), although facilitators were available for consultation for up to 12 months after baseline. Additional details of the RAP intervention implemented in this study are published elsewhere.24
Practices in the CQI group received practice facilitation based on the Model for Improvement.25–27 Practice facilitators also performed an initial multimethod practice assessment, provided feedback to the practices, and helped form and initially facilitated practice improvement teams; however, the CQI facilitators provided a structure and process for quality improvement using CQI tools that particularly focused on sequential plan-do-study-act cycles guided by quality measurement data. A separate supervisory team met weekly with the facilitators to discuss practice progress and ensure fidelity to the intervention model. Implementation of a system for obtaining reliable quality measures was a time-consuming first step, and the length of the intervention was allowed to vary to accommodate this process. Facilitators worked with practices for up to 18 months (mean = 15 months) depending on practice needs. The facilitators met with improvement teams an average of 9.7 times (range, 4–18 times).
Practices in the SD group did not receive practice facilitation, but received limited feedback on their baseline practice culture and level of implementation of the Chronic Care Model based on practice surveys. SD practices were then given access to a website with information about quality improvement and the Chronic Care Model for diabetes, with follow-up surveys and chart audits at the same intervals as in the other groups.
Measures
Patient Outcomes
The process of diabetes care, the primary outcome, consisted of documentation, as ascertained by chart audits, that a patient had received 9 items from the American Diabetes Association Physician Recognition Program: hemoglobin A1c (HbA1c) level measurement, foot examination, blood pressure measurement, dilated eye examination, cholesterol level measurement, nephropathy screen (check for urinary protein), influenza vaccination, nutrition counseling, and provision of self-management support.34 Each practice generated a list of patients with diabetes who had at least 1 visit to the practice in the 18 months before practice enrollment and at least 1 visit in the 18 months after enrollment. We audited a random sample of charts, with a target of at least 20 to 25 patients per practice. Institutional review board–approved procedures for deidentifying the data were followed. Each item was considered up to date if it occurred within the 12 months before the end of each audit period (baseline, 9 months, 18 months). We derived a composite score for diabetes process of care consisting of the total number of up-to-date measures for each audit period according to guidelines. The possible range was 0 to 9, with higher scores indicating better and more guideline-concordant care. Additionally, patient blood pressure, HbA1c level, and low-density lipoprotein (LDL) cholesterol levels were abstracted.
Clinician and Staff Surveys
At baseline, 9 months, and 18 months, clinicians and staff members completed the Practice Culture Assessment, which was developed to measure perceptions of practice characteristics thought to be important to practice functioning and successful implementation of practice quality improvement. Item selection was based on a literature review, prior experience, and exploratory factor analysis combined with expert consensus and item analysis.12,35 Items selected for the Practice Culture Assessment were also separately used, along with related items, in the evaluation of the National Demonstration Project.36 Each practice also provided basic practice information, including patient demographics, patient volume, and implementation of health information technology.
Patient-Level Covariates
Patient characteristics collected in the chart audit included age, sex, and presence of chronic medical and psychiatric conditions. Race and ethnicity were generally not recorded in the medical record.
Statistical Analysis
We generated descriptive statistics (means, standard deviations, proportions, frequency distributions) for patient sociodemographics, diabetes process of care measures, and practice characteristics. General (or generalized for binary outcomes) linear mixed effects models, adjusting for patient-level covariates and clustering of patients within practices, were used to examine differences in outcomes over time by study group.
In addition to assessing overall differences in trajectories across study group, we assessed all 2-way comparisons and whether change occurred within each group. Covariates in patient outcome models included age, sex, and number of medical and psychiatric comorbidities. We used principal factor analysis with oblique rotation on Practice Culture Assessment items and retained 3 factors as determined by the proportion criterion. We computed the Cronbach α for each subscale to confirm internal consistency. Subscale scores were computed and used as outcome variables in the analysis of differential change over time by study group, with respondent position included as a covariate. For clinical outcomes, time was coded as days since baseline using date of service, with all measurements before baseline defined as preintervention (time = 0). As all statistical tests were specified a priori, we followed recommendations to report P values rather than adjust for multiple comparisons.37,38 Analyses were performed using SAS version 9.3 (SAS Institute Inc).39
RESULTS
The CONSORT (Consolidated Standards of Reporting Trials) diagram of practice, patient, clinician, and staff flow is shown in Figure 1. Before baseline data collection, 2 practices dropped out: 1 was disqualified for sharing personnel with another enrolled practice, and 1 was closed because of death of a physician. Between baseline and 9 months, 4 practices discontinued work with the facilitators. From practice-generated lists of patients with diabetes, we audited a random sample of 822 charts for diabetes process of care and outcomes. Clinician and staff response rates for the Practice Culture Assessment survey were 64% at baseline, 47% at 9 months, and 52% at 18 months.
Practice and Patient Characteristics
Baseline practice and patient characteristics are given in Table 1. Age and sex were similar across groups in the chart review sample, but baseline HbA1c level, systolic blood pressure, and total cholesterol level differed significantly across groups (all P <.05), with slightly better baseline control of each in RAP practices.
Patient Chart Audit
Analysis of chart audit data for diabetes process of care over time (Table 2) indicated differential improvement across groups in the number of elements that were up to date at the end of each audit period by study group (P <.0001). Although total process of care scores improved in all 3 groups (all P <.05), improvement was greater in CQI practices compared with both SD practices (P <.0001) and RAP practices (P <.0001). Improvement in process of care scores was also greater in SD practices compared with RAP practices (P = .03).
We examined individual trajectories for all study practices to determine whether patient outcomes were adversely affected in any practice. Most practices (72%) improved, and none had a significant worsening in diabetes process of care (all P >.05). Although not statistically significant, heterogeneity in practice-level change scores going from baseline to 18 months among the RAP practices (range, −0.51 to 2.12, SD 0.70) was less than that among CQI practices (range, −1.14 to 3.83, SD 1.47) and SD practices (range, −0.83 to 3.17, SD 1.09). We did not identify any factors (eg, practice type, location) clearly associated with improvement or worsening in process of care on closer inspection of individual trajectories.
The individual process of care elements are also shown in Table 2. There was overall differential change by group for several elements, including having feet checked (P = .02), having cholesterol levels checked (P <.0001), receiving the influenza vaccine (P = .02), and nutritional counseling (P = .03), with the greatest improvement in CQI practices. Additionally, there was greater improvement in CQI practices compared with RAP practices in having HbA1c levels checked and nephropathy screening (both P <.05). CQI practices had greater improvement than SD practices in eye examinations (P <.05).
Among the 803 patients with HbA1c levels checked during the study period (mean = 3.85 measures per person), the level remained stable over time in all groups. Among the 821 patients with blood pressure readings during the study period (mean = 6.0 measures per person), systolic blood pressure decreased slightly (1.04 mm Hg/year; P = .03), but trajectories did not differ by group. Diastolic blood pressure remained stable over time in all groups. Among the 782 patients with LDL cholesterol measures (mean = 2.87 measures per person), there was an overall decrease in level (5.27 units/year; P = .02), but trajectories were similar across groups. Results were much the same when analyses were restricted to patients above target at baseline.
Practice Culture Assessment
Factor analysis of Practice Culture Assessment items indicated that 3 factors should be retained. Factor loadings and individual items are shown in Table 3. Factor loadings can be interpreted as the correlations between the individual items and the rotated factors derived from the factor analysis, and indicate how strongly an item is associated with each factor. Factor scores on the 3 subscales were created as the sum of scores (reverse scored for negatively worded items) on the individual items with the highest loadings on that dimension, scaled to a 0 to 100 range with higher scores representing “more” of the concept. Internal consistency was high for all subscales: Change Culture (0.91), Work Culture (0.91), and Chaos (0.78). Results of analyses of Practice Culture Assessment subscales over time across study groups are shown in Table 4. Change Culture scores differed significantly over time in RAP practices compared with SD practices (P = .04), with a trend toward initial improvement (baseline to 9 months, P = .07) followed by a significant decline from 9 months to 18 months (P <.05), while these scores remained stable in SD practices over time. Work Culture scores over time also differed significantly in RAP practices compared with SD practices (P = .01), with a decrease from baseline levels at 18 months in RAP practices (P <.05) and stable scores over time in SD practices. CQI practices had stable scores for both Change Culture and Work Culture over time and did not differ significantly from SD practices. Chaos scores increased in both the RAP and CQI groups compared with the SD group (RAP vs SD: P = .03; CQI vs SD: P = .007).
DISCUSSION
In this comparative trial of 3 strategies for implementing the Chronic Care Model to improve diabetes care, all 3 study groups had significant improvement in diabetes process of care. CQI practices, which followed a prescribed strategy of implementing registries to have diabetes quality measures to motivate practice action, identify needed changes, and monitor progress, had significantly greater improvement in diabetes process measures than practices in the other 2 groups, but little change in practice culture. RAP practices, which followed a more practice-determined approach particularly aimed at improving practice culture, had significantly less improvement in diabetes measures than CQI or SD practices. RAP practices showed marginal improvement in Change Culture at 9 months followedby a decline at 18 months, along with a decline in Work Culture at 18 months. The slight but significant increases in practice chaos levels in the 2 facilitated study arms is interesting but expected, as implementation of such a change process tends to cause some chaos and disruptions in practice routines.
Traditional quality improvement strategies can be successful in implementing specific tools to improve quality measures in a focused, disease-specific area (such as the CQI intervention in our study), but this approach has been criticized for failing to achieve sustained improvements or highly adaptable practice cultures capable of innovating effectively. In this study, most improvements in overall diabetes process of care occurred during the initial 9-month period, and this improvement was sustained at 18 months.
The RAP model attempts to facilitate an adaptive learning organization that can generate ongoing efforts to enhance overall practice functioning, including quality measures. Similarto EPIC, the ULTRA project (Using Learning Teams for Reflective Adaptation) hypothesized that a facilitated RAP intervention would produce effective communication and change strategies that would subsequently lead to improved quality measures.30 That study found that practices identified and addressed issues related to communication, access to care, and access to information, but none focused on improving adherence to chronic disease care guidelines, and patient outcomes were not affected.30 Other studies have demonstrated that certain aspects of practice culture are associated with successful implementation of quality improvement interventions and improved patient process of care31–33; thus, improvements in practice culture through facilitation may enhance the practice’s ability to successfully change. In this study, practice culture may have been adversely affected by facilitators leaving the practice before stable improvements were achieved.
Although significant improvements were made in diabetes measures in all 3 study groups, care was still suboptimal at the end of the study period. Implementation of major changes in practices to improve care is not a quick process, and a sustained effort by practices (and organizations supporting them) to achieve the long-term improvements is needed. Further testing of intervention models for achieving these long-term improvements is warranted. Incremental changes are helpful, but major, transformational changes are needed.
It is an interesting and fairly consistent finding in quality improvement projects that process of care measures are easier to improve and respond more quickly than patient outcome measures, a pattern seen again in our study. In Glasgow’s Diabetes Priority Program study, patients of physicians randomized to the intervention had greater improvement in diabetes process of care than patients of physicians in the control group.40 These changes did not translate into statistically greater improvement in clinical outcomes, however. In a subsequent analysis of data from that study, Nutting et al41 demonstrated that greater clinician-reported use of elements of the Chronic Care Model was associated with improved patient clinical outcomes. This association certainly needs continued attention in our quality improvement and practice redesign efforts, as improvement in clinical outcomes are, in the long term, much more important than improved process measures.
Limitations of our study included differences in some baseline practice characteristics across groups, despite attempts to balance the groups by stratification. This imbalance could have resulted in greater room for improvement in some quality measures in CQI and SD practices. Also, the self-report Practice Culture Assessment data were potentially subject to bias, although such bias would likely be similar across groups.
The EPIC trial, framed in the context of related efforts, suggests several things: (1) traditional CQI interventions are effective at improving chronic disease quality metrics but may not improve practice culture; (2) motivated primary care practices can make some improvements with minimal assistance; (3) a short-term RAP-based intervention to improve practice culture may not be effective in primary care practices; (4) not all primary care practices may need the same type, intensity, or duration of assistance, and tailoring the approach to the practice is desirable; however, more information is necessary to guide the tailoring process; and (5) new models are needed to produce the long-term improvements in both quality measures and practice culture necessary for sustained enhancement of primary care practices. Additionally, heterogeneity in practices’ response to interventions suggests potential avenuesfor further exploration regarding which practices may respond best to which approaches, as well as identifying potential practice-level mediators of improvement.
The insights gained from the EPIC trial have ramifications for the further development and deployment of advanced primary care models that will be central in the redesigned health care system. Transforming the nation’s primary care practices will require a serious investment, and practice facilitation has been proposed as a foundation for change through a health extension service.42 Transformation to a PCMH requires more extensive change than improving diabetes care. The National Demonstration Project showed that even with 2 years of intense facilitation, implementing all PCMH components was very difficult.21,22 Although we can learna great deal from disease-specific studies, the policy issues they inform must consider more complex, interdependent changes to achieve comprehensive primary care and health system redesign. Practice facilitators attempting to promote sustained change in practices will need a broad range of skills in assessing practice culture, managing group process over time, and providing instrumental assistance to strengthen the 4 pillars of primary care, develop proactive population-based chronic and preventive care, and enhance patient centeredness and self-management support. Those who deploy practice facilitators must be prepared to assist practices over time, as a quick-in, quick-out approach will be ineffective in producing sustained comprehensive change. Further study is needed to compare different approaches, to identify salient characteristics of successful practice facilitation models, to determine the optimum “dosing” of practice facilitation for various targeted outcomes, to determine the characteristics of practices that respond to various change strategies, and to refine current approaches to achieve the dual goals of implementing innovations and enhancing practice change culture.
Footnotes
-
Conflicts of interest: authors report none.
-
Clinicaltrials.gov Protocol Registration Receipt NCT00414986.
-
Previous presentations: Some of these data were presented at the North American Primary Care Research Group meeting in Banff, Alberta, in November 2011.
-
Funding support: Funding for this work was supported by the National Institute of Diabetes and Digestive and Kidney Diseases (DF067083) and the National Institute of Mental Health (MH069809-04).
- Received for publication August 20, 2012.
- Revision received May 18, 2013.
- Accepted for publication June 10, 2013.
- © 2014 Annals of Family Medicine, Inc.