Improving Quality Improvement Capacity and Clinical Performance in Small Primary Care Practices

INTRODUCTION

Primary care practices in the United States are in a period of transition, with considerable pressure from payers and policy makers to improve quality of care and reduce costs.^1-4 Value-based pay contracts, quality reporting requirements, and market consolidation are accelerating the pace of change in many markets.⁵ Larger, well-organized primary care practices participate in state and national initiatives to improve measurement, transform care, and stay abreast of these rapid changes. These large-scale quality improvement (QI) efforts, however, require investments of time and resources that make engagement in and findings from these efforts impractical or irrelevant for smaller practices.^6-13

Smaller practices often lack key infrastructure and personnel capabilities—including multidisciplinary care teams, health information technology, and other resources¹⁴—necessary to be successful at implementing changes to quality or business systems. To survive in this new marketplace, smaller practices must build their capacity to adapt and improve. Building this capacity has been described as “…the planned development of knowledge, skills and other capabilities of a system or an organization to improve quality.”^15,16 Efforts to build QI capacity in primary care must involve appropriate strategies that can work in smaller practices,¹⁷ where almost two-thirds of US office-based physicians work.¹⁸

The Agency for Healthcare Research and Quality launched the EvidenceNOW initiative to better understand optimal approaches to building quality improvement capacity in smaller practices—defined as practices with fewer than 10 clinicians—that may have been unable or unwilling to participate in previous large-scale QI efforts.¹⁹ The EvidenceNOW initiative encouraged researchers and practice-change experts to test practice support strategies explicitly adapted to meet the needs of smaller practices across 7 regional cooperatives within the United States.²⁰ The overall purpose was to improve measures of cardiovascular disease risk known as the ABCS of heart health—appropriate aspirin prescribing, blood pressure (BP) control, cholesterol control, and tobacco screening/cessation counseling—by building QI capacity. To achieve these aims, 1 of the 7 cooperatives, Healthy Hearts Northwest (H2N), provided practice facilitation as a unifying strategy for all practices, with varied combinations of external support, academic detailing, and shared learning opportunities, to smaller practices in Washington, Oregon, and Idaho.²¹ Results from the randomized trial of different combinations of external support on observed changes in cardiovascular risk factor performance have been previously reported.²² Briefly, compared with facilitation only, practices randomized to all 3 support strategies had a small but significant improvement in performance on BP control.

The analysis reported here extends our understanding of the previously reported changes in clinical performance measures by testing a hypothesized mechanism whereby practices achieved improvements in clinical performance. A prespecified aim of the study was to examine whether small practices were able to improve their QI capacity in response to the external support provided and, if so, to assess whether that improvement was associated with gains in performance on clinical quality measures (CQMs) related to cardiovascular risk factors. To our knowledge, this is the first article to report on a validated measure of implementation of quality improvement capacity strategies at 2 points in time that assesses change in response to external practice support. It is also the first to evaluate whether those improvements in quality improvement capacity were associated with a change in clinical performance.

METHODS

Data Measures and Collection

Practice and Patient Characteristics

A baseline practice questionnaire was completed by the office/practice manager in each enrolled practice to collect information about practice characteristics including size, ownership, staffing, and patient characteristics such as insurance status and age groups.

Quality Improvement Capacity Assessment

The Quality Improvement Capacity Assessment (QICA) is a validated measure of the QI capacity of primary care practices and is organized around 7 strategies, termed high-leverage changes (HLCs), that guide efforts to expand clinical QI capacity.²⁴ Within improvement science, an HLC is defined as an intervention point within a system that has a high likelihood of causing a transformational change that improves outcomes.²⁵ These HLCs acted as the curriculum for the Healthy Hearts Northwest practice facilitators and practice teams to guide their practice change activities.²⁴ The 7 hgh-leverage changes were as follows:

1. Embed clinical evidence into daily work to guide how care is delivered to patients

2. Utilize data to understand and improve clinical performance measures

3. Establish a regular QI process involving cross-functional teams

4. Identify at-risk patients through proactive population management and outreach

5. Define roles and responsibilities across the team to improve care

6. Deepen patient self-management support to improve clinical outcomes

7. Link patients to resources outside of the clinic to support patients

Clinical teams were trained on and encouraged to use rapid-cycle tests of change (Plan-Do-Study-Act cycles) to iteratively test and then implement the HLCs. Most commonly, the tests were tied to improving measurement of BP and identification of hypertension, for example, “Medical assistant tried taking BP at the end of rooming process.” Other examples of rapid-cycle tests included trying daily “huddles” and “counting how many patients will receive a follow-up appointment for 2 weeks and how many actually come in over the next month.”²⁶ This local testing and adaptation enabled sites to adjust the sequence, timing, and pace of implementation of these HLCs to fit their context.

The total QICA is composed of 20 individual items, each scored between 1 and 12. A score of 1, 2, or 3 is noted as a Level D, which describes the element (eg, use of a registry to conduct outreach to high-risk patients) as not present in the practice. A score of 4, 5, or 6 is noted as Level C and describes the element as available or present, but often without routine application. A score of 7, 8, or 9 is noted as Level B, with the element present and frequently but not consistently applied, and a score of 10, 11, or 12 is noted as Level A, indicating consistent use of or application of best practice care. A change in score from a 5 to a 7 would therefore indicate that a practice has moved from an element of QI capacity that is available but not used, to one that is frequently but not consistently used during routine patient care.

A copy of the QICA questionnaire is available as a supplement in a previously published article.²⁴ Each HLC is assessed by 1 to 4 items on the questionnaire, and the score for each HLC is the average score given across all items in that category. The QICA score is the average score across all 20 individual items; hence, HLCs are weighted unevenly in the total score. Scores for each HLC and for the QICA overall range between 1 and 12. The QICA was to be completed twice during the study: once at the first in-person facilitation meeting and again at the fourth in-person quarterly facilitation meeting.

Cardiovascular CQMs

We collected 3 CQMs assessing cardiovascular risk factor management among practice patients: appropriate aspirin use, BP control (<140/90 mm Hg), and smoking screening and cessation counseling. For more detail on the CQM specifications, see the Supplemental Appendix (available at https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2733/-/DC1). All measure sets and definitions were endorsed by the Centers for Medicare & Medicaid Services.^27-30 We intended to collect the lipid/statin therapy measure (Centers for Medicare & Medicaid Services measure 347); however, this measure was under revision at the start of the study because of recent changes in evidence-based clinical guidelines, and it was therefore not systematically collected and is excluded from analyses.

Each CQM is reported as the proportion of the eligible patient population meeting the quality standard. Practices were asked to report CQMs each quarter, using a 12-month lookback period. This analysis uses data corresponding to clinical care provided for calendar year 2015 (the year before the 15-month study intervention) and calendar year 2017. External practice support interventions started in some practices in the last 2 months of 2015 and concluded by the end of calendar year 2017 (Figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Timing of QICAs relative to CQM lookback periods 1 and 2.

CQM = clinical quality measure ( appropriate aspirin prescribing, blood pressure control, cholesterol control, and tobacco screening and cessation counseling [ABCS]) ; Jan = January; QICA = Quality Improvement Capacity Assessment.

Note: Figure shows the timing of QICA completions relative to the period of time included in 2015 (lookback period 1) and 2017 (lookback period 2) CQM summary measures. The vertical lines denote the first and last observed dates of QICA 1 and QICA 2 completion. The CQMs assess outcomes over an entire calendar year; lookback period 1 covered 2015, while lookback period 2 covered 2017.

RESULTS

Of the 209 enrolled practices, 165 completed QICA surveys at both baseline and follow-up and were included in this analysis (Figure 2). Practices included in the analysis did not differ significantly from those not included in terms of organization characteristics (urban-rural location, size, organizational ownership type), patient characteristics (payer mix, and age and sex distribution), or baseline BP CQM performance (Supplemental Table 1, available at https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2733/-/DC1). Practices included in the analysis, however, had lower QICA scores at baseline (P = .03), and disproportionately represented those in Washington and Idaho over Oregon because the former more often returned both baseline and follow-up QICA questionnaires.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Flowchart of practice inclusion in various analyses.

BP = blood pressure; CQM = Clinical Quality Measure; QICA = Quality Improvement Capacity Assessment.

The mean baseline QICA score was 6.45 (SD 1.39), which increased on average by 1.44 points (95% CI, 1.20-1.68) (Table 1). The mean changes in both overall QICA score and individual HLC subscores differed significantly from zero (P <.001). The largest improvements were noted in using data to improve performance and establishing a regular QI process.

Average improvements for all 3 CQMs—the percent of eligible patients using aspirin (from 66.8% to 70.8%), having controlled BP (from 61.5% to 64.8%), and receiving smoking screening and cessation counseling (from 73.8% to 81.3%)—were statistically significant as well.

There was also strong evidence in favor of an association between change in QICA and change in the continuous BP CQM in the adjusted analysis: for every 1-point increase in QICA score, we would expect an additional 1.25% of patients with hypertension to have well-controlled BP defined as a measurement of less than 140/90 mm Hg (95% CI, 0.41%-2.09%; P = .003) (Table 2). There was little evidence to support associations between change in QICA score and changes in either aspirin use or smoking intervention CQMs as continuous measures, given that neither observed association was statistically significant.

We conducted analyses to determine whether the adjusted association between change in QICA and change in the BP CQM were driven by results from outlier practices by calculating DFBETA statistics, which show the effect that removing each observation has on the estimates for regression coefficients.³³ Removing clinics that caused the biggest changes in regression estimates did not result in any changes to inference. We conducted the same analysis for aspirin and smoking CQMs to determine whether outlying clinics attenuated the estimated association but found no evidence of such influence.

Table 3 shows the association between change in QICA score and likelihood of meeting the 70% threshold in 2017 for each of the clinical quality outcomes. With each 1-point increase in QICA score, a clinic was estimated to be 1.24 times more likely to reach the 70% threshold for BP control (95% CI for relative risk, 1.09-1.40; P <.001); however, evidence of associations for the aspirin and smoking outcomes was weak.

DISCUSSION

With 15 months of external practice support, we observed significant improvements in both QI capacity and all 3 cardiovascular risk factor performance measures in these smaller primary care practices. In addition, the observed improvement in QI capacity was associated with improved performance on the BP control CQM, but not on the appropriate aspirin use or smoking cession measures. For each 1-point increase in the QICA score, practices were 24% more likely to reach the Million Hearts campaign goal of 70% of patient with well-controlled BP (<140/90 mm Hg).

Why was improved QI capacity observed to be associated with improvements in BP control, but not in appropriate aspirin use or smoking screening and cessation counseling? One possible explanation is that improving BP control is a much more complex organizational process than the other 2 measures that can show rapid improvement through more thorough documentation in the electronic health record.³⁴ Existing literature suggests that team-based multicomponent efforts are required to improve BP control clinical performance measures and include engaging staff in proper measurement, intensifying medication therapy by clinicians, and supporting patients around medication adherence and self-management.^35,36 The measure of QI capacity used in this study accounted for many if not most of these BP improvement activities, including support for patients in self-management activities. Another possible explanation is that practices built their QI capacity by focusing on BP control. In a separate analysis of notes kept by practice facilitators after each encounter with a practice, improving BP measurement was the most common focus of the Plan-Do-Study-Act cycles of improvement that practices engaged in.²⁶

Several limitations of this analysis deserve mention. First, the QICA is a self-reported tool that was completed by clinical teams whose composition changed over the course of the project. As such, the change in score may not reflect change in QI capacity. Facilitators reviewed baseline practice QICA scores with the teams, however, and used these scores to guide improvement efforts. This assistance is reflected in the finding that the baseline QICA score was predictive of both the number and the types of topics discussed.³⁷ For follow-up scores, facilitators were asked if they agreed with the practice’s self-scores, specifically, that the scores accurately reflected externally observable changes in care processes; most agreed. In addition, changes in the HLC subscores were greatest for the HLCs where the facilitators focused most of their time and effort, which increases confidence that the tool is measuring real improvements.³⁸ Another study limitation is that the second QICA evaluation occurred during the observation period for the follow-up CQMs. This overlap may have biased results toward the null hypothesis of no association between change in QICA score and change in CQM, because the changes made in the practice to improve QI capacity may have not had adequate time to impact improvements in quality measures. Finally, there was no control group of practices that did not receive the practice facilitation intervention against which to compare these findings.

By prospectively defining a package of evidence-based strategies, then assessing their implementation, this study provides evidence for a plausible pathway from external practice support to improvements in patient cardiovascular risk factors via improvements in QI capacity. It also provides a key opportunity to close the gap between known best practices in the literature for addressing cardiovascular risk and common practice by defining a set of 7 effective strategies that can be spread to others.^39,40 Practices were able to make improvements in all areas, with most improvements occurring in the domains related to QI where facilitators focused their efforts. Measuring implementation progress on those HLCs using the 20-item QICA was feasible for small practices and may provide an intermediate measure that is sensitive to change in the short term. In addition to measuring progress on improving QI capacity, change in the QICA score was associated with changes in a clinical quality measure. The research field continues to grapple with how best to implement and disseminate evidence-based strategies, especially to smaller practices that lack robust QI infrastructure, but where many Americans continue to receive care.^41-43 Using a prospectively designed set of evidence-based strategies with a concordant tool for assessing implementation may be a strategy for accelerating our translation of evidence into practice as it can serve as a tested guide for future spread and sustainment efforts.⁴⁴

This study makes a unique contribution to the literature by demonstrating that the 7 HLCs used may provide a reasonable set of activities for small practices to undertake over a relatively short time period to build their QI capacity for the purpose of improving clinical outcomes. Of interest to policy makers, it appears the relatively light support provided by a facilitator was sufficient in fostering change within the complex system of a primary care practice,^45,46 although variation in context is important and more research is needed to understand how practice characteristics influence their ability to implement and sustain change.