Research ArticleOriginal Research

Use of Quality Improvement Strategies Among Small to Medium-Size US Primary Care Practices

Bijal A. Balasubramanian, Miguel Marino, Deborah J. Cohen, Rikki L. Ward, Alex Preston, Rachel J. Springer, Stephan R. Lindner, Samuel Edwards, K. John McConnell, Benjamin F. Crabtree, William L. Miller, Kurt C. Stange and Leif I. Solberg
The Annals of Family Medicine April 2018, 16 (Suppl 1) S35-S43; DOI: https://doi.org/10.1370/afm.2172

Abstract

PURPOSE Improving primary care quality is a national priority, but little is known about the extent to which small to medium-size practices use quality improvement (QI) strategies to improve care. We examined variations in use of QI strategies among 1,181 small to medium-size primary care practices engaged in a national initiative spanning 12 US states to improve quality of care for heart health and assessed factors associated with those variations.

METHODS In this cross-sectional study, practice characteristics were assessed by surveying practice leaders. Practice use of QI strategies was measured by the validated Change Process Capability Questionnaire (CPCQ) Strategies Scale (scores range from −28 to 28, with higher scores indicating more use of QI strategies). Multivariable linear regression was used to examine the association between practice characteristics and the CPCQ strategies score.

RESULTS The mean CPCQ strategies score was 9.1 (SD = 12.2). Practices that participated in accountable care organizations and those that had someone in the practice to configure clinical quality reports from electronic health records (EHRs), had produced quality reports, or had discussed clinical quality data during meetings had higher CPCQ strategies scores. Health system–owned practices and those experiencing major disruptive changes, such as implementing a new EHR system or clinician turnover, had lower CPCQ strategies scores.

CONCLUSION There is substantial variation in the use of QI strategies among small to medium-size primary care practices across 12 US states. Findings suggest that practices may need external support to strengthen their ability to do QI and to be prepared for new payment and delivery models.

INTRODUCTION

Health care delivery is evolving in the United States, with increasing emphasis on insurance payments based on improving quality instead of merely delivering health care services.1 This emphasis has brought a greater focus on population health management, quality measurement, and health care outcomes. Primary care practices use quality improvement (QI) strategies, such as monitoring and assessing outcomes, having skilled QI teams, and using system redesigns to improve patient and population health outcomes, system performance, and clinician experience, and to reduce health care costs.2–6 Implementing QI strategies can help practices deliver appropriate health services efficiently and improve health outcomes,7–10 yet much of what we know about the use of QI strategies in medical care comes from surveys conducted mostly in hospitals and health systems.11–17

Little is known about the use of QI strategies in small to medium-size primary care practices, where more than one-half of Americans receive care for their chronic conditions.18 Earlier studies have lacked generaliz-ability because of small sample sizes and limited contextual diversity. Better understanding of the prevalence and reasons for variation in use of QI strategies among smaller practices is needed, because this information can help Regional Extension Centers, Area Health Education Centers, and providers of health information technology enhance their capacity to promote QI initiatives.19–35 Accordingly, we leveraged a large sample of small to medium-size primary care practices in the United States to (1) assess the extent to which they use QI strategies as measured by the Change Process Capability Questionnaire (CPCQ) Strategies Scale, and (2) evaluate practice characteristics and contextual factors that may explain variations in use of QI strategies.

METHODS

Setting

In 2015, the Agency for Healthcare Research and Quality (AHRQ) launched EvidenceNOW: Advancing Heart Health in Primary Care, a multiyear pragmatic trial. EvidenceNOW is designed to generate information about the effectiveness of external QI support in helping small to medium-size primary care practices improve the heart health of their patients. The project is also designed to identify opportunities for developing organizational capacity for ongoing primary care practice improvement. AHRQ funded 7 regional cooperatives—grantees in each region—that were responsible for creating external support infrastructures to improve delivery of cardiovascular disease preventive care. The 7 participating regions that cover 12 US states include Colorado and New Mexico (Southwest Region); Illinois, Wisconsin, and Indiana (Midwest Region); New York City; North Carolina; Oklahoma; Virginia; and Washington, Oregon, and Idaho (Northwest Region).36–38 AHRQ also funded an independent national evaluation of the overall initiative—ESCALATES (Evaluating System Change to Advance Learning and Take Evidence to Scale).39

Survey Sample and Administration

EvidenceNOW cooperatives recruited 1,710 small to medium-size primary care practices, defined as practices with up to 10 clinicians (including physicians, physician assistants, or nurse practitioners).39 Because of recruitment challenges, the funder allowed some cooperatives to recruit a small number of practices with up to 15 full-time clinicians. A person knowledgeable about the practice structure and organization (eg, medical director or practice manager) from each participating practice was invited to complete a survey questionnaire before the start of any intervention (baseline). They were encouraged to consult with other members of the practice to gather relevant information to complete different parts of the questionnaire. From September 2015 through April 2017, 1,489 practice questionnaires (87% response rate) were collected. Each cooperative selected the mode of administration of the questionnaire based on their on-the-ground experiences with successfully collecting questionnaires to ensure high response rates. Details about practice recruitment and survey data collection are described elsewhere.39

Outcome Measure

The study outcome measure was practice use of QI strategies as measured by the CPCQ Strategies Scale. The CPCQ Strategies Scale includes 14 items assessing the extent to which a practice has used specific QI strategies to improve cardiovascular preventive care in the prior year. The CPCQ strategies score is computed as a sum of items rated from −2 (strongly disagree) to 2 (strongly agree): it ranges from −28 to 28. Higher scores indicate greater use of QI strategies. This scale was developed by expert clinic implementers in an iterative modified Delphi process.40 The scale has been previously validated in smaller practices, is reliable in measuring practice use of QI strategies, and correlates well with change in practice and care quality outcomes.9,41

Independent Variables

At the study’s initiation, the ESCALATES evaluation team led harmonization of survey measures with all cooperatives to identify and collect a core set of common survey domains and items. These domains and items were derived from previous primary care research studies and demonstration projects focused on improving quality and from the National Ambulatory Medical Care Survey Electronic Medical Records Questionnaire.18,42–47 The survey assessed practice characteristics (eg, size, ownership, location), factors external to the practice (eg, participation in demonstration projects, accountable care organizations [ACOs], or external incentive programs for reporting quality), and factors internal to the practice (eg, health information technology characteristics, use of clinical quality data for reporting and QI, and use of evidence in practice). See the Supplemental Table 1, available at http://www.annfammed.org/content/16/Suppl_1/S35/suppl/DC1/.

Statistical Analysis

We calculated descriptive statistics (mean, standard deviation, proportions) to characterize EvidenceNOW practices. We constructed a frequency histogram to examine variability in the CPCQ strategies score across practices. We then assessed the mean difference in the CPCQ strategies score by practice characteristics in 2 ways: (1) we used univariable linear regression to model each practice characteristic separately on the CPCQ strategies score; and (2) we used multivariable linear regression to create a model based on all of the practice characteristics. To select the final multivariable model, we utilized an Akaike information criterion backward variable selection method, which selects relevant independent variables. This method enhances interpretability and reduces multicollinearity. The model with the smallest Akaike value, regardless of statistical significance, is the best fit.48,49 This approach is grounded in the data and able to discover associations that may not have been tested previously. It does not make assumptions as to the comparative importance of some group of variables, because a priori knowledge on what practice characteristics are more important is limited. We used Cohen’s d effect sizes to evaluate whether differences between practice characteristic groups were clinically meaningful.50 Typical cutoff values for Cohen’s d are 0.2 for small, 0.5 for medium, and 0.8 for large effect sizes, corresponding to CPCQ strategies scores of 2.42, 6.05, and 9.68. The coefficient of determination (R2) was used to estimate the extent to which variation in CPCQ strategies score is explained by the regression model.

To evaluate potential bias that is due to missing practice characteristics data, we used multiple imputation by chained equations.51 Standard errors for logistic model parameters were corrected using standard multiple imputation adjustments. All statistical tests were 2-sided and α level of .05. Statistical analyses were performed using R version 3.3.2 (R Foundation). This study was approved by the Institutional Review Board at Oregon Health and Science University and UTHealth School of Public Health and was registered as an observational study at clinicaltrials.gov (NCT02560428).

RESULTS

Table 1 describes the 1,181 small to medium-size EvidenceNOW practices with complete data on CPCQ Strategies Scale.

View this table:
Table 1

Characteristics of Participating EvidenceNOW Study Practices (N = 1,181)

Eighty-two percent of practices had 10 or fewer clinicians; 20% were solo practices. The sample varied widely on types of ownership (clinician, health/hospital system, federally qualified health center, other). Sixteen percent of practices were located in rural areas, and 35% were in a medically underserved area. More than one-third of the practices reported being recognized as patient-centered medical homes, one-third as part of an ACO, and around 30% had participated in payment or quality demonstration programs. Almost one-half reported receiving external incentives or payments based on measurement of performance on clinical quality, adoption or use of information technology, or patient satisfaction.

Twenty-one percent of participating practices reported experiencing multiple major disruptions in the previous 12 months (eg, implementing a new electronic health records [EHRs] or billing system, moving to a new location, having clinician turnover, or being purchased by another organization). Almost two-thirds of practices reported producing a clinical quality measure report for aspirin, blood pressure, or tobacco cessation counseling in the previous 6 months, and 17% did not have the ability to create their own reports. Two-thirds of practices also used chronic disease registries, and more than one-half used EHR prompts and reminders or standing orders to promote evidence-based practices for cardiovascular disease prevention (60.9%) and management (57.0%).

Use of QI Strategies (Outcome)

The mean CPCQ strategies score was 9.2 (SD = 12.1, range = −28 to 28) with a slightly left-skewed distribution (Figure 1). There was considerable variation across cooperatives (range = 5.7–14.5). As shown in Table 2, most practices reported providing information and skills training to staff (75.4%) and changing or creating practice systems that make it easier to provide high-quality care (79.2%). Only 46.4% of practices reported delegating care to nonclinician staff, however, and only 41.7% used such methods as rapid cycle QI to implement organization-wide changes.

Figure 1
Figure 1

Distribution of the CPCQ strategies scores among 1,181 EvidenceNOW practices.

CPCQ = Change Process Capability Questionnaire.

View this table:
Table 2

Components of the Change Process Capability Questionnaire Strategies Scale (N = 1,181 practices)

Practice characteristics included in the final multivariable model explained 22.4% of the variance observed in CPCQ strategies scores (Table 3 and Supplemental Table 2, available at http://www.annfammed.org/content/16/Suppl_1/S35/suppl/DC1/). Adjusted for other practice characteristics, hospital or health system–owned practices had a lower mean CPCQ strategies score than did clinician-owned practices. Practices that were part of an ACO had higher mean scores; participating in demonstration projects or receiving external incentives for reporting quality were not associated with use of QI strategies.

View this table:
Table 3

Adjusted Mean Difference in Change Process Capability Questionnaire Strategies Scores by Practice Characteristics Among 1,181 Small to Medium-Size Primary Care Practices

Practices that experienced multiple disruptive changes during the previous year also scored significantly lower (mean difference = −2.68, P <.05). Practices that partly used EHR systems but were partly on paper had lower mean CPCQ strategies scores (−3.59, P <.05). Practices that had generated clinical quality measure reports in the previous 6 months had higher mean scores (5.09, P <.001) than those that had not. Further, practices that had someone in house to create clinical quality measure reports and those that discussed clinical quality data often during their staff meetings scored significantly higher. Finally, practices that had patient registries and those that used guidelines for cardiovascular disease management via EHR prompts and reminders or standing orders also had higher mean scores (4.45, P <.001) (Table 3).

Multivariable linear regression models using multiple imputation to examine the potential bias related to missing practice characteristics data (Supplemental Tables 3 and 4, available at http://www.annfammed.org/content/16/Suppl_1/S35/suppl/DC1/) showed qualitatively similar findings to the models without imputation.

DISCUSSION

This study is the first to characterize the extent of use of QI strategies in a large, diverse sample of 1,181 small to medium-size primary care practices. Practices were located in both urban and rural areas; they spanned 12 US states; and, they included a wide array of ownership structures. Such practices often lack the capacity to make and sustain changes and are typically the ones most in need of support to implement QI strategies.52 Although more than 70% of practices provided information and skills training to their staff and changed practice systems to improve care, only 40% used such QI methods as rapid cycle tests to implement system-wide changes. Other studies have described use of QI strategies as measured by the CPCQ Strategies Scale,9,41,53 but these represented smaller regions or single states, thus lacking the contextual and geographic diversity of EvidenceNOW.

Including a wide array of potential practice characteristics explained almost 22% of the variance observed in the CPCQ strategies score. Three practice features (use of registries, guideline implementation systems, and discussing clinical quality measures for QI) explained the largest amount of variance in the strategies score in our study. These features have been previously found to be important for developing quality improvement capacity, and QI-savvy practices must have these at the minimum. Validation of this finding in the large EvidenceNOW practice sample adds to the internal validity of the CPCQ strategies score. Other practice characteristics not included in the EvidenceNOW harmonized survey may further account for unexplained variation in the strategies score. These characteristics include aspects of practice culture and team organization, leadership structure, financial stability, and patient demographic characteristics that have also been shown to be associated with practices using QI strategies.52,54–56

Our study also showed that practices with ability to extract and use EHR clinical quality data had higher mean CPCQ strategies scores. This finding is especially relevant in light of recent research from EvidenceNOW showing that smaller practices face considerable challenges in extracting and using the EHR data needed for QI. We found that not having a completely electronic system could be detrimental to higher use of QI strategies and may make it harder for practices to have access to data needed for reporting and QI. The benefits of increasing use of QI strategies may be counterbalanced by the increasing disruptions we found practices are experiencing in the current health care environment. More independent practices are being purchased by health care systems, and EHRs are being updated with the advent of new technologies. Considerable practice resilience and capacity is required to continue making improvements in the midst of major disruptions.

Interestingly, health system or hospital-owned practices showed small but statistically significantly lower mean CPCQ strategies scores than independent, clinician-owned practices. This finding is somewhat counterintuitive because it is often the independently owned small practice that lacks the capacity to do QI.57,58 Emerging qualitative findings from EvidenceNOW (data not reported), however, show that smaller practices owned by larger systems may not have the autonomy to make QI changes and, more importantly, may only represent a business acquisition rather than benefitting from QI support by the larger system.59,60 In contrast, we found that practices participating in ACOs had higher CPCQ strategies scores. In the United States, groups of physicians, hospitals, and other health care clinicians that come together voluntarily to give coordinated high-quality care to the Medicare patients they serve are termed as ACOs. When an ACO succeeds in both delivering high-quality care and spending health care dollars more wisely, it will share in the savings it achieves for the Medicare program.61 In light of the shifting national emphasis on new and emerging payment models focused on quality rather than provision of services,1,62 this finding suggests that small to medium-size practices seeking to engage in ACOs might be on the forefront of increasing use of QI strategies or adopting new ones to be better prepared for the changing health care environment.

Our study findings are limited in that the EvidenceNOW practices were recruited for participation in a large dissemination and implementation initiative rather than a representative survey; they may not represent the underlying distribution of practices in each region with respect to such practice characteristics as size, ownership, and location. In addition, there were differences in modes of survey administration (Web survey, e-mail, paper) to tailor it to the individual region’s need. This flexibility was needed to ensure high response rates and to minimize the amount of missing data. Even so, some items had more than 10% missing data that could have affected the study’s findings. The observation that associations with response categories involving missing data became nonsignificant in multivariable analyses provides additional reassurance that findings are not due to nonresponse. In contrast to these limitations, our conclusions are based on one of the largest studies of small to medium-size, geographically dispersed, primary care practices, and statistical testing suggests that the results are not due to missing or biased data.63

Based on an unprecedented sample of more than 1,000 small to medium-size practices, those with registries, guideline implementation systems, and the ability to use quality measures data to guide improvements were most likely to use QI strategies. Practices with adequate QI support and autonomy and fewer major disruptions were also significantly more likely to use QI strategies to improve cardiovascular preventive services. Our study’s findings are especially relevant because the health care environment in the United States is shifting; new delivery system and payment reforms, such as advanced alternative payment models, the merit-based incentive payment system, and ACOs, require practices to emphasize value rather than service volume. Smaller practices may lag behind in adoption of these new models because they lack capacity. Our study emphasizes the importance of supporting these practices that are having the most difficulty implementing improvements to enhance their patients’ care and outcomes.

Acknowledgments

We would like to thank Dr Thomas Kottke, Cynthia Perry, and Jennifer R. Hemler for their contribution to this manuscript.

Footnotes

  • Conflicts of interest: authors report none.

  • Funding support: Publication of this article was supported by the Agency for Healthcare Research and Quality (AHRQ) through contract No. HHSA290201200019I, grant No. R01 HS023940.

  • Disclaimer: This work represents the opinions of the authors and should not be interpreted as official positions of the Agency for Healthcare Research and Quality or the US Department of Health and Human Services.

  • Supplementary Materials: Available at http://www.AnnFamMed.org/content/16/Suppl_1/S35/suppl/DC1/.

  • Received for publication June 15, 2017.
  • Revision received November 7, 2017.
  • Accepted for publication November 15, 2017.

References

  1. ↵
    1. Rajkumar R,
    2. Conway PH,
    3. Tavenner M
    . CMS—engaging multiple payers in payment reform. JAMA. 2014;311(19):1967–1968.
    OpenUrlCrossRefPubMed
  2. ↵
    1. Batalden PB,
    2. Davidoff F
    . What is “quality improvement” and how can it transform healthcare? Qual Saf Health Care. 2007;16(1):2–3.
    OpenUrlFREE Full Text
    1. Taylor E,
    2. Peikes D,
    3. Geonnotti K,
    4. McNellis R,
    5. Genevro J,
    6. Meyers D
    . Quality Improvement in Primary Care: External Supports for Practices.Rockville, MD: Agency for Healthcare Research and Quality; 2014. AHRQ publication No. 14-0048-EF. https://www.ahrq.gov/sites/default/files/publications/files/qipcfactsheet.pdf. Published Jun 2014. Accessed Mar 3, 2017.
    1. Ricci-Cabello I,
    2. Ruiz-Perez I,
    3. Rojas-García A,
    4. Pastor G,
    5. Gonçalves DC
    . Improving diabetes care in rural areas: a systematic review and meta-analysis of quality improvement interventions in OECD countries. PLoS One. 2013;8(12):e84464.
    OpenUrl
    1. Schriefer J,
    2. Leonard MS
    . Patient safety and quality improvement: an overview of QI. Pediatr Rev. 2012;33(8):353–360.
    OpenUrlFREE Full Text
  6. ↵
    1. Landon BE,
    2. Hicks LS,
    3. O’Malley AJ,
    4. et al
    . Improving the management of chronic disease at community health centers. N Engl J Med. 2007;356(9):921–934.
    OpenUrlCrossRefPubMed
  7. ↵
    1. Scholle SH,
    2. Asche SE,
    3. Morton S,
    4. Solberg LI,
    5. Tirodkar MA,
    6. Jaén CR
    . Support and strategies for change among small patient-centered medical home practices. Ann Fam Med. 2013;11(Suppl 1):S6–S13.
    OpenUrlAbstract/FREE Full Text
    1. Leeman J,
    2. Calancie L,
    3. Hartman MA,
    4. et al
    . What strategies are used to build practitioners’ capacity to implement community-based interventions and are they effective?: a systematic review. Implement Sci. 2015;10:80.
    OpenUrl
  9. ↵
    1. Solberg LI,
    2. Stuck LH,
    3. Crain AL,
    4. et al
    . Organizational factors and change strategies associated with medical home transformation. Am J Med Qual. 2015;30(4):337–344.
    OpenUrlCrossRefPubMed
  10. ↵
    1. Gold R,
    2. Bunce AE,
    3. Cohen DJ,
    4. et al
    . Reporting on the Strategies Needed to Implement Proven Interventions: An Example From a “Real-World” Cross-Setting Implementation Study. Mayo Clin Proc. 2016;91(8):1074–1083.
    OpenUrl
  11. ↵
    1. Cohen AB,
    2. Restuccia JD,
    3. Shwartz M,
    4. et al
    . A survey of hospital quality improvement activities. Med Care Res Rev. 2008;65(5):571–595.
    OpenUrlCrossRefPubMed
    1. Gray J,
    2. Razmus I
    . Improving venous thromboembolism prevention processes and outcomes at a community hospital. Jt Comm J Qual Patient Saf. 2012;38(2):61–66.
    OpenUrlPubMed
    1. Macias CG
    . Quality improvement in pediatric emergency medicine. Acad Pediatr. 2013;13(6)(Suppl):S61–S68.
    OpenUrl
    1. Mauger B,
    2. Marbella A,
    3. Pines E,
    4. Chopra R,
    5. Black ER,
    6. Aronson N
    . Implementing quality improvement strategies to reduce healthcare-associated infections: A systematic review. Am J Infect Control. 2014; 42(10)(Suppl):S274–S283.
    OpenUrl
    1. Paice JA,
    2. Barnard C,
    3. Creamer J,
    4. Omerod K
    . Creating organizational change through the Pain Resource Nurse program. Jt Comm J Qual Patient Saf. 2006;32(1):24–31.
    OpenUrlPubMed
    1. Soban LM,
    2. Hempel S,
    3. Munjas BA,
    4. Miles J,
    5. Rubenstein LV
    . Preventing pressure ulcers in hospitals: A systematic review of nurse-focused quality improvement interventions. Jt Comm J Qual Patient Saf. 2011;37(6):245–252.
    OpenUrlPubMed
  17. ↵
    1. Hing E,
    2. Burt CW
    . Office-based medical practices: methods and estimates from the national ambulatory medical care survey. Adv Data. 2007;(383):1–15.
  18. ↵
    1. Hing E,
    2. Rui P,
    3. Palso K
    . National Ambulatory Medical Care Survey: 2013 State and National Summary Tables. https://www.cdc.gov/nchs/data/ahcd/namcs_summary/2013_namcs_web_tables.pdf. Published 2013. Accessed Apr 14, 2017.
  19. ↵
    1. Wessell AM,
    2. Liszka HA,
    3. Nietert PJ,
    4. Jenkins RG,
    5. Nemeth LS,
    6. Ornstein S
    . Achievable benchmarks of care for primary care quality indicators in a practice-based research network. Am J Med Qual. 2008; 23(1):39–46.
    OpenUrlCrossRefPubMed
    1. Hysong SJ,
    2. Best RG,
    3. Pugh JA
    . Audit and feedback and clinical practice guideline adherence: making feedback actionable. Implement Sci. 2006;1:9.
    OpenUrlCrossRefPubMed
    1. Ivers N,
    2. Jamtvedt G,
    3. Flottorp S,
    4. et al
    . Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;(6):CD000259.
    1. Baskerville NB,
    2. Liddy C,
    3. Hogg W
    . Systematic review and meta-analysis of practice facilitation within primary care settings. Ann Fam Med. 2012;10(1):63–74.
    OpenUrlAbstract/FREE Full Text
    1. Knox L,
    2. Taylor E,
    3. Geonnotti K,
    4. et al
    . Developing and Running a Primary Care Practice Facilitation Program: A How-To Guide (Prepared by Mathematica Policy Research under Contract No. HHSA290200900019I TO 5.). Rockville, MD: Agency for Healthcare Research and Quality; 2011. AHRQ publication No. 12-0011.
    1. Grumbach K,
    2. Bainbridge E,
    3. Bodenheimer T
    . Facilitating improvement in primary care: the promise of practice coaching. Issue Brief (Commonw Fund). 2012;15:1–14.
    OpenUrlPubMed
    1. Nagykaldi Z,
    2. Mold JW,
    3. Robinson A,
    4. Niebauer L,
    5. Ford A
    . Practice facilitators and practice-based research networks. J Am Board Fam Med. 2006;19(5):506–510.
    OpenUrlAbstract/FREE Full Text
  26. University of Vermont. Vermont academic detailing program. http://www.uvm.edu/medicine/ahec/?Page=vermontacademicdetailing.html&SM=programssubmenu.html. Accessed Apr 14, 2017.
  27. National Resource Center for Academic Detailing. About us. https://www.narcad.org/about-us.html. Accessed Apr 14, 2017.
    1. O’Brien MA,
    2. Rogers S,
    3. Jamtvedt G,
    4. et al
    . Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007;(4):CD000409.
    1. Boom JA,
    2. Nelson CS,
    3. Laufman LE,
    4. Kohrt AE,
    5. Kozinetz CA
    . Improvement in provider immunization knowledge and behaviors following a peer education intervention. Clin Pediatr (Phila). 2007;46(8):706–717.
    OpenUrlCrossRefPubMed
  30. Agency for Healthcare Research and Quality. Former projects. https://effectivehealthcare.ahrq.gov/about/former/#detailing. Accessed Mar 19, 2018.
    1. Curry WJ,
    2. Lengerich EJ,
    3. Kluhsman BC,
    4. et al
    . Academic detailing to increase colorectal cancer screening by primary care practices in Appalachian Pennsylvania. BMC Health Serv Res. 2011;11:112.
    OpenUrlCrossRefPubMed
    1. Bricker PL,
    2. Baron RJ,
    3. Scheirer JJ,
    4. et al
    . Collaboration in Pennsylvania: rapidly spreading improved chronic care for patients to practices. J Contin Educ Health Prof. 2010;30(2):114–125.
    OpenUrlPubMed
  33. Agency for Healthcare Research and Quality. PBRN Resource Center. https://pbrn.ahrq.gov/about/resource-center. Accessed Mar 19, 2018.
  34. Fleischfresser S Wisconsin Medical Home Learning Collaborative. Wisconsin Medical Home Learning Collaborative: a model for implementing practice change. WMJ. 2004;103(5):25–27.
    OpenUrlPubMed
  35. ↵
    1. Henriksen K,
    2. Battles JB,
    3. Keyes MA,
    4. Grady ML
    1. Goeschel CA,
    2. Pronovost PJ
    . Advances in Patient Safety: Harnessing the Potential of Health Care Collaboratives: Lessons from the Keystone ICU Project. In: Henriksen K, Battles JB, Keyes MA, Grady ML, eds. Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 2: Culture and Redesign). Rockville, MD: Agency for Healthcare Research and Quality; 2008.
  36. ↵
    1. Weiner BJ,
    2. Pignone MP,
    3. DuBard CA,
    4. et al
    . Advancing heart health in North Carolina primary care: the Heart Health NOW study protocol. Implement Sci. 2015;10:160.
    OpenUrl
    1. Parchman ML,
    2. Fagnan LJ,
    3. Dorr DA,
    4. et al
    . Study protocol for “Healthy Hearts Northwest”: a 2 × 2 randomized factorial trial to build quality improvement capacity in primary care. Implement Sci. 2016;11(1):138.
    OpenUrl
  38. ↵
    1. Shelley DR,
    2. Ogedegbe G,
    3. Anane S,
    4. et al
    . Testing the use of practice facilitation in a cluster randomized stepped-wedge design trial to improve adherence to cardiovascular disease prevention guidelines: HealthyHearts NYC. Implement Sci. 2016;11(1):88.
    OpenUrl
  39. ↵
    1. Cohen DJ,
    2. Balasubramanian BA,
    3. Gordon L,
    4. et al
    . A national evaluation of a dissemination and implementation initiative to enhance primary care practice capacity and improve cardiovascular disease care: the ESCALATES study protocol. Implement Sci. 2016;11(1):86.
    OpenUrl
  40. ↵
    1. Solberg LI,
    2. Brekke ML,
    3. Fazio CJ,
    4. et al
    . Lessons from experienced guideline implementers: attend to many factors and use multiple strategies. Jt Comm J Qual Improv. 2000;26(4):171–188.
    OpenUrlPubMed
  41. ↵
    1. Solberg LI,
    2. Asche SE,
    3. Margolis KL,
    4. Whitebird RR
    . Measuring an organization’s ability to manage change: the change process capability questionnaire and its use for improving depression care. Am J Med Qual. 2008;23(3):193–200.
    OpenUrlCrossRefPubMed
  42. ↵
    1. Balasubramanian BA,
    2. Cohen DJ,
    3. Clark EC,
    4. et al
    . Practice-level approaches for behavioral counseling and patient health behaviors. Am J Prev Med. 2008;35(5)(Suppl):S407–S413.
    OpenUrlCrossRefPubMed
    1. Cohen DJ,
    2. Balasubramanian BA,
    3. Isaacson NF,
    4. Clark EC,
    5. Etz RS,
    6. Crabtree BF
    . Coordination of health behavior counseling in primary care. Ann Fam Med. 2011;9(5):406–415.
    OpenUrlAbstract/FREE Full Text
    1. Ferrante JM,
    2. Balasubramanian BA,
    3. Hudson SV,
    4. Crabtree BF
    . Principles of the patient-centered medical home and preventive services delivery. Ann Fam Med. 2010;8(2):108–116.
    OpenUrlAbstract/FREE Full Text
    1. Hung DY,
    2. Glasgow RE,
    3. Dickinson LM,
    4. et al
    . The chronic care model and relationships to patient health status and health-related quality of life. Am J Prev Med. 2008;35(5)(Suppl):S398–S406.
    OpenUrlCrossRefPubMed
    1. Jaen CR,
    2. Crabtree BF,
    3. Palmer RF,
    4. et al
    . Methods for evaluating practice change toward a patient-centered medical home. Ann Fam Med. 2010;8 Suppl 1:S9–20; s92.
    OpenUrlAbstract/FREE Full Text
  47. ↵
    1. Balasubramanian BA,
    2. Cohen DJ,
    3. Davis MM,
    4. et al
    . Learning Evaluation: blending quality improvement and implementation research methods to study healthcare innovations. Implement Sci. 2015;10:31.
    OpenUrlCrossRefPubMed
  48. ↵
    1. Akaike H
    . A new look at the statistical model identification. IEEE Trans Automat Contr. 1974;19(6):716–723.
    OpenUrlCrossRef
  49. ↵
    1. Venables WN,
    2. Ripley BD
    . Modern Applied Statistics with S.New York, NY: Springer; 2002.
  50. ↵
    1. Cohen J
    . Statistical Power Analysis for the Behavioral Sciences.Hillsdale, NJ: Lawrence Earlbaum Associates; 1988.
  51. ↵
    1. White IR,
    2. Royston P,
    3. Wood AM
    . Multiple imputation using chained equations: Issues and guidance for practice. Stat Med. 2011;30(4): 377–399.
    OpenUrlCrossRefPubMed
  52. ↵
    1. Nutting PA,
    2. Crabtree BF,
    3. Miller WL,
    4. Stange KC,
    5. Stewart E,
    6. Jaén C
    . Transforming physician practices to patient-centered medical homes: lessons from the national demonstration project. Health Aff (Millwood). 2011;30(3):439–445.
    OpenUrlAbstract/FREE Full Text
  53. ↵
    1. Solberg LI,
    2. Crain AL,
    3. Jaeckels N,
    4. et al
    . The DIAMOND initiative: implementing collaborative care for depression in 75 primary care clinics. Implement Sci. 2013;8:135.
    OpenUrlCrossRefPubMed
  54. ↵
    1. Miller WL,
    2. Crabtree BF,
    3. Nutting PA,
    4. Stange KC,
    5. Jaen CR
    . Primary care practice development: a relationship-centered approach. Ann Fam Med. 2010;8 Suppl 1:S68–79; s92.
    OpenUrlAbstract/FREE Full Text
    1. Lanham HJ,
    2. Palmer RF,
    3. Leykum LK,
    4. et al
    . Trust and Reflection in Primary Care Practice Redesign. Health Serv Res. 2016;51(4): 1489–1514.
    OpenUrlPubMed
  56. ↵
    1. Kaplan HC,
    2. Brady PW,
    3. Dritz MC,
    4. et al
    . The influence of context on quality improvement success in health care: a systematic review of the literature. Milbank Q. 2010;88(4):500–559.
    OpenUrlCrossRefPubMed
  57. ↵
    1. Phillips RL Jr.,
    2. Kaufman A,
    3. Mold JW,
    4. et al
    . The primary care extension program: a catalyst for change. Ann Fam Med. 2013;11(2):173–178.
    OpenUrlAbstract/FREE Full Text
  58. ↵
    Agency for Healthcare Research and Quality. Creating Capacity for Improvement in Primary Care: The Case for Developing a Quality Improvement Infrastructure. Rockville, MD: Agency for Healthcare Research and Quality; 2013. AHRQ publication No. 13-0044-1-EF. https://www.ahrq.gov/professionals/prevention-chronic-care/improve/capacity-building/pcmhqi1.html. Published Apr 2013. Accessed Jun 12, 2017.
  59. ↵
    1. DeVoe J,
    2. Fryer GE Jr.,
    3. Hargraves JL,
    4. Phillips RL,
    5. Green LA
    . Does career dissatisfaction affect the ability of family physicians to deliver high-quality patient care? J Fam Pract. 2002;51(3):223–228.
    OpenUrlPubMed
  60. ↵
    1. Landon BE,
    2. Reschovsky J,
    3. Blumenthal D
    . Changes in career satisfaction among primary care and specialist physicians, 1997–2001. JAMA. 2003;289(4):442–449.
    OpenUrlCrossRefPubMed
  61. ↵
    Centers for Medicare and Medicaid Services. Accountable care organizations (ACOs): general information. https://innovation.cms.gov/initiatives/ACO/. Published 2017. Accessed Nov 3, 2017.
  62. ↵
    1. Song Z,
    2. Lee TH
    . The era of delivery system reform begins. JAMA. 2013;309(1):35–36.
    OpenUrlCrossRefPubMed
  63. ↵
    1. Balasubramanian B,
    2. Cohen DJ,
    3. Crabtree BF,
    4. et al
    . Harmonizing measures for EvidenceNOW - a national initiative to improve cardiovascular disease preventive care and practice capacity. Paper presented at: NAPCRG Annual Meeting; November 13, 2016; Colorado Springs, CO.
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Get Permissions

Jump to section

Related Articles

Cited By...

More in this TOC Section

Similar Articles

Subjects

Keywords