Practice Transformation Under the University of Colorado’s Primary Care Redesign Model
========================================================================================

* Peter Chabot Smith
* Corey Lyon
* Aimee F. English
* Colleen Conry

## Abstract

**PURPOSE** We compared the transformation experience of 2 family medicine practices that implemented the Primary Care Redesign (PCR) team-based model to improve access, quality, and experience without increasing cost. The University of Colorado’s A.F. Williams Family Medicine clinic (pilot practice) implemented the model in February 2015, and a smaller, community-based practice (wave 2 practice) did so 2 years later, in February 2017.

**METHODS** The PCR model increased the ratio of medical assistants to clinicians from about 1:2 to 2.5:1 while expanding the role of the medical assistants, through enhanced rooming procedures, in-room support (eg, scribing), postclinician wrap-up, and in-basket assistance. We assessed access, clinical quality metrics, staffing costs, and clinician and staff experience and burnout for at least 7 months before and 42 months after the intervention.

**RESULTS** In the pilot practice, compared with preimplementation, there were improvements in total appointments and rates of hypertension control, colorectal cancer screening, and most diabetic quality metrics. In the wave 2 practice, total appointments increased slightly when clinicians were added pre-PCR and then increased substantially after implementation; initially variable hypertension control improved rapidly after implementation. The wave 2 practice’s colorectal cancer screening improved gradually, then accelerated postimplementation, while diabetic metrics initially remained stable or declined, then improved postimplementation. New patient appointments began to increase for both practices in late 2015, but grew faster in the pilot practice under PCR. Over time, all experiential domains improved for clinicians; most remained stable for staff. Clinician burnout was reduced by at least one-half in both practices except during low staffing periods, which also adversely affected staff. After a ramp-up period, the number of staff hours per visit remained stable.

**CONCLUSIONS** The PCR model is associated with simultaneous improvements in quality, access, and clinician experience, as well as reductions in burnout, while maintaining staffing costs.

Key words
*   professional practice
*   team-based care
*   quality improvement
*   health care delivery
*   primary care
*   organizational change
*   health information technology
*   practice-based research

## INTRODUCTION

The quadruple aim—improved patient experience, population health, care team well-being, and cost—is recognized as a strategy for improving the US health care system.1 Tackling one element in isolation, however, may produce undesirable consequences in another.2,3 Meanwhile, physician and medical staff burnout represents an existential threat to the primary care workforce. One-half of all primary care physicians report symptoms of burnout, and 1 in 3 would not choose their specialty again.4,5 Physicians spend less than one-third of their time face to face with patients, and nearly one-half their time on the electronic health record (EHR) and administration tasks.6 Modifiable factors contributing to burnout include chaotic, inefficient, and underresourced practices; burdensome documentation requirements; EHR difficulties; misaligned individual and institutional values; and absence of community or meaning derived from work.7–10

Team-based strategies—including expanding the scope of practice of medical assistants (MAs), implementing team-based documentation, and delegating care including routine tests, medication reconciliation, and refills11—can help reduce burnout and move toward the quadruple aim.1 The University of Colorado School of Medicine and the University of Colorado Health system (UCHealth) pilot-tested an expanded MA clinic model adapted from University of Utah’s Care by Design.12 Our model, Primary Care Redesign (PCR, previously APEX/Ambulatory Process Excellence), aims to achieve the quadruple aim without additional per-visit costs through use of advanced team-based strategies. In this article, we report up to 54 months of longitudinal data—including clinical quality metrics, patient access measures, staff and clinician burnout, and staffing costs—for a pilot practice and a comparator (wave 2) practice that implemented the PCR model 2 years later, as well as clinician and staff experience in the pilot practice.

## METHODS

### PCR Model

The PCR model and implementation, including quality and access data 6 months before and after implementation, and quality data in the pilot practice, have been described elsewhere.13,14 In the PCR model, the ratio of MAs to clinicians is increased from about 1:2 to 2.5:1 per clinic session and the role of MAs is expanded. During a 20-minute rooming process, MAs elicit patients’ agendas, update patient histories, reconcile medications, pend refills (ie, enter but do not sign the order), and administer a history of present illness questionnaire and a review of systems questionnaire in the EHR (Epic, v.2012-2017, Epic Systems Corporation). MAs use protocols to address gaps in chronic disease and preventive care, either completing or pending orders for immunizations and certain tests (eg, glycated hemoglobin [HbA1c], monofilament examination, colonoscopy, ophthalmology referral, urine microalbumin). After this rooming process, the MA huddles briefly with his or her clinician, then provides in-room support by scribing and pending additional orders. The MA then stays with the patient to complete clinical tasks (eg, phlebotomy, scheduling future appointments) and escorts the patient from the office. Between patients, MAs help manage clinicians’ EHR inboxes. Practice coaches facilitated PCR implementation in each practice.15–17

### Intervention Setting and Context

The A.F. Williams Family Medicine Residency, located in Denver, Colorado and associated with UCHealth, had about 9,500 active patients with 25,000 office visits per year before pilot-testing PCR in February 2015 (“go-live”). Forty-six clinicians, including 18 residents, supplied 11 clinical full-time equivalents (cFTE). The wave 2 practice is a UCHealth-operated, community-based, nonresidency practice composed of 8 family medicine faculty (4.5 cFTE) caring for approximately 5,500 active patients and having about 16,000 annual visits in 2015. The practice increased by 2 cFTE 4 months before their February 2017 go-live.

Before go-live, each practice adjusted clinic schedules to ensure consistent clinician coverage. Some advanced rooming procedures were pilot-tested 1 to 2 months before go-live. Pilot practice faculty received the full model; resident support varied by year of training. After initial PCR implementation, visit lengths in each practice were eventually adjusted to increase access and offset staffing costs by reducing lengths of certain visits (eg, new patient, preventive, and geriatric visits) from 40 to 20 minutes. UCHealth began a growth initiative in late 2015 that affected both practices.

### Metrics and Sources

For analyses, clinicians were physicians in both practices and nurse practitioners functioning as primary care clinicians in the pilot practice; staff were all non-clinician staff members (MAs, nurses, front desk staff, etc). Definitions of specific measures used to evaluate the PCR model are listed in Table 1. Clinical quality metrics were tracked monthly in the EHR starting in July 2014.13 UCHealth primary care representatives previously agreed on definitions based on national standards in support of the National Committee for Quality Assurance’s Patient Centered Medical Home (PCMH) certification. Access data were similarly available. Data were extracted from discrete EHR fields and exported monthly into spreadsheets for analysis. Quality metrics were available until July 2017, after which changes to procedures and definitions precluded consistent reporting. Access data were available through December 2018. Total and new patient appointments and the number of new patient appointments made within 2 days served as proxies for access. Number of staff hours per visit were determined based on attendance tracking by the Kronos Time Solution System (Kronos Inc). A Kronos-derived MA availability measure became available in July 2016.

View this table:
[Table 1](http://www.annfammed.org/content/17/Suppl_1/S24/T1)

Table 1 
Definitions of Measures Used to Evaluate the PCR Model

Different domains of faculty and staff experience in the pilot practice were assessed with questions from existing validated surveys including attitudes about health care teams,18 participatory safety,19 and PCMH process measures.20,21 Burnout was assessed in both practices using the single-item measure from the Physician Worklife Study.22–24 We used a modified Delphi approach to develop PCR-specific questions,25 which were not subjected to formal psychometric testing. Other than burnout, all responses used 5-point Likert scales. Staff burnout assessment began in November 2016. We preserved anonymity among the small number of respondents by not collecting demographics. Questionnaires were administered before go-live then biannually over 42 months via Survey Monkey (SurveyMonkey Inc).

### Analysis

We evaluated quality, access, and cost using statistical process control charts in which a horizontal line derived from a preintervention mean is bounded by upper and lower control limits defined as 3 standard deviations of the preintervention observations (±3σ). Visual tests indicating nonrandom “special cause variation” include at least 6 data points all above or below the mean (shift) or all consecutively going either up or down (trends); values outside ±3σ have been regarded as approximations of significance.26,27

We calculated scaled scores for all survey responses. Negatively worded questions, including burnout, were reverse-coded so that higher scores indicate more favorable outcomes. Pilot practice burnout rates were graphed along with MA availability.

We used Microsoft Excel 2010 (Microsoft Corporation) for all analyses. This quality improvement project was considered exempt from human subjects review by the Colorado Multiple Institutional Review Board.

## RESULTS

Figure 1 illustrates longitudinal access measures for both practices. In the pilot practice, total appointments increased by ≥3σ soon after PCR model go-live, and this improvement was sustained except for slight decreases in July 2016 and July 2017. New patient appointments increased gradually at first, then rapidly after visit lengths were adjusted, peaking at more than 600 per month and remaining at a minimum of 3σ for almost the entire evaluation period. The wave 2 practice’s total number of appointments remained unchanged until an October 2016 expansion. After PCR implementation, they accelerated, increasing up to 44% over the 2015 baseline. New appointments in that practice increased gradually coincident with UCHealth’s growth initiative, then accelerated with expansion and remained at greater than 3σ postimplementation.

![Figure 1](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/17/Suppl_1/S24/F1.medium.gif)

[Figure 1](http://www.annfammed.org/content/17/Suppl_1/S24/F1)

Figure 1 
Patient access: total and new patient appointments, pilot practice vs wave 2 practice.

PCR = Primary Care Redesign.

Note: Vertical lines mark formal start of PCR for each practice. Dashed lines represent the mean before February 2015. Shaded areas represent ±3σ upper and lower control limits.

The pilot practice’s number of empaneled patients increased from 13,545 to 18,402 (36%) between February 2015 and February 2019. Mean panel size per cFTE increased overall from 1,079 to 1,396 (29%), a pattern seen for both faculty (from 1,149 to 1,597; 39%) and residents (from 981 to 1,096; 12%). For the wave 2 practice, total empanelment increased from approximately 5,500 patients in 2015 to 7,800 by December 2018; mean panel size per cFTE remained constant as the practice expanded. After PCR implementation, the number of new patients scheduled within 2 days increased up to threefold and sixfold in the pilot practice and wave 2 practice, respectively, before returning to baseline (Supplemental Figure 1, available at [http://www.AnnFamMed.org/content/17/Suppl_1/S24/suppl/DC1/](http://www.AnnFamMed.org/content/17/Suppl_1/S24/suppl/DC1/)).

In the pilot practice, an improvement in hypertension control coincided with the start of advanced rooming in January 2015, increasing by ≥3σ by May, and persisting throughout the study (Figure 2). Colorectal cancer screening followed a similar pattern (Supplemental Figure 2, available at [http://www.AnnFamMed.org/content/17/Suppl_1/S24/suppl/DC1/](http://www.AnnFamMed.org/content/17/Suppl_1/S24/suppl/DC1/)). Hypertension control in the wave 2 practice improved in late 2015, then worsened noticeably before trending upward immediately after PCR implementation (Figure 2).

![Figure 2](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/17/Suppl_1/S24/F2.medium.gif)

[Figure 2](http://www.annfammed.org/content/17/Suppl_1/S24/F2)

Figure 2 
Clinical quality metrics: hypertension control, pilot practice vs wave 2 practice.

PCR = Primary Care Redesign.

Note: Vertical lines mark formal start of PCR for each practice. Dashed lines represent the mean before February 2015. Shaded areas represent ±3σ upper and lower control limits.

Data on colorectal cancer screening and diabetic measures can be found in Supplemental Figures 2 through 4, respectively, available at [http://www.AnnFamMed.org/content/17/Suppl\_1/S24/suppl/DC1/](http://www.AnnFamMed.org/content/17/Suppl_1/S24/suppl/DC1/). For the wave 2 practice, colorectal cancer screening improved gradually before PCR implementation then sharply postimplementation. Rates of HbA1c values exceeding 9% gradually increased in each practice. Over the study period, the number of diabetic patients increased by 54% in the pilot practice and 31% in the wave 2 practice. Foot and retinal examinations and diabetic blood pressure control improved in the pilot practice after go-live, and 3 diabetic measures improved further after a quality improvement project in late 2016. Four of the wave 2 practice’s diabetic measures remained unchanged or worsened after 2015, then improved postimplementation.

Paid staff hours per visit remained fairly constant despite the more than doubling of the number of MAs (Supplemental Figure 5, available at [http://www.AnnFamMed.org/content/17/Suppl_1/S24/suppl/DC1/](http://www.AnnFamMed.org/content/17/Suppl_1/S24/suppl/DC1/)). A slight upward shift (less than 1σ) was observed after the pilot practice achieved full staffing in November 2015. Staffing cost increased for 6 months after go-live in the wave 2 practice, then returned to baseline. Brief increases of 2σ to 3σ were commonly observed in July.

Scaled scores and response rates from the experience surveys are detailed in the Supplemental Tables 1 and 2, available at [http://www.AnnFamMed.org/content/17/Suppl\_1/S24/suppl/DC1/](http://www.AnnFamMed.org/content/17/Suppl_1/S24/suppl/DC1/). Grouped experiential domains are shown in Figure 3. Clinician experience improved from baseline after initial declines. Efficiency, communication, and coordination fell initially for staff. All domains except communication and coordination eventually returned to or exceeded the staff’s high baseline scores. Over time, dichotomized clinician burnout in the pilot practice declined from a baseline of 56% to 23.5% (May 2017) and 25% (July 2018) (Figure 4). Worse burnout was observed during periods of low MA availability, especially for staff. The prevalence of clinician burnout in the wave 2 practice fell from a baseline of 40% in November 2016 to 0% (May, November 2017) and 16% (July 2018). This practice’s staff burnout also fell, from a baseline of 42% to 21.4% (May 2017), 33% (November 2017), and 21.4% (July 2018).

![](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/17/Suppl_1/S24/F3/graphic-3.medium.gif)

[](http://www.annfammed.org/content/17/Suppl_1/S24/F3/graphic-3)

![](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/17/Suppl_1/S24/F3/graphic-4.medium.gif)

[](http://www.annfammed.org/content/17/Suppl_1/S24/F3/graphic-4)

Figure 3 
Experience: pilot practice clinician and staff experience by domain.

aNegatively worded items were reverse coded so higher values are desirable for all items including burnout. UCHealth began measuring staff burnout in November 2016.

Note: All questions except that for burnout used 5-point Likert scales for level of endorsement (not at all, slightly, somewhat, mostly, completely) or agreement (strongly disagree, disagree, neither agree nor disagree, agree, strongly agree).

![Figure 4](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/17/Suppl_1/S24/F4.medium.gif)

[Figure 4](http://www.annfammed.org/content/17/Suppl_1/S24/F4)

Figure 4 
Burnout: pilot practice clinician and staff burnout vs MA availability.

C = clinician; MA = medical assistant; S = staff.

Note: Left axis/bars represent burnout. Staff burnout (marked S) was first evaluated in November 2016. Solid bars indicate not burned out; textured bars indicate burned out. Right axis/dotted line represents MA actual/expected minutes per month. January 2015 represents the clinician baseline before go-live.

## DISCUSSION

In this evaluation of the PCR model, we observed sizeable and sustained improvements in clinical quality metrics and patient access, an improved practice experience for clinicians, an overall good experience for staff, and, with adequate staffing, a 50% reduction in clinician burnout despite the increases in volume required to keep per-visit costs stable.

Transferring time-consuming EHR work to more (and more capable) MAs permitted shortening of longer visits which led to marked improvements in access. A growing population, increasingly insured under the Affordable Care Act, and UCHealth’s growth initiative also likely contributed. Although both practices began to grow in late 2015, the degree of growth in each appeared to be higher with PCR in place. The eventual slight receding of new patient appointments likely reflects practices responding to the impact of unrestrained growth on competing priorities (continuity, established patient access, panel size). The panels of the pilot practice faculty increased by one-third. Similar growth was absorbed in the wave 2 practice by expanding the practice and adding clinicians. Once clinician panels reach a sustainable maximum, continued growth will require new forms of transformation, some of which (virtual visits, telehealth, and ancillary outreach) are already underway. This considerable growth appeared to be acceptable to clinicians as reflected in improved experience and burnout scores. Periodic drops in volume during the month of July are likely related to reduced demand and clinician vacations, as well as educational demands on faculty and longer appointments for new interns. More aggressive management of supply and demand could mitigate brief increases in staffing costs at these times.

Despite these practices’ rapid and sustained growth, multiple quality metrics improved concurrently. In some cases, modest absolute changes reflected dramatic improvements relative to baseline; for example, an 8% absolute increase in colorectal cancer screening reflected 19 standard deviations in the pilot practice. This level of improvement, sustained over several years across several measures in the context of practice growth, is particularly notable.

The causes of observed quality improvements were likely multifactorial, including better discrete data entry in the EHR, an increase in the number of guideline-concordant blood pressure measurements by moving this task to the end of rooming, delegation of certain care elements to the MAs via protocols, and improved shared decision making prompted by MA-pended orders and facilitated by clinicians being less distracted by EHR tasks.

Hypertension control and colorectal cancer screening began to improve 1 to 2 months before go-live in the pilot practice. We hypothesize this was due to early initiation of rooming protocols before the formal start date. Slight decreases in hypertension control and colorectal cancer screening in 2016 may have resulted from increased staff turnover at that time. Diabetic control worsened gradually in both practices from early on, a trend that may reflect the large increases in diabetic populations, case finding, or national trends.28

Although PCR is designed to improve the quality of relationships between clinician and patient, it does not directly address other factors in diabetes control such as medication cost or clinical inertia. Diabetes process measures improved remarkably beginning in September 2016 in response to a quality improvement project, suggesting that, once mature, PCR may create a particularly robust platform for focused quality improvement without reversing other improvements. The observed 54% increase in diabetic patients in the pilot practice is unsurprising given both the large influx of new patients and a local diabetes prevalence of 9.3%.29 This increase may also reflect better case finding through more reliable team-based screening workflows.

Over time, clinician experience improved in all domains, including burnout. The literature on primary care practices as complex adaptive systems suggests that adaptive capacity will improve as team function improves.30 Whether improved attitudes about teams in PCR are a cause or effect of improvement in other domains is unclear. Regardless, we conceived of improved team function as a critical element of the model, supported by the presence of a practice facilitation coach.

Baseline staff experience was initially high and, after a period of decline, mostly returned to baseline. The intervening decline may indicate that, unlike the consistent cohort of clinicians, ongoing MA turnover resulted in fluctuating respondents. Given the high baseline level, this trend may also have represented regression to the mean. We suspect, however, it was most likely due to the initial burden of change experienced by the staff, as well as increased responsibility and relational complexity in a growing cohort, as suggested by low scores for scope of practice and falling coordination and communication scores.

Our study has several important limitations. Comparison of the 2 practices functioned as a natural experiment in the form of an interrupted time series, which helped control for secular trends. In general, improvements observed in close proximity to implementation of the PCR model in one practice did not occur simultaneously in the other. This design, however, did not permit controlling for other important confounders such as case mix, unidentified contemporaneous events, or conduct of the pilot intervention in a large university residency clinic. Postimplementation improvements seen in the wave 2 practice do seem to suggest that the benefits of PCR are not limited to such training programs. Both practices are hospital outpatient departments with faculty clinicians, many of whom do not have full patient panels. The model’s financial viability is predicated on the hospital’s estimated per-visit margin. These factors may limit generalizability. Only 7 months of preimplementation data were available; 10 to 20 months of observation would have been more robust. Finally, intermittently low response rates could have resulted in biased clinician experience scores.

In our study, PCR was associated with considerably improved clinician burnout, reducing baseline rates in both practices by more than one-half, despite substantial increases in appointments. This finding likely reflects perceived improvements across clinician experience such as practice efficiency, team functioning, and both documentation and inbox assistance. The hospital’s substantial investments may also have improved clinician perceptions of cultural alignment, as seen in improved “we are in it together” and “patients first” attitudes. Maintaining adequate staffing appears to be crucial for improving burnout in this model, especially among staff. PCR appeared to mitigate burnout most effectively when MA availability was at or near 90%.

The gradual improvement of clinician experience and fluctuating burnout over 42 months reflects important lessons for transformation. Although the experience of staff remained largely favorable, they appeared to be more adversely effected by periods of low staffing than clinicians. Unlike a recent study of Lean workflow redesign that found an inverse relationship between staff and clinician burnout,31 we found staff and clinician burnout to be directly related. Team-based transformation is slow and difficult: things get worse before they get better, vigilance is required, and collaboration is key (“we are in it together”).

In conclusion, in our study, the PCR model of team-based primary care transformation was associated with simultaneous and sustained improvements in indicators of access to care, practice growth, quality of clinical care, and clinician experience, along with a halving of clinician burnout when staffing levels were maintained. After full implementation, staffing-related costs remained unchanged. Further research on PCR and related team-based care models is needed, especially in other settings.

## Footnotes

*   Conflicts of interest: authors report none.

*   To read or post commentaries in response to this article, see it online at [http://www.AnnFamMed.org/content/17/Suppl\_1/S24](http://www.AnnFamMed.org/content/17/Suppl_1/S24).

*   **Funding support:** This research was funded by the University of Colorado Department of Family Medicine.

*   **Disclaimer:** The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the University of Colorado or UCHealth.

*   **Previous presentations:** Lyon C, English AF, Smith PC, How well does the University of Colorado’s APEX model of primary care achieve the quadruple aim?; 2016 NAPCRG Annual Conference; November 15, 2016; Colorado Springs, Colorado; and Lyon C, English AF, Smith PC, If you build it, the patients will come: outcomes of a primary care model redesign; 2016 STFM Conference on Practice Improvement; December 1-4, 2016; Newport Beach, California; Sieja A, Smith PC, Practice transformation: A novel care model that reduces provider burnout and demonstrates high quality patient care using top of scope care and integrated EHR workflows, SGIM Annual Conference; April 11-14, 2018; Denver, Colorado.

*   **Supplementary materials:** Available at [http://www.AnnFamMed.org/content/17/Suppl\_1/S24/suppl/DC1/](http://www.AnnFamMed.org/content/17/Suppl_1/S24/suppl/DC1/).

*   Received for publication August 31, 2018.
*   Revision received April 26, 2019.
*   Accepted for publication May 15, 2019.


*   © 2019 Annals of Family Medicine, Inc.

## References

1.  Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014;12(6):573–576.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTIvNi81NzMiO3M6NDoiYXRvbSI7czoyOToiL2FubmFsc2ZtLzE3L1N1cHBsXzEvUzI0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

2.  Luig T, Asselin J, Sharma AM, Campbell-Scherer DL. Understanding implementation of complex interventions in primary care teams. J Am Board Fam Med. 2018; 31(3): 431–444.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMzEvMy80MzEiO3M6NDoiYXRvbSI7czoyOToiL2FubmFsc2ZtLzE3L1N1cHBsXzEvUzI0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

3.  Khan S, Vandermorris A, Shepherd J, et al. Embracing uncertainty, managing complexity: applying complexity thinking principles to transformation efforts in healthcare systems. BMC Health Serv Res. 2018; 18(1): 192.
    
    

4.  Kane L, Peckham C. Medscape physician compensation report 2014. Medscape. [http://www.medscape.com/features/slideshow/compensation/2014/public/overview#24](http://www.medscape.com/features/slideshow/compensation/2014/public/overview#24). Published Apr 15, 2014. Accessed May 30, 2017.
    
    

5.  Shanafelt TD, Hasan O, Dyrbye LN, et al. Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014. Mayo Clin Proc. 2015; 90(12): 1600–1613.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.mayocp.2015.08.023&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=26653297&link_type=MED&atom=%2Fannalsfm%2F17%2FSuppl_1%2FS24.atom) 

6.  Sinsky C, Colligan L, Li L, et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med. 2016; 165(11): 753–760.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=27595430&link_type=MED&atom=%2Fannalsfm%2F17%2FSuppl_1%2FS24.atom) 

7.  Yarnall KSH, Østbye T, Krause KM, Pollak KI, Gradison M, Michener JL. Family physicians as team leaders: “time” to share the care. Prev Chronic Dis. 2009; 6(2): A59–A64.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=19289002&link_type=MED&atom=%2Fannalsfm%2F17%2FSuppl_1%2FS24.atom) 

8.  Rothenberger DA. Physician burnout and well-being: a systematic review and framework for action. Dis Colon Rectum. 2017; 60(6): 567–576.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1097/DCR.0000000000000844&link_type=DOI) 

9.  Willard-Grace R, Hessler D, Rogers E, Dubé K, Bodenheimer T, Grumbach K. Team structure and culture are associated with lower burnout in primary care. J Am Board Fam Med. 2014; 27(2): 229–238.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjcvMi8yMjkiO3M6NDoiYXRvbSI7czoyOToiL2FubmFsc2ZtLzE3L1N1cHBsXzEvUzI0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

10. Olayiwola JN, Willard-Grace R, Dubé K, et al. Higher perceived clinic capacity to address patients’ social needs associated with lower burnout in primary care providers. J Health Care Poor Under-served. 2018; 29(1): 415–429.
    
    

11. Sinsky CA, Willard-Grace R, Schutzbank AM, Sinsky TA, Margolius D, Bodenheimer T. In search of joy in practice: a report of 23 high-functioning primary care practices. Ann Fam Med. 2013;11(3):272–278.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTEvMy8yNzIiO3M6NDoiYXRvbSI7czoyOToiL2FubmFsc2ZtLzE3L1N1cHBsXzEvUzI0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

12. Egger MJ, Day J, Scammon DL, Li Y, Wilson A, Magill MK. Correlation of the Care by Design primary care practice redesign model and the principles of the patient-centered medical home. J Am Board Fam Med. 2012; 25(2): 216–223.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjUvMi8yMTYiO3M6NDoiYXRvbSI7czoyOToiL2FubmFsc2ZtLzE3L1N1cHBsXzEvUzI0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

13. Smith PC, Brown Levey SM, Lyon C. Evaluating transformation with available resources: the influence of APEX on depression screening. Fam Syst Health. 2017; 35(2): 238–247.
    
    

14. Lyon C, English AF, Chabot Smith P. A team-based care model that improves job satisfaction. Fam Pract Manag. 2018; 25(2): 6–11.
    
    

15. Crabtree BF, Nutting PA, Miller WL, McDaniel RR, Stange KC, Jaen CR, Stewart E. Primary care practice transformation is hard work: insights from a 15-year developmental program of research. Med Care. 2011; 49(Suppl): S28–S35.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1097/MLR.0b013e3181cad65c&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=20856145&link_type=MED&atom=%2Fannalsfm%2F17%2FSuppl_1%2FS24.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000208623700005&link_type=ISI) 

16. Lanham HJ, McDaniel RRJ Jr., Crabtree BF, et al. How improving practice relationships among clinicians and nonclinicians can improve quality in primary care. Jt Comm J Qual Patient Saf. 2009; 35(9): 457–466.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=19769206&link_type=MED&atom=%2Fannalsfm%2F17%2FSuppl_1%2FS24.atom) 

17. Miller WL, Crabtree BF, Nutting PA, Stange KC, Jaén CR. Primary care practice development: a relationship-centered approach. Ann Fam Med. 2010;8(Suppl 1):S68–S79, S92.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTM6IjgvU3VwcGxfMS9TNjgiO3M6NDoiYXRvbSI7czoyOToiL2FubmFsc2ZtLzE3L1N1cHBsXzEvUzI0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

18. Heinemann GD, Schmitt MH, Farrell MP, Brallier SA. Development of an attitudes toward health care teams scale. Eval Health Prof. 1999; 22(1): 123–142.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1177/01632789922034202&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10350960&link_type=MED&atom=%2Fannalsfm%2F17%2FSuppl_1%2FS24.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000078953900007&link_type=ISI) 

19. Anderson NR, West MA. Measuring climate for work group innovation: development and validation of the team climate inventory. J Organ Behav. 1998; 19(3): 235–258.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1002/(SICI)1099-1379(199805)19:3<235::AID-JOB837>3.0.CO;2-C&link_type=DOI) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000073949500002&link_type=ISI) 

20. Fernald D, Hall T, Montgomery L, et al. Colorado Residency PCMH Project: results from a six-year transformation effort. J Am Board Fam Med. 2019; May 21.
    
    

21. Hall TL, Holtrop JS, Dickinson LM, Glasgow RE. Understanding adaptations to patient-centered medical home activities: The PCMH adaptations model. Transl Behav Med. 2017; 7(4): 861–872.
    
    

22. Williams ES, Konrad TR, Linzer M, et al., SGIM Career Satisfaction Study Group, Society of General Internal Medicine. Refining the measurement of physician job satisfaction: results from the Physician Worklife Survey. Med Care. 1999; 37(11): 1140–1154.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1097/00005650-199911000-00006&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10549616&link_type=MED&atom=%2Fannalsfm%2F17%2FSuppl_1%2FS24.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000083447700006&link_type=ISI) 

23. Rohland BM, Kruse GR, Rohrer JE. Validation of a single-item measure of burnout against the Maslach Burnout Inventory among physicians. Stress Health. 2004; 20(2): 75–79.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1002/smi.1002&link_type=DOI) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000221060600004&link_type=ISI) 

24. Dolan ED, Mohr D, Lempa M, et al. Using a single item to measure burnout in primary care staff: a psychometric evaluation. J Gen Intern Med. 2015; 30(5): 582–587.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-014-3112-6&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=25451989&link_type=MED&atom=%2Fannalsfm%2F17%2FSuppl_1%2FS24.atom) 

25. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000; 32(4): 1008–1015.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1046/j.1365-2648.2000.01567.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11095242&link_type=MED&atom=%2Fannalsfm%2F17%2FSuppl_1%2FS24.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000165405100046&link_type=ISI) 

26. Benneyan JC, Lloyd RC, Plsek PE. Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care. 2003; 12(6): 458–464.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjEyLzYvNDU4IjtzOjQ6ImF0b20iO3M6Mjk6Ii9hbm5hbHNmbS8xNy9TdXBwbF8xL1MyNC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

27. Cheung YY, Jung B, Sohn JH, Ogrinc G. Quality initiatives: statistical control charts: simplifying the analysis of data for quality improvement. Radiographics. 2012; 32(7): 2113–2126.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1148/rg.327125713&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=23150861&link_type=MED&atom=%2Fannalsfm%2F17%2FSuppl_1%2FS24.atom) 

28. National Impact Assessment of the Centers for Medicare & Medicaid Services (CMS) Quality Measures Report. Baltimore, MD: US Department of Health and Human Services, Centers for Medicare & Medicaid Services; 2018. [https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures/National-Impact-Assessment-of-the-Centers-for-Medicare-and-Medicaid-Services-CMS-Quality-Measures-Reports.html](https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures/National-Impact-Assessment-of-the-Centers-for-Medicare-and-Medicaid-Services-CMS-Quality-Measures-Reports.html). Accessed Feb 22, 2019.
    
    

29. State of American well-being: 2017 state and community rankings for the prevalence of diabetes. Sharecare. [https://wellbeingindex.sharecare.com/reports/](https://wellbeingindex.sharecare.com/reports/). Published Nov 2018. Accessed Feb 22, 2019.
    
    

30. Chase SM, Crabtree BF, Stewart EE, et al. Coaching strategies for enhancing practice transformation. Fam Pract. 2015; 32(1): 75–81.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1093/fampra/cmu062&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=25281823&link_type=MED&atom=%2Fannalsfm%2F17%2FSuppl_1%2FS24.atom) 

31. Hung DY, Harrison MI, Truong Q, Du X. Experiences of primary care physicians and staff following lean workflow redesign. BMC Health Serv Res. 2018; 18(1): 274.