Abstract
PURPOSE Assess effectiveness of Primary Care 2.0: a team-based model that incorporates increased medical assistant (MA) to primary care physician (PCP) ratio, integration of advanced practice clinicians, expanded MA roles, and extended the interprofessional team.
METHODS Prospective, quasi-experimental evaluation of staff/clinician team development and wellness survey data, comparing Primary Care 2.0 to conventional clinics within our academic health care system. We surveyed before the model launch and every 6-9 months up to 24 months post implementation. Secondary outcomes (cost, quality metrics, patient satisfaction) were assessed via routinely collected operational data.
RESULTS Team development significantly increased in the Primary Care 2.0 clinic, sustained across all 3 post implementation time points (+12.2, +8.5, + 10.1 respectively, vs baseline, on the 100-point Team Development Measure) relative to the comparison clinics. Among wellness domains, only “control of work” approached significant gains (+0.5 on a 5-point Likert scale, P = .05), but was not sustained. Burnout did not have statistically significant relative changes; the Primary Care 2.0 site showed a temporal trend of improvement at 9 and 15 months. Reversal of this trend at 2 years corresponded to contextual changes, specifically, reduced MA to PCP staffing ratio. Adjusted models confirmed an inverse relationship between team development and burnout (P <.0001). Secondary outcomes generally remained stable between intervention and comparison clinics with suggestion of labor cost savings.
CONCLUSIONS The Primary Care 2.0 model of enhanced team-based primary care demonstrates team development is a plausible key to protect against burnout, but is not sufficient alone. The results reinforce that transformation to team-based care cannot be a 1-time effort and institutional commitment is integral.
INTRODUCTION
For 2 decades health care systems have pursued transitions to team-based primary care, hoping it would support health care’s Quadruple Aim of improving quality, cost, and patient experience while reducing workforce burnout.1,2 There is increased focus on high-functioning team models as a potential solution to address clinician burnout.3 Burnout is harmful to clinicians and health care systems as it predicts increased clinician turnover and high attributable costs.4,5 The Patient Centered Medical Home1 is the most widely adopted and studied archetype. Early experience is often positive, but longitudinal results are mixed. In a second generation of this movement, several health systems (eg, Bellin Health,6-8 University of Colorado Family Medicine,9,10 Cleveland Clinic,11,12 Stanford Primary Care13-15) arrived at similar innovative models that combat the burnout epidemic through increased teamwork. Each transformation optimizes the role of medical assistants (MA) to support the patient and primary care physician (PCP) throughout and beyond the face-to-face visit, via team-based documentation and care coordination.16 In addition to expanded MA roles and greater than 1:1 ratio of MA to full-time clinicians (physicians, nurse practitioners, and physician assistants), these models include increased support from an extended interprofessional team.
Recent work by Sinksky and Bodenheimer highlights the promise from these early adopters.16 While anecdotally successful with reportedly high acceptability8 and suggestions of reduced burnout,10 rigorous evaluation of such models is scarce in the literature.
We present findings from a prospective evaluation of 1 model, Stanford’s Primary Care 2.0 pilot.13,14 We rigorously evaluate the impact of the Primary Care 2.0 model on team development and team member wellness, and explore the association between team development and burnout over time. We hypothesized that implementation of this care design would improve team dynamics, which may be protective against burnout. To capture the effect on the Quadruple Aim,2 we tracked secondary outcomes of quality, cost, and patient satisfaction as well.
The Primary Care 2.0 Model
The Primary Care 2.0 team-based care model, launched in summer 2016, has been previously described13 and includes a mixture of role-based and systems elements intended to support population health, preventive care, and behavioral health goals for patients. Role-based elements include: (1) care teams with pairings of physician (MD) and advanced practice clinician (ie, nurse practitioner or physician assistant) supported by MAs with a goal 2:1 MA to clinician ratio; (2) an expanded role for MAs including scribing, population health management, and between-visit care management; and (3) on-site extended care team members including clinical pharmacist, dietitian, physical therapist, and behavioral health clinician. System elements include: (1) time for care coordination; (2) telehealth; (3) health coaching; and (4) structures that support medical teamwork, such as interprofessional quality improvement activities, daily “huddles,” and case conferences. Medical assistants trained as a group and individually for in-room documentation; full team training included health coaching, motivational interviewing, integrated behavioral health, and mindset training. Physician/advanced practice clinician/MA pods function as cohesive units (with efforts made to keep teams intact, even in times of MA shortages) and meet monthly for PCP-MA feedback and problem solving.
METHODS
Evaluation Design
We performed a quasi-experimental, difference-indifference evaluation of prospectively collected survey data on team development and staff/clinician wellness, comparing the Primary Care 2.0 implementation clinical site to 4 other primary care clinics within our academic health care system (Table 1). Difference-in-difference analysis assumes common trends and responses to external shocks (outside of the intervention being evaluated) and we chose comparison clinics to make this assumption most plausible, with all 5 clinics part of same health care system, being operated in similar ways (apart from the Primary Care 2.0 intervention), under same central leadership, and with physician faculty all being members of a single Primary Care division. All clinics were in a single county and serve primarily private- and Medicare-insured populations.
Description of Clinics Included in Primary Care 2.0 Evaluation
We surveyed staff and clinicians before the launch of Primary Care 2.0 (summer 2016) and 3 more times (every 6-9 months) up to 24 months post implementation. We sought 100% sampling. Secondary outcome analysis was based on data routinely collected and reported by the health care system. This evaluation received a quality improvement, non-human subjects determination by the Stanford School of Medicine Institutional Review Board.
Primary Outcomes
Our primary analysis estimates the impact of Primary Care 2.0 implementation on 2 main outcomes, team development and wellness scores, with particular interest in burnout. Team development was measured using the Team Development Measure (TDM), a 31-question survey with rigorous validation in clinical settings17 that includes 4 domains (cohesion, communication, roles and goals, team primacy). Total scores range from 0-100 mapping to 10 stages of team development (pre-team to fully developed team)18 and were the dependent variable in our analyses. Wellness was measured using a shortened version of the Stanford Professional Fulfillment Index, a validated19 survey already administered regularly and shown to predict physician turnover within our health care system.4 The index has been validated against the widely-used Maslach Burnout Inventory (correlation greater than 0.7 for its measure of overall burnout compared with Maslach’s measures of emotional exhaustion + depersonalization).19 Given our focus on interprofessional teams, we included 16 questions relevant to all staff and clinicians, covering 4 domains (burnout, sense of control, sense of fulfillment [fulfillment], and meaningfulness of work [meaning]), each scored on a 5-point Likert scale. We specifically focused on burnout given our hypothesis and prior evidence that improved team functioning is associated with lower burnout.20 To this end, we ran an additional burnout model that included the TDM as a predictor.
Secondary Outcomes
We obtained aggregate measures for quality, cost, and patient satisfaction from existing data collection and monitoring systems. We assessed quality at 3 yearly time points, beginning soon after Primary Care 2.0 launched, using averages of 3 chronic disease monitoring measures defined by the Healthcare Effectiveness Data and Information Set as proxies for population health management: diabetic hemoglobin A1c testing, diabetic nephropathy screening, and monitoring of patients on angiotensin-converting enzyme inhibitor/angiotensin II receptor blocker medications. These indicators were chosen because they were consistently reported, with adequate denominators (n >200), across all sites and time points. Patient satisfaction was similarly assessed at 3 time points, using the clinic-level percentage of patients who gave highest score for likelihood-to-recommend in the Press Ganey survey (Press Ganey Associates LLC). Costs were assessed using health system financial reporting of total labor cost (staff plus MD and advanced practice clinician services) per patient visit.
Data Analysis
Over 4 time points (baseline, and 9, 15, and 24 months post implementation) we assessed association of the effect implementing Primary Care 2.0 had on team development (total TDM score) and each domain of wellness in the Professional Fulfillment Index (each treated as a continuous score). The other clinics (control group) were surveyed at the same time points.
We used a pre-post, difference-in-difference approach to compare outcomes, examining short-term impacts (baseline vs 9 months post implementation) and sustained results (baseline vs 15 months and vs 24 months post implementation). This granular examination over time was necessary because fidelity to the original Primary Care 2.0 model changed substantially with the MA to PCP ratio of 2:1 reduced to 1.5:1 at 16 months post implementation. The difference-indifference approach allowed us to estimate if changes in our main outcomes were significantly greater in Primary Care 2.0 than the background changes observed over time in the comparison sites. Specifically, we used linear mixed-models (SAS software version 9.4; SAS Institute Inc), with an individual’s random intercept and repeated measures correlated using exchangeable correlation, to assess the association between Primary Care 2.0 and our outcomes. This model effectively handles complete and partial data across individuals; all correlation over individuals and time are accounted for. Variables in the model included the outcome of interest, clinic (implementation vs comparison), and interaction terms of “intervention x time.” The latter estimated the difference-in-difference effect of the intervention at each time point (referenced to baseline). We ran sensitivity models excluding staff/clinicians who only responded in the baseline survey to assess whether estimates differed. We also modeled collapsing the first 2 survey time periods (given similarity of estimates) to gain power. None of these alternative models changed conclusions; thus, only the original models are reported. Based on our hypothesis that team development could be a factor in reducing burnout, we also ran a model (Analysis #2) with burnout as the outcome to explore the relationship, using TDM as our main predictor covariate.
Secondary outcomes of quality and patient satisfaction scores underwent cross-sectional analysis at 4, 12, and 24 months after the launch of Primary Care 2.0, while costs were followed longitudinally from 2016 through 4 fiscal years (September to August). Operational metrics were documented over time as they were only accessible as aggregate measures and therefore not amenable to rigorous modeling analysis.
RESULTS
Over the 4 time points of both the TDM and Professional Fulfillment Index surveys, a total of 188 unique physicians and staff completed surveys, 44 from the implementation clinic and 144 from the comparison clinics. Table 2 shows the number of surveys completed and response rates. Response rates varied from 58% to 96% in the implementation clinic and 49% to 80% at the comparison clinics. At all sites, the majority (52% to 57%) of respondents completed 2 or more surveys; only 2 employees did not complete any surveys.
Surveys Completeda at Each Time Point, by Clinic Group and Survey (N = 188)
Table 3 presents estimated difference-in-difference scores for each primary outcome at each post implementation time point. The Primary Care 2.0 clinic showed a significant increase in team development that was sustained across all 3 post implementation time points relative to the comparison clinics (+12.2, +8.5, +10.1 points vs baseline on the 100-point TDM scale). Overall P value for Team Development was significant at <.001. Wellness domain P values were .46 for burnout, .20 for sense of control, .21 for fulfillment, and .37 for meaning. These P values were determined with a type III test of the fixed effect for the interaction of clinic type and survey time point.
Difference-in-Difference Analysisa of Team Development and Wellness Scores by Time Point
Figure 1 shows estimated average scores at each time point from the same analytic model used for data in Table 3. While the comparison clinics average range remained stable at 56-57 (matching TDM Stage 3: Building Clarity of Roles),18 the intervention site started with a TDM score of 50 (matching TDM Stage 2: Building Communication) and increased to 62 (matching TDM Stage 4: Building Clarity of Goals, Means), reflecting a clinically meaningful 2-stage gain in team development.
Average Team Development Measure scores at each survey time point.
MA = medical assistant; Pre = pre implementation.
Notes: Range bars indicate 95% CIs. The dashed line indicates change in MA to clinician ratio from 2:1 to 1.5:1 at 16 months post implementation.
Adjusted models of the Professional Fulfillment Index (Table 3) show that only the sense of control domain approached statistically significant gains for Primary Care 2.0 in the first (9 month) post implementation period. Improvement at 9 months was moderate (+0.5 on a 5-point Likert scale) but not sustained. While burnout showed no statistically significant improvement at the intervention site, the effect estimates (Table 3) and graph (Figure 2a) suggest a trend of stepwise improvement at 9 and 15 months post implementation (a drop of 0.2 and 0.3, respectively), but a reversal at 2 years. A similar pattern of initial improvement then reversal was observed through small changes (Figure 2b-2d) in the other wellness measures.
Average employee wellness scores, at each survey time point, adjusted for repeated measures.
MA = medical assistant; Pre = pre-implementation.
Notes: Range bars indicate 95% CIs. The dashed line indicates change in MA to clinician ratio from 2:1 to 1.5:1 at 16 months post implementation.
In Analysis #2, examining the association of team development and burnout across all respondents, team development was significantly and inversely associated with burnout. An increase of TDM score of 10 points was associated with a decrease of 0.25 (P <.0001) on the 5-point burnout scale; this difference in burnout (equal to plus 0.26 standard deviation in our baseline populations), albeit small, would represent a clinically meaningful difference based on guidance from the Professional Fulfillment Index19 authors that substantive effect size in their scales are greater than or equal to 0.2 standard deviations. This estimate was consistent in both unadjusted regression and our full model adjusting for time, clinic site, and within-subject correlation. This association does not imply causality.
Routinely collected health system data showed that the Primary Care 2.0 transformation did not decrease performance in cost, quality, or patient satisfaction. At the intervention site, quality measures began higher and remained stably high; the other clinics gradually approached the same high level (Table 4) in the context of increased institutional focus on quality and accountable care. Patient experience measured by Press Ganey remained stable across sites (Table 5). Labor cost per visit was lower at the Primary Care 2.0 site than the other clinics, and cost decreased over the 4 fiscal years following the model’s launch. Increased use of advanced practice clinicians in place of physicians offset the added cost of increased MA staffing and the extended care team.
Quality Metrics Pre/Post Implementation of Primary Care 2.0
Patient Experience Pre/Post Implementation of Primary Care 2.0
DISCUSSION
To our knowledge, this is the first rigorous quasi-experimental evaluation of a “powered up” team-based primary care model, described by Sinsky and Bodenheimer16 that is characterized by increased MA to PCP ratio, in-room support, and extended interprofessional teams. Primary Care 2.0 produced more advanced team development that was sustained for 24 months. Across all individuals surveyed, higher team development scores were significantly associated with lower individual burnout. Unfortunately, the collective gains in team development observed in Primary Care 2.0 did not translate to a statistically significant reduction in burnout, when compared with the control sites. Rather, the initial trend toward reduced burnout reversed at our fourth time point, after MA to PCP staffing ratios were disrupted. With decreased MA staffing, clinic leadership reported that rooming and in-basket tasks monopolized MA time; MAs could not consistently continue in-room documentation, care coordination outside of the visit, or health coaching.
We hypothesize that had the intervention’s original staffing continued, the initial trend in reduced burnout would likely have reached statistical significance. Our observation and hypothesis is consistent with longitudinal findings reported from the University of Colorado’s similar primary care redesign in which “worse burnout was observed during periods of low MA availability.”10
Still, the implementation of Primary Care 2.0 produced several findings suggesting a successful redesign. During this culture transformation, which inherently carries potential for negative impacts, other monitored metrics (cost, quality, patient satisfaction) remained acceptable and within overlapping ranges across sites. The implementation clinic’s labor costs were in the lower end of the range of all clinics, attributable in part to the lower salary costs of advanced practice clinicians compared with physicians. Moreover, previously reported qualitative data14 demonstrated general acceptability and positive patient and staff experience.
We add to a growing body of research examining the association between team-based care and burnout. Research from the Veterans Administration Healthcare Systems’ Patient Centered Medical Home implementation demonstrated that a fully staffed team, participatory decision making, and staff working at top-of-competency levels, predicted less burnout.20 But overall, adoption of Patient Centered Medical Home did not correspond with decreased clinician burnout.21 In a survey of community-based PCPs in New York, the ratio of support staff full-time equivalents (FTEs) to PCP ranged widely (0.2 to 2), but did not correlate with clinician burnout.22 In contrast, other research has shown that the availability of specific support roles (team-based mental health integration,23 in-clinic support to address social needs of patients24) does correlate with reduced PCP burnout.
Our results support proponents of interprofessional teams and enhanced MA roles as a way forward in primary care, especially if reducing burnout is a priority. Though early trends of reduced burnout in the implementation clinic had attenuated by 24 months, our quantitative findings must be interpreted with key contextual elements,25 notably institution-wide cuts resulting in reduced MA to PCP ratios midway through our evaluation. As Drs Sinsky and Bodenheimer argue,16 status quo expectations hinder adoption of these more robust team models of care, requiring a new mindset from leadership. Models with expanded MA roles face challenges to adoption including “leadership and clinician resistance to change, cost of additional MA training, and lack of reimbursement for nonbillable services.”26 Beyond financial support, effective MA engagement is key to model success20 and might improve employee retention. We observed slightly less MA turnover (13% vs 23% in the previous year) at the Primary Care 2.0 clinic vs the other sites.
In the 1 other publication of comparable rigor, a mixed methods comparative case study applied to a similar model (Cleveland Clinic’s team-care redesign), 9 clinics were examined for degree of model uptake and concurrent analysis of clinician burnout. They evaluated sites with MA to PCP ratios of 2:1 (“Team-care”) and 4:3 (“Modified team-care”),11 but did not analyze burnout levels by team-care ratio. Clinics identified with high uptake of team-care performed slightly higher on quality metrics, but showed no significant difference in burnout.11
Limitations
This study’s main limitation is the single site of implementation, which limits generalizability. Our measure of wellness has yet to be validated beyond physicians; however, we believe including all staff in team-based care assessments should be requisite, especially since task-shifting can increase burnout in other members.27 Statistical limitations include the fact that we treated team development and wellness scores as single outcomes, using a P value cut-off of .05, while arguably adjustment for multiple outcomes (eg, Bonferroni) could have altered what would be considered statistically significant. Regardless, the conclusion that TDM was the only change to meet statistical significance would hold (with P <.001). Our sensitivity to detect reduction in burnout attributable to our intervention was likely reduced by broader influences, including wellness efforts across our Primary Care division, and tightened financing and reduced staffing across the health care system. It is also possible that the common trend assumption underpinning difference-in-difference analysis was not met insofar as the intervention site was newer and thus had staff who had not worked together as long. We did not adjust for employee or team duration and this may have biased our findings, but the fact that absolute team development score at the intervention site rose and was sustained well above the other clinics’ scores (while those clinics scores remained stable) suggests the culture created by the intervention supported development of greater team cohesion than would otherwise have occurred.
Future Directions
While teamwork can improve clinical performance across care settings,28 it remains to be seen if it generally reduces burnout. Future primary care redesign evaluation must endeavor, as we have, to clarify this relationship. Next steps include examining if structured teamwork training (eg, the Agency for Healthcare Research and Quality’s TeamSTEPPS29) improves burnout. Finally, our findings reinforce10 the importance of MA support and availability—all future evaluations of primary care redesign and burnout should explicitly consider levels and fluctuations in MA staffing.
CONCLUSION
Enhanced models of team-based primary care (with extended interprofessional support, expanded MA roles, and increased MA staffing) have garnered increasing attention, especially in the context of a burnout epidemic. Our rigorous evaluation of one such model demonstrates team development as a plausible ingredient to prevent burnout; our quantitative findings paired with our qualitative14 observations of Primary Care 2.0 reinforce that transformation to team-based care cannot be a one-time effort and is likely a multiyear culture change endeavor. Steadfast leadership, and commitment to model staffing and time for teamwork activities, are essential to realize the benefits of team-based care.
Acknowledgments
No public funding was used for activities reported in this manuscript. The Evaluation Sciences Unit team is grateful to Jimmy Dang for his support in providing population health data, and Mickey Trockel for support of use of the Professional Fulfillment Index. The project reported in this manuscript was financially supported by Stanford Health Care. The article contents are solely the responsibility of the authors and do not necessarily represent the official views of Stanford Health Care, an entity affiliated with, but not a part of, the Stanford School of Medicine where the Evaluation Sciences Unit resides.
Footnotes
↵* These authors contributed equally as co–first authors.
Annals Journal Club selection; see inside back cover or https://www.AnnFamMed.org/content/AJC/.
Conflicts of interest: authors report none.
To read or post commentaries in response to this article, go to https://www.AnnFamMed.org/content/19/5/411/tab-e-letters.
Funding support: The project was financially supported by Stanford Health Care.
Previous presentations: Preliminary findings from this work were shared in an oral presentation at the Society of General Internal Medicine Conference, April 20, 2017; Washington, DC, and in a poster presentation at the Academy Health Annual Research Meeting; June 26, 2017; in New Orleans, Louisiana.
- Received for publication January 27, 2020.
- Revision received December 8, 2020.
- Accepted for publication February 8, 2021.
- © 2021 Annals of Family Medicine, Inc.
