INTRODUCTION

Team-based care models are broadly recognized as important to primary care reform efforts.1,2,3,4,5,6 While new models of care delivery may be associated with changes in quality of care as well as patient and provider experience,7, 8 the transition to team-based care is complicated.9,10,11,12,13,14,15,16,17 Understanding local contextual factors influencing application of team-based models in primary care is likely to be important to their implementation. The ability to successfully respond to practice change, or capacity for change, measured as practice adaptive reserve, is often based upon relationships—both among team members and between team members and patients.18,19,20 Team characteristics such as work experience may affect a practice’s capacity for change,13, 21, 22 can be associated with team staffing or turnover,23 and can further influence the success of model implementation.

The Cleveland Clinic Health System (CCHS), a large, integrated health system, in 2013–2014 implemented team-based models redefining roles for medical assistants (MAs) and licensed practical nurses (LPNs) for groups of physicians at diverse practice sites (urban, rural, and suburban). Usual care at these sites included one medical assistant assigned to each physician for a clinic session. In “team-care (TC),” two MAs or LPNs assigned to one physician performed team documentation (scribing) and administrative work. In a second model, or “modified team-care (MTC),” an additional MA assigned to a group of three physicians (for a total of 4 MAs per team) helped with clinical workflow or administrative work. Each practice site had at least one assigned nurse care coordinator (NCC) focused on outreach to medically complex patients. Physicians participated voluntarily in these new models which improved access by offering an additional three patient slots per half day per physician in TC or per group of three physicians in the MTC model. In 2016, there were 14 physicians at 7 sites participating in TC and 8 physicians at 3 sites in the MTC model. Our goal was to identify characteristics in the practice settings that contributed to uptake of these new models.

METHODS

We performed a comparative case study of nine primary care practices where physicians implemented either (or both) of these models using a convergent mixed methods approach24 (Fig. 1). Qualitative data collection included field observations and interviews. Quantitative data collection included survey administration, analysis of quality metrics, and patient panel comorbidity scores. Data were collected simultaneously then merged and interpreted.

Figure 1
figure 1

Convergent mixed methods study design.

Full size image

Qualitative Data Collection and Analysis

Physician leaders at each practice were informed about the study through presentations and electronic communication. Field observations and informal interviews were conducted by a physician-researcher (AMH) and a research nurse (JF) over 4 h at practices with at least one physician and team practicing in a new model. Field notes were recorded independently using an iteratively created observation template. Follow-up observations were conducted at seven of the nine sites based on provider agreement.

To understand practice characteristics that may be unrelated to the practice model, employees and patients of providers practicing in the new models as well as usual care were invited to participate in informal and formal interviews. Separate interview guides for employees and patients were created, with questions about primary care experiences, perception of practice changes, unique practice characteristics, relationships, and thoughts about the future of primary care. All interview participants gave verbal consent. Patients were offered a $10 gift card for formal interview participation, with no incentive offered to employees. We interviewed physicians, at least one of their team MAs or LPNs, NCC, and a convenience sample of patients with up to two patients per team. The formal interviews were audio-recorded and professionally transcribed.

Field notes and informal interviews were coded by AMH and reviewed by JF and MBM. Formal interview transcripts were coded and analyzed by AMH and MBM. Themes from field notes and interviews were derived by AMH using an editing style qualitative analysis,25, 26 with themes identified as they emerged from the data, reviewed by MBM and discussed. The discoveries from the field observations and interviews were then organized using a framework method of analysis.27 Validation included (1) initial findings presented to members of a primary care stakeholder panel comprised of patients and primary care employees (physicians, research nurse, nurse care coordinator, medical assistants, quality administrator, program manager) and (2) thematic results audited by researchers not directly involved in the line-by-line coding (AP, DA and KS). Qualitative analyses were performed using NVivo software, version 11.28

Quantitative Data Collection and Analysis

To understand practice staff adaptive reserve and burnout, we created a survey including a three-item Adaptive Reserve Measure20 and the Maslach Burnout Inventory (MBI).29 The survey was distributed electronically using REDCap30 in November–December 2016 (Time 0), then at a 6-month follow-up in June 2017 (Time 1) to 813 (Time 0) and 738 (Time 1) outpatient primary care employees including physicians, nurse practitioners, physician assistants, MAs, LPNs, nurses, and administrative staff at all 37 sites in the CCHS. Three reminders followed each initial request. Mean Adaptive Reserve scores and MBI scores for emotional exhaustion, depersonalization, and personal accomplishment were calculated.

Routinely collected data on provider performance on Accountable Care Organization (ACO) Quality Measures31 was obtained, reported as percentage of a provider’s eligible patients meeting criteria for the measure. Preventive care measures for influenza and pneumococcal vaccination, colorectal cancer screening, and mammography were obtained. Chronic disease measures were obtained for diabetes control (glycosylated hemoglobin (Hba1c) < 8%) and hypertension control (blood pressure < 140/90). Mean scores for each measure were calculated for providers at each site for 2016 then tabulated as mean rate of compliance for each measure by site. To assess variation in patient disease severity at each site, patient panel comorbidity was estimated by Charlson scores32 and reported as a mean for the group of providers by site.

We used means, standard deviations and percentiles to describe continuous measures, and frequencies and percentiles to summarize categorical measures. The Pearson’s chi-square tests, Fisher’s exact tests, Cochran-Armitage trend tests, or Mantel-Haenszel chi-square tests were used to assess associations between categorical factors. Analysis of variance (ANOVA) models were performed to evaluate the relationship between the continuous outcomes and categorical group variables. Variable significance was tested using Tukey/Bonferroni to adjust for multiple comparisons. All tests were performed at a significance level of 0.05. SAS 9.4 software (SAS Institute, Cary, NC) and R 3.4 software were used for analyses. The quantitative comparisons had a power of 80% to detect effect size of 2.2 in mean scores of adaptive reserve between the groups of practices.

Mixed Methods Analysis

Practices were classified into groups based upon qualitative analyses. In the merged analysis, the quantitative data was then compared across the practices and practice groupings. General linear mixed effects models were created to assess the association between the summarized measures and group, where the correlation within the same site was taken into consideration. This protocol was approved by the institutional review board at Cleveland Clinic.

RESULTS

Our analyses (1) classify practices based on their degree of uptake of the team-based care model, (2) describe themes influencing care model uptake and depict their relationship, and (3) present quantitative analyses that provide context for interpreting these qualitative findings.

Qualitative Field Observations and Interviews

In total, we conducted practice observations and informal interviews at 9 sites, follow-up observations at 7 sites, and 75 in-depth interviews including 19 physicians (13 practicing in team-based models, 5 in usual care at the same sites, 1 was a key informant), 18 MAs/LPNs, 8 NCC, and 30 patients.

Three themes initially emerged from field observations and informal interviews: flexible roles/practice modifications, practice challenges, and relationships, with subsequent open coding of the 75 in-depth interviews yielding 5 major themes that encompassed the initial themes: practice characteristics related to response to change, flexible workflow as related to team roles, relationships, practice challenges, and future of primary care.

Variability in Observed Uptake to Team-Based Models and Practice Site Groupings

A key finding that was apparent from the initial field observations was the variability in uptake of the new team-based models as originally intended. Sites were eventually classified into groups based upon the level of uptake of the new models. The level of uptake was discerned from our extensive data analysis that included (1) our initial observations of fidelity to the originally proposed model, (2) our informal interviews during the observation phase, and (3) our formal interviews. As we synthesized our data and the themes emerged through analysis, we concluded that the observed fidelity was defined by differences in two of the five emergent themes: practice characteristics related to response to change and flexible workflow as related to team roles. From observed differences in these two themes, the following three group classification structures for the practices emerged: group 1: high uptake = sites 1, 3, 4, and 6; group 2: partial uptake = sites 5, 8, and 9; and group 3: lower uptake = sites 2 and 7. High vs. low responsiveness to change and high vs. low flexibility related to team roles differentiated high vs. low uptake practices. During further qualitative analysis, the strength of local leadership and stable staffing emerged as mediators to these two themes in their effects on model uptake. Low uptake practices lacked both strong local leadership and stable staffing; the partial uptake practices lacked either strong leadership or stable staffing whereas high uptake practices had both (Fig. 2). Example quotes supporting the two major differentiator themes are described below.

Figure 2
figure 2

Differentiating characteristics of practices influencing uptake of new delivery models.

Full size image

Practice Characteristics as Related to Response to Change

Group 1—high uptake

Group 1, observed to have the highest uptake and to maintain the greatest fidelity to the initial implementation of TC or MTC, was highly responsive to change. As a physician in group 1 (practice site 1) commented,

“… the way I see it, you’ve got to be on the move…things are rapidly evolving in … healthcare and you’ve got to remain nimble.”

A NCC at practice site 6 stated,

“So change, in my opinion, is very good. I think it shows growth. It shows an investment in the area”

and a patient (practice site 6) noted,

“So everything is constantly, it’s changing in a good way.”

The role of leadership in supporting change was highlighted by a physician in this group (practice site 3):

“I think we’ve had a lot of advantages in just the attitude and personality of our people, and certainly the tone has been set by our leadership of ‘Hey. Let’s figure this out, instead of having somebody else tell us how to do it.”

This data suggests that viewing change positively and support from leadership could contribute to maintaining high uptake of new models. In addition, stability of team MA staffing was observed in this group, with a consistent number of available staff to participate in the team-based models.

Group 2—partial uptake

Group 2 practices were observed to respond to change through adapting the implementation of the models to meet their needs. This included varying the roles of MAs/LPNs, especially related to the amount of time they spent in the patient exam room as scribes (often based upon patient’s presenting complaint) vs. administrative work. As one LPN (practice site 9) commented, highlighting the ability to adapt the original team-based model,

“I think as a whole, our group of individuals do pretty well with change and adapt as we go along.”

Physicians adapted their own workflow including scheduling and documentation to utilize the available MAs/LPNs efficiently. At one site, two of the physicians prepared ahead of time for the MA scribe visit documentation to maximize face time with patients. In this group, unstable staffing, such as provider or MA turnover or call-offs, presented challenges and required adaptation of the model. Of note, in two of the practices, teams discontinued participation in a team-based model because of staffing issues. Again, however, the importance of needing the support of leadership in adapting to change was highlighted by a NCC (practice site 8):

“I think we will see a lot of positive come out of this, working together… I think we will have some pushback, and I think we’re gonna need support from leadership to really help this be successful.”

These qualitative findings suggest the importance of leadership support, stable staffing, and positive view of adapting to change to maintain model uptake.

Group 3—lower uptake

For the group 3 sites, staffing changes were a major barrier to adoption of the new models as initially implemented. For example, at one of these sites, implementation ceased for almost a full year because of MA staffing and other providers stopped practicing in a team-based model prior to the start of the study. The other site experienced significant physician turnover. However, lower interest in responding to changes may have also played a role, as one physician commented (practice site 2),

“I’m too old-fashioned…don’t want MA in the room typing.”

Flexible workflow as related to team roles

Group 1—high uptake

Practices successful in model implementation demonstrated willingness among team members to adapt to new roles. For example, an MA in group 1 (practice site 6) stated:

“…I basically do everything. I work with the doctors. I’ll room patients and we’ll do the backup for the patients. I also help on phones… I also do front desk…”

Group 2—partial uptake

A physician in group 2 (practice site 8) highlighted the need for changing team roles, in this case for the MA, based upon a patient’s presenting symptom, where the choice may be to limit the role of the MA in certain situations:

“The MAs, they’ve been trained to review Family Medical History, Past Medical Histories. They’re supposed to review the Medication List, Smoking History… and then they can take a brief, subjective exam…It works well if it is only a known diagnosis… but if a patient comes in with a problem, that doesn’t work as well.”

Group 3—lower uptake

In contrast, more rigid MA roles, and less flexibility, were observed in group 3 practices. For example, in practice site 2, one MA was described as spending 75% of her time on paperwork for the providers, and 25% on covering staffing needs.

However, team support was still noted in this group, as a physician (practice site 7) noted:

“I could say some of the stuff I’m doing was what a Social Worker should be doing, and now we seem we have much more support in terms of doing that.”

Appendix Table 1 presents further examples supporting the axis of differentiating thematic results.

Three additional themes were similar across all of the practices in the study including relationships, practice challenges, and thoughts on the future of primary care. Team members valued relationships, both among team members and with patients, as a physician in group 1 stated, “part of the reason you went into primary care is the relationships.” Perceptions of practice challenges were similar and included pressures for external reporting taking away from face time with patients, interruptions, and competing demands in clinic and staffing challenges. Additional appointment slots, particularly when the added patients were not known to the physician, after visit work, care delivery for complex patients including cost, transportation, and access to specialists were challenges. As noted by an employee in group 2, “so maybe our primary care clinicians have to do more things that they used to maybe refer to specialty.” Perceptions of the future of primary care included hope for more team-based care delivery, improved care for patients, and concern about meeting the demands of day-to-day practice to deliver high-quality care. As a physician in group 1 mentioned, “I think we’re starting to recognize the value in Primary Care in delivering higher quality care at lower cost…”

Additional illustrations of these three themes are presented in Appendix Table 2.

Patient Perceptions

Patient opinions were similar across sites. Patients were satisfied with their care, either did not notice significant changes in care delivery or frequently felt the changes were positive. Even in lower uptake practices, the changes in team roles were viewed positively by patients, as noted in the following exchange at practice site 7:

Interviewer: “Do you like that there are two workers in the room at one time? Does that bother you, or would you rather be with the physician alone?”

Patient: “No. It doesn’t, because it’s some things he needs, so she’s right there… The team is where she’s working with him, and sometimes she’s on the computer and sometimes she’s not, but he might need a question answered, and she got the answer for him.”

Patients valued shorter wait times, time and relationship with their physician, and involvement of other team members. They noted issues of health care cost and role of insurance. Thoughts on the future of primary care were related to convenience as well as maintaining the physician-patient relationship.

Stakeholder Panel Input

Insights from the initial field observations and interviews were confirmed with the stakeholder panel at meetings in October 2016 and April 2017. Many patient members had not noted significant changes in care delivery, valued time spent with the physician, and smooth workflow procedures and reported an overall acceptance of team-based care.

Survey

After excluding employee respondents who reported not working in primary care, there were 276 respondents (response rate 35%) at time 0 and 238 at time 1 (response rate 33%). Characteristics of survey respondents are shown in Table 1 at time 0 and time 1.

Table 1 Characteristics of Survey Respondents
Full size table

There was no significant difference in results of the mean adaptive reserve scores and mean MBI scores for the nine qualitative observation sites and no significant differences in patient panel comorbidity scores among the nine sites (Appendix Table 3). Results of linear mixed effects models of Adaptive Reserve and MBI scores both at time 0 and time 1 by uptake group are displayed in Table 2. There were no significant differences between high, partial, or low uptake practices in Adaptive Reserve estimates at either time point except for slightly higher MBI emotional exhaustion estimates at time 1 compared to time 0 in the low uptake group 3 compared to the partial uptake group 2. Across practices, no significant differences in MBI scores were noted between physicians or MAs and LPNs who worked in a scribe model compared to those who did not either in time 0 or time 1 (Appendix Table 4).

Table 2 Linear Mixed Effects Models: Adaptive Reserve and Maslach Burnout Scores by Group
Full size table

Quality Scores

The ACO metrics for influenza and pneumococcal vaccination, colorectal cancer screening, and mammography were higher for group 1 practices compared to groups 2 or 3 (Table 3). The percentage of patients who had controlled diabetes was higher in group 1 compared to groups 2 and 3 while control of hypertension was not better in group 1. For the hypertension metric, group 2 scored best while performance for groups 1 and 3 were similar (Table 3).

Table 3 Accountable Care Organization (ACO) Quality Metrics 2016 by Group
Full size table

Mixed Methods Interpretation

A visual display of qualitative and quantitative results is presented in Table 4 summarizing the relative performance on the two differentiating qualitative themes and quality metrics by model uptake grouping.

Table 4 Relative Performance on Qualitative Themes and Quantitative Metrics by Uptake Grouping Level
Full size table

DISCUSSION

In this mixed methods analysis, we identified an emergent classification structure of primary care practices defined by level of uptake of team-based models. The importance of practice characteristics related to response to change and ability to adapt workflow/team roles were key themes that captured the successful uptake of new models. Local leadership and stable staffing were essential components of mediating model uptake. Support from leadership was perceived as important to sustaining acceptance to the change in practice models and promoted an attitude of responsiveness to change. Lack of stable staffing often contributed to the need for physicians to opt out of the new models and did not allow for flexibility of work roles. Practice characteristics reflected the practice site and not the care delivery model, as similar observation and interview themes were noted across the TC, MTC, and usual care models. Perceptions of the value of relationships, practice challenges, and the future of primary care appeared to reflect the current state of primary care practice overall, and were not related to degree of uptake of the models.

Cohen et al. identified four key elements in a model for practice change in primary care including (1) motivation of key stakeholders, (2) resources for change, (3) outside motivators, and (4) opportunities for change.33 In our work, strong local leadership served as motivated key stakeholders, and stable staffing was a resource for change, with both of these influencing responsiveness to change, and allowing for flexible team roles. Outside motivators, specifically the need for improved practice efficiency, and opportunities to change, were similar across sites.

The level of model uptake was not explained by practices’ adaptive reserve, provider burnout, and patient comorbidities. The slight increase in emotional exhaustion scores in the low uptake group on the 6-month survey suggests the need for further exploration of whether factors that prevent model uptake may also contribute to provider burnout. Interestingly, MBI scores were overall moderate to high for personal accomplishment, suggesting internal motivation to succeed in all practice groupings. Quality scores were also similar among the sites. These quantitative results suggest generally well-functioning practices providing quality care.

However, performance on preventive care and diabetes disease control measures was stronger in practices where team-based model uptake was observed to be the highest. This suggests that perhaps the same qualities required to maintain new models of care are required to deliver higher-quality clinical care. While the high-uptake group did not perform better on the hypertension metric, the partial uptake group did perform well. The ability to successfully make modifications in each setting may also help with model success.34

The importance of studying context in evaluating success of implementation efforts is now well-recognized.35,36,37 Our finding of individual practice characteristics and response to change as well as flexibility with workflow and team roles as the important factors in successful implementation of team based-primary care is consistent with previous findings,9 and the association of team-based primary care with quality of care has also been described.38

While our study was limited to one health system, the diversity of practice sites included may allow our findings to be further generalizable. Where practice efficiency is the primary goal, operational leaders should consider assessment of practices as to the presence and engagement of local leadership, understanding that stable staffing that can adapt team roles, and the ability to respond to change are of paramount importance in implementing new care delivery models. Quality metrics may also be used to identify practices with characteristics that predispose certain teams to high uptake of new models. Such practices may make ideal test sites.

CONCLUSION

Uptake of team-based models at the practice level can be powerfully influenced by a practice’s ability to respond to change and to adapt workflow and team roles, affected by local leadership and stable staffing. Practices with high uptake of new models also performed better overall on quality metrics. Our findings can inform expectations for operational and policy leaders seeking to implement change in primary care practices.