Care Practices to Promote Patient Engagement in VA Primary Care: Factors Associated With High Performance
=========================================================================================================

* David A. Katz
* Chaorong Wu
* Erin Jaske
* Greg L. Stewart
* David C. Mohr

## Abstract

**PURPOSE** Patient engagement has been broadly defined as the process of actively involving and supporting patients in health care and treatment decision making. The aim of this study was to identify organizational factors that are associated with greater use of patient engagement care practices in Veterans Health Administration primary care clinics.

**METHODS** We conducted a cross-sectional analysis of data from the 2016 Patient-Aligned Care Team (PACT) national survey of direct care clinicians (primary care clinicians, registered nurses, and clinical associates). Exploratory factor analysis was used to group conceptually related patient engagement survey items into 3 subscales: planning and goal setting; motivational interviewing; and organizational strategies to promote self-management. Our independent variables included literature-based factors reported to promote team-based care and interdisciplinary collaboration in primary care. We used generalized estimating equations with multivariate logistic regression analysis to identify independent correlates of high performance on each patient engagement domain (top 25th vs bottom 25th percentile).

**RESULTS** A total of 2,478 direct care clinicians from 609 clinics completed all patient engagement items in the PACT survey. For all patient engagement sub-scales, respondents at high-performing clinics were more likely to report having regular team meetings to discuss performance improvement and having leadership responsible for implementing PACT. For 2 of 3 patient engagement subscales, high performance was also associated with having fully staffed PACT teams (≥3 team members per primary care clinician) and role clarity.

**CONCLUSIONS** Several desirable organizational and contextual factors were associated with high performance of patient engagement care practices. Strategies to improve the organizational functioning of primary care teams may enhance patient engagement in care.

Key words
*   patient participation
*   patient-centered care
*   health care administration
*   professional practice
*   primary care redesign
*   organizational change
*   health services research
*   patient care team
*   veterans
*   practice-based research

## INTRODUCTION

Patient engagement is a relationship-based approach that builds trust and promotes patient autonomy by actively involving and supporting patients in health care and treatment decision making.1 There is growing recognition that patient engagement is a key component of a high-performing, cost-efficient health care system.2 Patients who are actively engaged in their care are more likely to adhere to treatment, to perform regular self-monitoring, to have better intermediate health outcomes (eg, hemoglobin A1c levels, blood pressure),3 and to report better mental health and physical functioning.4

In 2010, the Veterans Health Administration (VHA) launched the Patient-Aligned Care Team (PACT) initiative, which is based on the medical home model and aims to provide continuous, comprehensive, and coordinated team-based care to veterans while improving access to first-contact care.5 The PACT model considers the patient to be an active member of the primary care team and places a high priority on involving and actively supporting patients (and their families) in addressing the patient’s own treatment goals.6 Multiple patient-related, organizational, and societal factors influence patients’ ability to engage in their own care,7 whereas relatively little is known about the relation-ship between organizational factors and use of patient engagement strategies in primary care.

Developing and using patient engagement care processes is a first step in engaging patients in their own care and has been championed by key stakeholders within the VHA. In our conceptual model, team-based care is a key driver of the use of patient engagement care processes and may directly affect patients’ level of engagement (Figure 1). Factors related to patients, team members, and workload may moderate the influence of team-based care on use of patient engagement care processes. During the first 2 years of the PACT initiative, Helfrich et al8 demonstrated that several measures of team functioning and effectiveness were significantly associated with staff-reported improvements in patient-centered care. Building on this work, the study reported here aimed to determine which organizational and contextual factors are associated with greater use of patient engagement practices in primary care. In particular, this snapshot of patient engagement care practices and their correlates 6 years after the start of PACT may help to identify modifiable features of primary care teams for future quality improvement interventions.

![Figure 1](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/18/5/397/F1.medium.gif)

[Figure 1](http://www.annfammed.org/content/18/5/397/F1)

Figure 1 
Conceptual framework of patient engagement within VA primary care.

VA = Veterans Affairs.

The domains highlighted in bold are the focus of this study. The dotted lines indicate potential direct effects of team-based care and patient, workload-related, and team member characteristics on patient engagement (independent of their effects on use of engagement care processes). Double-headed arrows indicate potential bidirectional relationships between patient engagement– related outcomes, patient engagement, and use of engagement practices.

a Examples include improved clinician-patient communication, patient satisfaction with care, and adherence to recommended care.

## METHODS

### Overview

We analyzed cross-sectional data from a national survey of PACT team members in 2016 to monitor systemwide progress toward goals of the patient-centered medical home; prior analyses of data from this survey have examined staff perceptions of care delivery during PACT implementation.9–12 Survey questions were developed by a multidisciplinary Technical Advisory Group of field and subject matter experts, as well as the VHA Healthcare Analysis and Information Group. A link to the anonymous, voluntary web-based survey was disseminated via e-mail from the VA (Veterans Affairs) Office of Clinical Operations and Management to network and facility directors, who in turn distributed the link to PACT personnel. No incentives for survey completion were provided. This work is part of a larger quality improvement effort in the VHA and is not considered research activity that is subject to institutional review board review or waiver.

### Study Participants

Building on the teamlet concept originally proposed by Bodenheimer et al,13,14 PACT “teamlets” were com-posed of a primary care clinician, a registered nurse (RN) care manager, a clinical associate (licensed practical nurse [LPN] or medical assistant [MA]), and an administrative associate (clerk).15 Each teamlet was expected to care for a panel of approximately 1,200 veterans for each primary care clinician full-time equivalent (900 veterans for each nurse practitioner [NP] and physician assistant [PA]).15 To address the project goals, the source population included 4,122 direct care clinicians (primary care clinicians, RN care managers, and LPNs/MAs) who completed the PACT national survey, as these health professionals are most familiar with patient engagement care processes used in VHA primary care.4 For this reason, and because of concerns regarding missing patient engagement survey data, clerks and PACT “neighbors” (pharmacists, social workers, dieticians, and mental health counselors, who typically serve multiple teamlets) were excluded. To further ensure data quality, we ultimately included only the 2,478 direct care clinicians who answered all 13 of the patient engagement survey items (60% of eligible population).

### Patient Engagement Care Practices (Dependent Variables)

The PACT national survey included questions about the use of care processes and strategies to promote patient engagement in primary care. Specifically, the VHA has promoted a “whole health” system of care that emphasizes the importance of planning and goal setting,16 developed and implemented training programs in motivational interviewing for PACT clinicians,17 and promoted the use of shared medical appointments and/or group visits,18,19 peer support,20,21 and home telehealth22 to encourage self-management.15 Response choices for most patient engagement items were Yes, No, and Don’t know (unless indicated otherwise). One item asked participants to indicate which organizational strategies were used at their facility (“Check all that apply”). Conceptually related patient engagement items were grouped to create 3 independent subscales: (1) patient-planning and goal-setting practices (PLAN, 5 items), (2) motivational interviewing techniques (MI, 2 items), and (3) organizational strategies to promote self-management (ORG, 6 items) (Table 1). We identified high- and low-performing clinics by aggregating patient engagement subscale scores at the clinic level: high-performing clinics were defined as those with aggregate patient engagement scores exceeding the 75th percentile and low-performing clinics were those with scores less than the 25th percentile (Supplemental Table 1, available at [https://www.AnnFamMed.org/content/18/5/397/suppl/DC1/](https://www.AnnFamMed.org/content/18/5/397/suppl/DC1/)).

View this table:
[Table 1](http://www.annfammed.org/content/18/5/397/T1)

Table 1 
Patient Engagement Subscales, Based on Selected Items in the 2016 PACT Survey

### Clinic Organizational Factors (Independent Variables)

The PACT survey captured several organizational and contextual factors that have been associated with team-based care and interdisciplinary collaboration in the literature (see Supplemental Table 2, available at [https://www.AnnFamMed.org/content/18/5/397/suppl/DC1/](https://www.AnnFamMed.org/content/18/5/397/suppl/DC1/), for wording of each item). These factors were staffing ratio5,15,23; staff turnover11,24; role clarity25,26; daily team huddles11,12,15,25,27; regular team meetings to discuss performance improvement (an indicator of quality improvement culture)26; leadership responsible for implementing PACT5,23,25; review of performance reports; burnout11, 12, 23; working to the top of one’s license12; and psychological safety.28

### Covariates

We used data from the VA Outpatient Care File and Patient Treatment File, VA administrative data (Primary Care Management Module), and the 2016 Survey of Healthcare Experiences of Patients to capture several patient covariates that can affect the demand on primary care teams, the use of patient engagement strategies, or both: age, non-White race, marital status, income category (tertiles), modified medical comorbidity score using outpatient and inpatient *International Classification of Diseases, Ninth Revision, Clinical Modification* (ICD-9-CM) codes from fiscal years 2014-2015,29 self-reported health status from the Survey of Healthcare Experiences of Patients (single item: excellent, very good, good, fair, or poor),30 and longitudinal continuity with the assigned VA primary care clinician using the modified, modified continuity index.31 All patient covariates were aggregated to the clinic level (mean for continuous variables, proportion for binary variables). We also adjusted for respondents’ clinician type (team role), VA tenure, and practice setting (community-based outpatient clinic vs hospital-based) in all regression models, as prior studies of the PACT survey have shown that these variables are associated with nonresponse.10 Workload covariates included excessive workload modeled as the proportion of primary care clinicians with adjusted panel size exceeding capacity (more than 1,200 patients for full-time primary care clinicians); average panel size for clinicians was adjusted for full-time equivalents and clinician type (ie, physician vs NP and PA).

### Statistical Analysis

We examined the factor structure of the patient engagement survey items using exploratory factor analysis and performed confirmatory factor analyses to determine model fit (eg, root mean square error of approximation, comparative fit index). We compared respondents’ ratings of clinic organization and team-based care at high- vs low-performing clinics for each patient engagement subscale in bivariate analyses. To assess collinearity, we examined the correlations between individual candidate variables and calculated variance inflation factors. We used generalized estimating equations with multivariate logistic regression analysis to identify independent correlates of high performance on each patient engagement subscale at the respondent level, accounting for the clustering of respondents within the same clinic. We assumed an exchangeable correlation matrix. Using a best subsets approach, we identified the best-fitting model from among all possible combinations of the candidate independent variables; model fit was assessed using the quasi-likelihood under the independence model criterion. In all models, we adjusted for respondent characteristics, as well as patient and workload characteristics at the clinic level (as described above).

To address missing covariate data, we used multiple imputation, which assumes that the data are missing at random (ie, the probability of missing data depends only on the observed data and does not depend on the value of the outcome variable itself).32,33 SAS programming language, version 9.3 (SAS Institute Inc), was used for all analyses.

## RESULTS

### Study Sample

The estimated PACT survey response rate was 18.5%. Because the survey was anonymous (with only clinic identifiers) and because it is unknown whether the director of primary care disseminated the questionnaire to some or all PACT team members at any given facility, the precise denominator is unknown.

Derivation of our sample for analysis is shown in Figure 2. Compared with those who completed the VA All Employee Survey in 2016, the 4,122 direct care clinicians who participated in the PACT survey had similar demographic characteristics, but were more likely to be primary care clinicians and team leaders (Table 2). Compared with the 2,478 eligible PACT team members with complete patient engagement survey data (analysis sample), the 1,644 with incomplete patient engagement data were more likely to be clinical associates. The analysis sample included survey respondents from 61% of VHA primary care clinics (609 of 993) in 2016.

![Figure 2](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/18/5/397/F2.medium.gif)

[Figure 2](http://www.annfammed.org/content/18/5/397/F2)

Figure 2 
Derivation of analysis sample.

View this table:
[Table 2](http://www.annfammed.org/content/18/5/397/T2)

Table 2 
Characteristics of Respondents to the 2016 PACT Survey of Primary Care Personnel

### Factor Analysis

We next conducted factor analysis, as new measures of patient engagement care practices were developed for this study. Our analysis confirmed that the concept of patient engagement care practices is multidimensional. Specifically, we identified a 3-factor solution for patient engagement that showed acceptable model fit on confirmatory factor analysis (root mean square error of approximation = 0.073, comparative fit index = 0.93), after deleting 1 item that cross-loaded on all 3 domains. The reliability of internal consistency was good for the PLAN and MI subscales, but was marginal for the ORG subscale; corresponding Cronbach α values were 0.75, 0.90, and 0.67, respectively. Item-scale correlations showed good discrimination between the 3 subscales (Supplemental Table 3, available at [https://www.AnnFamMed.org/content/18/5/397/suppl/DC1/](https://www.AnnFamMed.org/content/18/5/397/suppl/DC1/)).

### Main Results

Possible scores on the 3 patient engagement subscales ranged from 0 to 100, with higher values indicating a higher level of the patient engagement care practice. Mean scores (and standard deviations) among the 2,478 PACT team members with complete patient engagement survey data were 70.6 (31.7) for the PLAN subscale, 50.5 (47.2) for the MI subscale, and 47.3 (26.0) for the ORG subscale.

In bivariate analyses, clinic organizational characteristics related to team-based care were significantly associated with higher PLAN, MI, and ORG scores, except that there was no significant difference in staff turnover between the top and bottom quartiles of the ORG subscale (Table 3). With regard to patient characteristics, clinics in the highest vs lowest quartile of PLAN had a higher mean patient income ($36,596 vs $33,923, *P* = .01) and higher proportions of married patients (61% vs 58%, *P* = .02) and patients in fair or poor health (41% vs 38%, *P* = .02). Clinics in the highest quartile of MI scores had a lower proportion of married patients (56% vs 61%, *P* <.05) and higher disease burden (comorbidity score = 0.77 vs 0.64, *P* <.05) (Supplemental Table 4, available at [https://www.AnnFamMed.org/content/18/5/397/suppl/DC1/](https://www.AnnFamMed.org/content/18/5/397/suppl/DC1/)). Respondents in community-based outpatient care settings were significantly more likely than hospital-based respondents to report use of PLAN practices (71% vs 49%), but less likely to report use of MI practices (64% vs 81%) (Table 3).

View this table:
[Table 3](http://www.annfammed.org/content/18/5/397/T3)

Table 3 
Characteristics of Respondents and Their Clinics in Top vs Bottom Quartile for Each Patient Engagement Subscale

Best subsets regression analysis showed that several characteristics were associated with patient engagement care processes: respondents who reported having fully staffed PACT teams, daily team huddles, regular team meetings to discuss performance improvement, and leadership responsible for implementing and maintaining PACT were more likely to score in the top quartile of PLAN scores (Table 4). Respondents who reported having clearly defined roles, regular team meetings, and a responsible leadership structure were more likely to score in the top quartile of MI scores. Similar results were observed for the ORG subscale, except that having fully staffed PACT teams was also significantly associated with high performance.

View this table:
[Table 4](http://www.annfammed.org/content/18/5/397/T4)

Table 4 
Association Between Respondent and Clinic Characteristics and Top Quartile of Each Patient Engagement Subscale (vs Bottom Quartile)

## DISCUSSION

### Findings in Context

In this cross-sectional study, use of patient engagement care practices was highly variable across VHA primary care clinics. Lower use of these practices may be related to clinicians’ perceptions that some elements of the patient-centered medical home may not be entirely relevant (or may be difficult to use) for particular groups of patients during visits.34

Direct care clinicians who endorsed the presence of desirable organizational attributes that facilitate effective team functioning reported high levels of performance of patient engagement practices at their assigned outpatient clinics. Our findings are consistent with the conceptual framework of high-performing primary care proposed by Bodenheimer et al,35 who posited that the partnership between patients and team members depends on successful implementation of 4 building blocks: empanelment (ie, linking each patient to a care team and primary care clinician), data-driven performance improvement, engaged leadership, and team-based care. Especially important in primary care, team-based care involves building relationships with patients over time and between members of the practice team.36

Our study showed that having a fully staffed team was significantly associated with use of patient-planning and organizational strategies to promote patient engagement. Although the VHA mandated that each primary care teamlet include 3 full-time equivalent staff per full-time equivalent primary care clinician, many primary care clinics have faced chronic under-staffing and the need to cross-cover other teamlets.37,38 Inadequately staffed teams and high turnover contribute to burnout of VHA primary care personnel10 and make it difficult to establish relationships among team members, reconfigure roles and responsibilities, and develop new work processes, including those related to patient engagement.39

Our results show that role clarity is a key determinant of the use of motivational interviewing and organizational patient engagement strategies. Working in a chaotic clinic environment with vague team boundaries is associated with higher barriers to providing patient-centered care.8 Clear expectations of each team member’s duties and responsibilities are needed to optimize the team’s efficiency, to promote interdisciplinary collaboration,37,40 and to enable the team to accomplish more than the sum of its parts.41 For example, MAs can be trained to assume an expanded role as health coaches; however, in addition to developing general ground rules, the MA and clinician need to briefly discuss the clinical goals for a given encounter and negotiate who will do what with regard to patient engagement during and after the clinic visit.13,35

We also found that participation in daily huddles, responsible PACT leadership, and meetings to discuss performance improvement were associated with greater use of patient engagement care practices. In a qualitative study of 27 primary care practices, team huddles were reported to assist in role delegation, to improve the consistency of information collected from patients, and to provide a framework for more structured communication among team members.25

Responsible leadership is essential to foster a shared vision for practice transformation, to provide time and resources for team development,38,40 and to enable team members to attain shared goals and to suggest new ideas.42,43 In a study of practice leadership, the odds of implementing tools for patient self-management were significantly greater for primary care practices with higher leadership scores.44 Quality and performance data are essential for good clinical leadership, and regular team meetings to review these data can help to sustain quality improvement initiatives.45

Although lower levels of burnout and greater psychological safety have been identified as correlates of high-functioning teams,12,41 we did not find a significant association between these variables and use of patient engagement care practices in multivariate analysis, possibly because the variance of the latter is better explained by team characteristics that are more tightly linked operationally to patient engagement.

### Limitations

Limitations of this study warrant discussion. First, we conducted a cross-sectional analysis and are unable to assess causality. Second, the results may be affected by participation bias, although the demographics and VA tenure of PACT survey respondents were similar to those of primary care personnel who completed the VHA All Employee Survey. Third, measures of patient engagement practices were based on self-report and thus were subject to socially desirable response bias; however, because the PACT survey was administered anonymously, participants may have been more willing to provide honest responses. Fourth, common method bias may potentially inflate the association between variables, which were collected simultaneously in the same survey.46 We note, however, that common method bias has typically been more of a concern for Likert-type ratings of attitudes; most of the survey items in the current analysis were based on factual observations of clinic and organizational factors (with Yes or No responses); in addition, independent and dependent variables were conceptually unrelated.

Fifth, we were unable to assess patient engagement practices at the teamlet level (eg, whether motivational interviewing was used for a specific primary care clinician’s panel). Most of the variables that we examined relate to the overlying practice context, however, and transcend teamlet boundaries. Sixth, the PACT survey did not capture other important dimensions of team-based care (eg, the shared mental model,47,48 participatory decision making by clinic staff4,49), but we accounted for a large set of organizational variables that are pertinent to the patient-centered medical home. Seventh, the analysis sample was missing patient engagement and covariate data for a large number of respondents. We used multiple imputation to minimize the loss of respondent data and the potential bias associated with restricting the analysis sample to those with complete data.50 Finally, the results of this study may not be applicable to non-VA primary care clinics. Although the practice context and conceptualization of patient engagement in the VHA may differ somewhat from non-VA primary care, our findings are particularly relevant to other integrated health care systems that have embraced the medical home model.

### Implications

Collectively, our findings suggest that strengthening the foundation of the medical home by promoting full staffing of primary care teams, clearly defined roles of team members, effective leadership, and a practice culture of performance improvement may increase adoption and use of patient engagement practices. Improved team collaboration and coordination are necessary for practices to manage the increasing complexity and unpredictability of clinical care51 and to become more accountable for patient-centered outcomes. Future research should advance our understanding of how the medical home model can promote patient-centered care, using granular measures of patient activation, shared decision making, and self-management at the level of the patient-teamlet dyad.

## Acknowledgments

The authors thank Mary Vaughn-Sarrazin, PhD, for her comments on an earlier draft of the manuscript and Charlotte Dean and Jamie Paul for assistance with manuscript preparation.

## Footnotes

*   Conflicts of interest: authors report none.

*   To read or post commentaries in response to this article, see it online at [https://www.AnnFamMed.org/content/18/5/397](https://www.AnnFamMed.org/content/18/5/397).

*   **Funding support:** This work could not have been done without the support of the VISN 23 Patient Aligned Care Team Demonstration Laboratory, which is funded by the VA Office of Patient Care Services.

*   **Disclaimer:** The funding agency had no role in the current study. The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

*   **Previous presentation:** Presented at the Society of General Internal Medicine Annual Meeting; April 11-14, 2018; Denver, Colorado.

*   **Supplementary materials:** Available at [https://www.AnnFamMed.org/content/18/5/397/suppl/DC1/](https://www.AnnFamMed.org/content/18/5/397/suppl/DC1/).

*   Received for publication September 18, 2019.
*   Revision received January 13, 2020.
*   Accepted for publication January 27, 2020.


*   © 2020 Annals of Family Medicine, Inc.

## References

1.  Grande SW, Faber MJ, Durand MA, Thompson R, Elwyn G. A classification model of patient engagement methods and assessment of their feasibility in real-world settings. Patient Educ Couns. 2014; 95(2): 281–287.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.pec.2014.01.016&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24582473&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

2.  James J. Health policy brief: patient engagement. Health Affairs. [https://www.healthaffairs.org/do/10.1377/hpb20130214.898775/full/healthpolicybrief\_86.pdf](https://www.healthaffairs.org/do/10.1377/hpb20130214.898775/full/healthpolicybrief_86.pdf). Published Feb 14, 2013. Accessed Aug 30, 2020.
    
    

3.  Hibbard JH, Greene J. What the evidence shows about patient acti-vation: better health outcomes and care experiences; fewer data on costs. Health Aff (Millwood). 2013; 32(2): 207–214.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjMyLzIvMjA3IjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xOC81LzM5Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

4.  Shortell SM, Poon BY, Ramsay PP, et al. A multilevel analysis of patient engagement and patient-reported outcomes in primary care practices of accountable care organizations. J Gen Intern Med. 2017; 32(6): 640–647.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-016-3980-z&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=28160187&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

5.  Klein S. Veterans Health Administration: Implementing Patient-Centered Medical Homes in the Nation’s Largest Integrated Delivery System. Washington, DC: The Commonwealth Fund; 2011.
    
    

6.  Chokshi DA, Schectman G, Agarwal M. Patient-centered innovation: the VA approach. Healthc (Amst). 2013; 1(3-4): 72–75.
    
    

7.  Carman KL, Dardess P, Maurer M, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood). 2013; 32(2): 223–231.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjMyLzIvMjIzIjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xOC81LzM5Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

8.  Helfrich CD, Dolan ED, Fihn SD, et al. Association of medical home team-based care functions and perceived improvements in patient-centered care at VHA primary care clinics. Healthc (Amst). 2014; 2(4): 238–244.
    
    

9.  Dolan ED, Mohr D, Lempa M, et al. Using a single item to measure burnout in primary care staff: a psychometric evaluation. J Gen Intern Med. 2015; 30(5): 582–587.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-014-3112-6&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=25451989&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

10. Helfrich CD, Simonetti JA, Clinton WL, et al. The association of team-specific workload and staffing with odds of burnout among VA primary care team members. J Gen Intern Med. 2017; 32(7): 760–766.
    
    

11. Edwards ST, Helfrich CD, Grembowski D, et al. Task delegation and burnout trade-offs among primary care providers and nurses in Veterans Affairs Patient Aligned Care Teams (VA PACTs). J Am Board Fam Med. 2018; 31(1): 83–93.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6NzoiMzEvMS84MyI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTgvNS8zOTcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

12. Helfrich CD, Dolan ED, Simonetti J, et al. Elements of team-based care in a patient-centered medical home are associated with lower burnout among VA primary care employees. J Gen Intern Med. 2014; 29(Suppl 2): S659–S666.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24715396&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

13. Bodenheimer T, Laing BY. The teamlet model of primary care. Ann Fam Med. 2007; 5(5): 457–461.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NzoiNS81LzQ1NyI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTgvNS8zOTcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

14. Bodenheimer T. Building teams in primary care: lessons from 15 case studies. California HealthCare Foundation. [https://www.chcf.org/publication/building-teams-in-primary-care-lessons-from-15-case-studies/](https://www.chcf.org/publication/building-teams-in-primary-care-lessons-from-15-case-studies/). Published Jul 17, 2007. Accessed Dec 30, 2019.
    
    

15. Rosland AM, Nelson K, Sun H, et al. The patient-centered medical home in the Veterans Health Administration. Am J Manag Care. 2013; 19(7): e263–e272.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=23919446&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000322681100010&link_type=ISI) 

16. Office of Patient Centered Care and Cultural Transformation. Whole health. [https://www.va.gov/PATIENTCENTEREDCARE/explore/about-whole-health.asp](https://www.va.gov/PATIENTCENTEREDCARE/explore/about-whole-health.asp). Accessed Aug 30, 2020.
    
    

17. Kinsinger LS. Disease prevention in the Veterans Health Administration. NC Med J. 2015; 76(5): 335–338.
    
    

18. Kirsh S, Watts S, Pascuzzi K, et al. Shared medical appointments based on the chronic care model: a quality improvement project to address the challenges of patients with diabetes with high cardiovascular risk. Qual Saf Health Care. 2007; 16(5): 349–353.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjE2LzUvMzQ5IjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xOC81LzM5Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

19. Edelman D, Gierisch JM, McDuffie JR, Oddone E, Williams JW Jr. Shared medical appointments for patients with diabetes mellitus: a systematic review. J Gen Intern Med. 2015; 30(1): 99–106.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-014-2978-7&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=25107290&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

20. Heisler M, Vijan S, Makki F, Piette JD. Diabetes control with reciprocal peer support versus nurse care management: a randomized trial. Ann Intern Med. 2010; 153(8): 507–515.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.7326/0003-4819-153-8-201010190-00007&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=20956707&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000283054600003&link_type=ISI) 

21. Daaleman TP, Fisher EB. Enriching patient-centered medical homes through peer support. Ann Fam Med. 2015; 13(Suppl 1): S73–S78.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTQ6IjEzL1N1cHBsXzEvUzczIjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xOC81LzM5Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

22. Darkins A, Ryan P, Kobb R, et al. Care coordination/home tele-health: the systematic implementation of health informatics, home telehealth, and disease management to support the care of veteran patients with chronic conditions. Telemed J E Health. 2008; 14(10): 1118–1126.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1089/tmj.2008.0021&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=19119835&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000262473600013&link_type=ISI) 

23. Wu FM, Rubenstein LV, Yoon J. Team functioning as a predictor of patient outcomes in early medical home implementation. Health Care Manage Rev. 2018; 43(3): 238–248.
    
    

24. Sylling PW, Wong ES, Liu CF, et al. Patient-centered medical home implementation and primary care provider turnover. Med Care. 2014; 52(12): 1017–1022.
    
    

25. O’Malley AS, Gourevitch R, Draper K, Bond A, Tirodkar MA. Over-coming challenges to teamwork in patient-centered medical homes: a qualitative study. J Gen Intern Med. 2015; 30(2): 183–192.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-014-3065-9&link_type=DOI) 

26. National Committee for Quality Assurance. PCMH 2014 Standards and Guidelines. [http://www.acofp.org/acofpimis/Acofporg/Apps/2014\_PCMH\_Finals/Tools/2\_PCMH\_Recognition\_2014\_Standards\_and\_Guidelines.pdf](http://www.acofp.org/acofpimis/Acofporg/Apps/2014\_PCMH\_Finals/Tools/2\_PCMH\_Recognition\_2014_Standards_and_Guidelines.pdf). Accessed Oct 14, 2018.
    
    

27. Fogarty CT, Schultz S. Team huddles: The role of the primary care educator. Clin Teach. 2010; 7(3): 157–160.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=21134174&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

28. Edmundson E. Psychological safety and learning behavior in work teams. Adm Sci Q. 1999; 44: 350–383.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.2307/2666999&link_type=DOI) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000081160000005&link_type=ISI) 

29. Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011; 64(7): 749–759.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.jclinepi.2010.10.004&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=21208778&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

30. DeSalvo KB, Fan VS, McDonell MB, Fihn SD. Predicting mortality and healthcare utilization with a single question. Health Serv Res. 2005; 40(4): 1234–1246.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1475-6773.2005.00404.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16033502&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000230292400017&link_type=ISI) 

31. Magill MK, Senf J. A new method for measuring continuity of care in family practice residencies. J Fam Pract. 1987; 24(2): 165–168.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=3806028&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1987F974200013&link_type=ISI) 

32. Raghunathan TE. What do we do with missing data? Some options for analysis of incomplete data. Annu Rev Public Health. 2004; 25: 99–117.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1146/annurev.publhealth.25.102802.124410&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15015914&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000221130600007&link_type=ISI) 

33. Allison P. Multiple imputation for missing data: a cautionary tale. Sociol Methods Res. 2000; 28(3): 301–309.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1177/0049124100028003003&link_type=DOI) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000085121000003&link_type=ISI) 

34. Westfall JM, Zittleman L, Ringel M, et al. How do rural patients benefit from the patient-centred medical home? A card study in the High Plains Research Network. London J Prim Care (Abingdon). 2014; 6(6): 136–148.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=25949735&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

35. Bodenheimer T, Ghorob A, Willard-Grace R, Grumbach K. The 10 building blocks of high-performing primary care. Ann Fam Med. 2014; 12(2): 166–171.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTIvMi8xNjYiO3M6NDoiYXRvbSI7czoyMzoiL2FubmFsc2ZtLzE4LzUvMzk3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

36. Chesluk BJ, Holmboe ES. How teams work—or don’t—in primary care: a field study on internal medicine practices. Health Aff (Millwood). 2010; 29(5): 874–879.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjI5LzUvODc0IjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xOC81LzM5Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

37. True G, Stewart GL, Lampman M, Pelak M, Solimeo SL. Teamwork and delegation in medical homes: primary care staff perspectives in the Veterans Health Administration. J Gen Intern Med. 2014; 29(Suppl 2): S632–S639.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-013-2666-z&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24737223&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

38. Rodriguez HP, Giannitrapani KF, Stockdale S, Hamilton AB, Yano EM, Rubenstein LV. Teamlet structure and early experiences of medical home implementation for veterans. J Gen Intern Med. 2014; 29(Suppl 2): S623–S631.
    
    

39. Forman J, Harrod M, Robinson C, et al. First things first: foundational requirements for a medical home in an academic medical center. J Gen Intern Med. 2014; 29(Suppl 2): S640–S648.
    
    

40. Solimeo SL, Hein M, Paez M, Ono S, Lampman M, Stewart GL. Medical homes require more than an EMR and aligned incentives. Am J Manag Care. 2013; 19(2): 132–140.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=23448110&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

41. Mitchell PM, Wynia R, Golden R, et al. Core Principles & Values of Effective Team-Based Health Care. Discussion Paper. Washington, DC: Institute of Medicine; 2012.
    
    

42. Bohmer RM. Leading clinicians and clinicians leading. N Engl J Med. 2013; 368(16): 1468–1470.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMp1301814&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=23594000&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

43. Butler AE, Wakefield BJ, Sapir NC, et al. VA Demo Lab Initiative Patient Engagement Workgroup Final Report. Internal VHA Report to the Office of Primary Care. Internal report for VHA leadership. 2016.
    
    

44. Donahue KE, Halladay JR, Wise A, et al. Facilitators of transforming primary care: a look under the hood at practice leadership. Ann Fam Med. 2013; 11(Suppl 1): S27–S33.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTQ6IjExL1N1cHBsXzEvUzI3IjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xOC81LzM5Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

45. Bray P, Cummings DM, Wolf M, Massing MW, Reaves J. After the collaborative is over: what sustains quality improvement initiatives in primary care practices? Jt Comm J Qual Patient Saf. 2009; 35(10): 502–508.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=19886089&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 

46. Podsakoff PM, MacKenzie SB, Podsakoff NP. Sources of method bias in social science research and recommendations on how to control it. Annu Rev Psychol. 2012; 63: 539–569.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1146/annurev-psych-120710-100452&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=21838546&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000299709900021&link_type=ISI) 

47. 1.  Castellan NJ
    
    Cannon-Bowers JA, Salas E, Convers S. Shared mental models in expert team decision making. In: Castellan NJ, ed. Individual and Group Decision Making. 1st ed. New York, NY: Psychology Press; 1993.
    
    

48. Hysong SJ, Best RG, Pugh JA, Moore FI. Not of one mind: mental models of clinical practice guidelines in the Veterans Health Administration. Health Serv Res. 2005; 40(3): 829–847.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1475-6773.2005.00387.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15960693&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000229279700012&link_type=ISI) 

49. Ohman-Strickland PA, John Orzano A, Nutting PA, et al. Measuring organizational attributes of primary care practices: development of a new instrument. Health Serv Res. 2007; 42(3 Pt 1): 1257–1273.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1475-6773.2006.00644.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17489913&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000246201400019&link_type=ISI) 

50. Newgard CD, Haukoos JS. Advanced statistics: missing data in clinical research—part 2: multiple imputation. Acad Emerg Med. 2007; 14(7): 669–678.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1197/j.aem.2006.11.038&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17595237&link_type=MED&atom=%2Fannalsfm%2F18%2F5%2F397.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000247776500014&link_type=ISI) 

51. Perez HR, Beyrouty M, Bennett K, et al. Chaos in the clinic: characteristics and consequences of practices perceived as chaotic. J Healthc Qual. 2017; 39(1): 43–53.