Research ArticleOriginal ResearchA

A Cluster Randomized Trial of Primary Care Practice Redesign to Integrate Behavioral Health for Those Who Need It Most: Patients With Multiple Chronic Conditions

Benjamin Littenberg, Jessica Clifton, Abigail M. Crocker, Laura-Mae Baldwin, Levi N. Bonnell, Ryan E. Breshears, Peter Callas, Prama Chakravarti, Kelly Clark/Keefe, Deborah J. Cohen, Frank V. deGruy, Lauren Eidt-Pearson, William Elder, Chester Fox, Sylvie Frisbie, Katie Hekman, Juvena Hitt, Jennifer Jewiss, David C. Kaelber, Kairn Stetler Kelley, Rodger Kessler, Jennifer B. O’Rourke-Lavoie, George S. Leibowitz, C. R. Macchi, Matthew P. Martin, Mark McGovern, Brenda Mollis, Daniel Mullin, Zsolt Nagykaldi, Lisa W. Natkin, Wilson Pace, Richard G. Pinckney, Douglas Pomeroy, Paula Reynolds, Gail L. Rose, Sarah Hudson Scholle, William J. Sieber, Jeni Soucie, Terry Stancin, Kurt C. Stange, Kari A. Stephens, Kathryn Teng, Elizabeth Needham Waddell and Constance van Eeghen
The Annals of Family Medicine November 2023, 21 (6) 483-495; DOI: https://doi.org/10.1370/afm.3027

Abstract

PURPOSE Patient outcomes can improve when primary care and behavioral health providers use a collaborative system of care, but integrating these services is difficult. We tested the effectiveness of a practice intervention for improving patient outcomes by enhancing integrated behavioral health (IBH) activities.

METHODS We conducted a pragmatic, cluster randomized controlled trial. The intervention combined practice redesign, quality improvement coaching, provider and staff education, and collaborative learning. At baseline and 2 years, staff at 42 primary care practices completed the Practice Integration Profile (PIP) as a measure of IBH. Adult patients with multiple chronic medical and behavioral conditions completed the Patient-Reported Outcomes Measurement Information System (PROMIS-29) survey. Primary outcomes were the change in 8 PROMIS-29 domain scores. Secondary outcomes included change in level of integration.

RESULTS Intervention assignment had no effect on change in outcomes reported by 2,426 patients who completed both baseline and 2-year surveys. Practices assigned to the intervention improved PIP workflow scores but not PIP total scores. Baseline PIP total score was significantly associated with patient-reported function, independent of intervention. Active practices that completed intervention workbooks (n = 13) improved patient-reported outcomes and practice integration (P ≤ .05) compared with other active practices (n = 7).

CONCLUSION Intervention assignment had no effect on change in patient outcomes; however, we did observe improved patient outcomes among practices that entered the study with greater IBH. We also observed more improvement of integration and patient outcomes among active practices that completed the intervention compared to active practices that did not. Additional research is needed to understand how implementation efforts to enhance IBH can best reach patients.

Key words:

INTRODUCTION

Comprehensive primary care for patients with chronic diseases that might include behavioral factors require targeted behavioral health services.1 Providing care for mental health, substance use disorders, and health behavior changes, and attending to psychosocial factors could decrease the negative health effects of tobacco, diet, physical inactivity, alcohol, and other substances, which together account for 38% of all US deaths,2,3 and prevent additional morbidity from treatment nonadherence, insomnia, anxiety, depression, and stress.4-6 Coordinating behavioral health and primary care services has shown improved outcomes for patients with depression (eg, Collaborative Care Model7) and patients with mixed depression and anxiety.8,9 Integrating primary and behavioral health care management across multiple chronic conditions (MCCs) has shown greater effectiveness than integration focused on a single diagnosis and could benefit patients, families, and the health care system.10-16

Integrated primary care and behavioral health (IBH) is “the care that results from a practice team of primary care and behavioral health clinicians, working together with patients and families, using a systematic and cost-effective approach to provide patient-centered care for a defined population.”17 Integration efforts can include population management, protocol-driven evidence-supported care, guideline-based external referrals, systematic needs identification, team-based care, and practice-wide approaches to patient engagement.17 Less intensive integration can include working with a Behavioral Health (BH) provider in the same building (ie, colocation) without shared scheduling and record systems.17 Integration of these services is known to be efficacious, but quantifying integration is complex, involving evaluation across domains of clinical services, workflow, workspace, integration of providers, patient identification procedures, and methods of engaging patients.18-21 More intensive integration efforts are hypothesized to result in better patient outcomes17 but have not been studied in broadly defined populations.

Practices that attempt to increase IBH often encounter barriers arising from health care system complexity.1 Structured implementation processes can address such barriers22 and some, such as implementation toolkits, have demonstrated innovation uptake in health care.23,24 Study authors (C.vE., R.K., B.L.) developed early versions of such an intervention to help practices enhance IBH by engaging frontline workers in redesigning processes while eliminating inefficient work. This approach was successful in establishing onsite BH in primary care, increasing care initiation, and decreasing time to first BH visit.25 The research team expanded on that approach, incorporating more implementation best practice strategies26,27 to create the current version of the integrated care toolkit tested in the present study.

The purpose of this initiative was to study the effect of a single approach to IBH improvement on outcomes for primary care patients experiencing MCCs and, secondarily, to study the effect on primary care efforts to enhance integration. We evaluated the real-world effectiveness of toolkit access and support in enhancing IBH and outcomes reported by patients with MCCs. We also explored associations between IBH and patient outcomes, toolkit completion and change in IBH, and toolkit completion and change in patient outcomes.

METHODS

Design

Our multidisciplinary research team included members in the fields of psychology, family medicine, internal medicine, clinical and translational science, education, art, biostatistics, communication, public health, bioinformatics, and health care operations, with a variety of backgrounds including clinicians, patients, family members, leaders, and researchers from across the United States. We performed a large-scale, pragmatic, cluster randomized, clinical effectiveness trial. Pragmatic trials speed the translation of research into practice28 by testing interventions under usual, rather than ideal, conditions.29 Primary care practices were randomized to the active or control group. We used an intention-to-treat analysis and conducted a difference-in-differences analysis of BH integration and patient-reported outcomes during the period 2018 to 2020, including the onset of the coronavirus disease 2019 (COVID-19) pandemic. Study methods30 and qualitative findings31 are reported elsewhere and are registered at ClinicalTrials.gov (NCT02868983). The University of Vermont Committee on Human Research in the Medical Sciences (#16-554) and institutional review boards at other participating locations approved the protocol.

Procedures

We recruited primary care practices during the period 2016 to 2018 that had a colocated behavioral health provider (BHP) (ie, psychologist, social worker, or licensed counselor) of ≥0.5 full-time equivalent, capable of billing insurers, using electronic health records, and with scores <75 on the Practice Integration Profile 1.0 (PIP), a measure of IBH.32 The sites included internal medicine, family medicine, small and large groups, for-profit, academic, and safety-net clinics recruited from practice networks across 13 US states using professional networks, association listservs, and conference presentations. Measures of practice integration were collected from 42 randomized practices.

We recruited patient participants from the 41 practices able to provide data on eligible patients. A third party reviewed electronic health records to identify adult patients from each practice with ≥1 chronic medical condition (arthritis; obstructive lung disease including emphysema, chronic bronchitis, or asthma; nongestational diabetes; or heart disease manifested as heart failure or hypertension), ≥1 chronic BH condition (diagnosis related to mood including anxiety or depression; chronic pain including headache, migraine, neuralgia, fibromyalgia, or chronic musculoskeletal pain; insomnia; irritable bowel syndrome; or substance misuse including substance use disorder, tobacco use, or problem drinking), or 3 medical conditions. Patients were randomly selected and contacted by e-mail, telephone, or postal mail with an invitation to join the study.

Intervention

Practices randomized to the active group were required to assemble a team with a facilitator. The intervention provided access to and support for workbooks to guide the planning, redesign, and implementation activities of a quality improvement (QI) project; online education tailored to clinic roles (physician, BHP, nurse, etc); an online learning community; and remote coaching for the team facilitator and QI team. Workbooks included specific tactics, such as care management, use of registries and population reports, and team-based care, known to be beneficial to delivering IBH to patients with MCCs. Given the pragmatic nature of the study, each QI team determined when to start and which elements of the intervention to use within an observation period from 2018 to 2020. An example of the toolkit can be found at https://sites.google.com/view/ibhpc/home.

Measures

Degree of practice integration was measured using the PIP, a survey of staff perceptions of integration32-34 (Supplemental Appendix). Practice Integration Profile total score (range 0-100) is the unweighted average of the following 6 survey domains: workflow, clinical services, workspace, integration of providers, patient identification, and patient engagement. A medical provider, BHP, administrator, and ≥1 additional practice member rated their practice’s performance using the PIP at baseline, midpoint, and 2 years. In addition, coaches monitored active practices’ progress and determined which practices completed workbook activities through implementation of ≥1 suggested practice change in ≤24 months.

Patients who agreed to participate completed the Patient-Reported Outcomes Measurement Information System35 (PROMIS-29) online survey at baseline, midpoint, and 2 years. The PROMIS-29 survey includes 8 domains (physical function, anxiety, depression, fatigue, sleep disturbance, social functioning, pain intensity, and pain interference) and 2 summary scores.36 Other measures included change in empathy,37 medication adherence,38 self-reported health care utilization,39 time lost due to disability,40 functional capacity,41 patient centeredness,42 depression,43 anxiety,44 asthma symptoms,45,46 substance use disorder,47 and problem drinking.48 All measures had published indications of adequate validity and reliability.32,33,35 For all measures, baseline data were collected before the COVID-19 pandemic; final measures were collected after the pandemic start.

Analysis

Intervention Effect on Patients’ PROMIS-29 Scores

Primary outcomes were the changes (baseline to 2-year) in the 8 PROMIS-29 domain scores. The 2 PROMIS-29 summary scores were secondary outcomes. For each domain, we built a multivariate mixed linear regression model with change in outcome from baseline to 2 years as the dependent variable. Practice assignment to the intervention was the independent variable, modeled as a fixed effect, as were baseline level of the outcome measure and potential confounders (see below). A practice identifier was included as a random effect. We did not look for trends at midpoint because the intervention had not been in place long enough to affect outcomes.

We examined the possible confounding effects of 29 characteristics (Tables 1 and 2) on the relation between the intervention and each outcome. We compared the coefficient on the predictor from a mixed model with no fixed effect covariates to that from a model with a single covariate. For each analysis, we retained potential confounders associated with both outcome and predictor with P < .10 and also retained those affecting the association between the predictor and outcome.

View this table:
Table 1.

Patient Characteristics at Baseline

View this table:
Table 2.

Practice Characteristics at Baseline

The study was designed for 90% power to detect differences as small as 2.5 points between active and control arms in any of the 8 PROMIS-29 domain scales, consistent with minimally important differences specified for other PROMIS instruments of 2 to 8 points.49,50 We used the Bonferroni correction to maintain α = 0.05 as we examined the effect of the intervention on each of the 8 PROMIS-29 domains, resulting in a threshold of P = .00625.51 Additional models explored the association between the intervention and patient outcomes in subgroups based on patient characteristics, presence of qualifying conditions, number of qualifying conditions, neighborhood characteristics, and practice characteristics. These analyses were exploratory rather than hypothesis testing and were not adjusted for multiple comparisons.

Intervention Effect on Practices’ PIP Scores

We tested whether the change in PIP total score differed between active and control practices. In a multivariate mixed linear regression model, change in PIP total score (baseline to 2-year) was the dependent variable, intervention assignment was the predictor, and covariates were selected from potential confounders identified using the procedure described above. Change in PIP domain scores was also explored. No random effects were modeled, and no correction for multiple comparisons was applied. With a sample size of 42 practices and an SD of 14.2 points, the study had 80% power to detect an effect of 12.7 points.

Exploratory Analyses

To better understand our findings, we built several mixed linear models to explore associations not part of the original study design. We did not correct for multiple comparisons but tested for confounding as described above and included potential confounders in our models. To evaluate the association between practices’ baseline IBH on patient outcomes, we modeled each baseline and 2-year PROMIS-29 domain and summary score as dependent variables, baseline PIP total score as a fixed effect, and practice as a random effect. Noting that active practices varied in their fidelity using the intervention, we explored the association between intervention workbook completion and changes in IBH and patient outcomes. To evaluate the relation between intervention completion and change in IBH, we modeled change in PIP scores, which were adjusted for baseline PIP scores, as dependent variables, completion status as a fixed independent effect, and practice as a random effect. To determine if there was an association between intervention completion and change in patient outcomes, we modeled changes in patient outcomes, also adjusted for baseline outcomes, as dependent variables, completion status as a fixed independent effect, and practice as a random effect. All data were analyzed using Stata 17 (StataCorp LLC).

RESULTS

Of 504,827 patient records reviewed electronically, 133,041 met the inclusion criteria, of which 66,804 were recruited and 4,407 responded from practices that stayed in the study. A total of 2,945 eligible patients did not drop out before randomization and completed the baseline PROMIS-29 survey; 2,426 (82%) completed follow-up 2 years later (Figure 1). The final sample (Table 1) had a greater prevalence of chronic disease and older average age (62 years vs 47 years) than the general population of US primary care patients, as described by the National Ambulatory Medical Care Survey,52 but was generally similar in race, ethnicity, gender, and rurality characteristics. Of 121 practices invited to participate, 52 declined, 24 were ineligible, 2 served as nonrandomized pilot sites (vanguard sites), and 1 dropped out before data collection, leaving 42 practices in the study. There were no significant differences in practice characteristics between active and control arms (Table 2).

Figure 1.
Figure 1.

CONSORT participant flow diagram.

CONSORT = Consolidated Standards of Reporting Trials.

a Eligible but chose not to participate.

b Wanted to participate but not eligible.

c Eligible but volunteered to pilot the intervention before study start.

Intervention Effect on Patients’ PROMIS-29 Scores

There was no difference in the amount of change in any of the 8 PROMIS-29 domains reported by patients receiving care from active practices compared with patients from control practices (Table 3 and Figure 2). Although mental health summary scores (adjusted for baseline and population density) improved in both groups (from 50.17 to 50.64 [Δ = 0.47; 95% CI, 0.10-0.84; P = .012] in the active group and 50.36 to 50.77 [Δ = 0.41; 95% CI, 0.11-0.70; P = .006] in the control group), the difference in improvement between the 2 groups was not statistically significant (Δ = −0.06; 95% CI, −0.51 to 0.39; P = .79) or clinically meaningful. Exploratory models based on patient subgroups revealed no consistent patterns among subgroups or outcomes.

View this table:
Table 3.

Adjusted Effects of Assignment to Active Group on Outcomes

Figure 2.
Figure 2.
Figure 2.

Effect of active group assignment on patient and practice outcomes.

PIP = Practice Integration Profile; PROMIS-29 = Patient-Reported Outcomes Measurement Information System.

Note: The center point of each bar is the mean adjusted effect (regression coefficient) of the intervention on a specific outcome, with the bars representing 95% CI. Values >0 indicate that the intervention was associated with an increase in the outcome measured. Bars that do not cross 0 indicate statistical significance (P < .05).

a Greater score indicates improved outcome.

Intervention Effect on Practices’ PIP Scores

Practice Integration Profile total score improved significantly during the study period for both active and control practices, but there was no statistically significant difference in improvement seen between the 2 groups (Table 3). Active practices (n = 20) improved their average PIP total scores from 59.4 to 71.6 (Δ = 12.2; 95% CI, 7.0-17.4; P <.001); control practices (n = 22) improved from 59.4 to 67.7 (Δ = 8.3; 95% CI, 1.5-15.2; P = .027). The difference in improvement between active and control groups was not significant (Δ = 3.9; 95% CI, −3.6 to 11.2; P = .3). However, PIP workflow domain scores improved significantly more in active than control practices (9.3 points; 95% CI, 1.7-16.9; P = .02), and all other PIP domains showed a pattern of nominally greater improvements (Table 3, Figure 2).

Baseline IBH

Regardless of assignment to active or control group, we observed a positive association between degree of integration (PIP total score) at baseline and patient mental health function (PROMIS-29 score) at baseline and 2 years (Table 4, Figure 3). Baseline anxiety, sleep disturbance, social participation, and mental health summary were significantly associated with greater PIP total score at baseline (eg, mental health summary: 0.05 points; 95% CI, 0-0.09; P = .05), with similar outcomes at 2 years (Table 4). Other PROMIS-29 scores showed a pattern of effect with nominally better patient function associated with greater PIP total score, although not significant when tested individually.

View this table:
Table 4.

Adjusted Association Between Baseline Practice Integration (PIP Total Score) and Patient Outcomes at Baseline and Follow-Up (N = 42; All Practices)

Figure 3.
Figure 3.
Figure 3.

Association of practice integration and patient outcomes.

PIP = Practice Integration Profile.

Note: The center point of each bar is the mean adjusted association (regression coefficient) of the baseline median total PIP score with specific patient outcomes, adjusting for potential confounders and using a random intercept for each practice. The bars represent 95% CI. Values >0 indicate that the intervention was associated with an increase in the outcome measured. Bars that do not cross 0 indicate statistical significance (P < .05).

a Greater score indicates improved outcome.

Intervention Workbook Completion

Active practices that completed intervention workbooks (n = 13) had significantly greater improvement in PIP total scores than practices that did not (n = 7) (9.9-point difference in change in PIP total score; 95% CI, 2.0-18.0; P = .02), with nominally greater changes in all PIP domain scores (Figure 4).

Figure 4.
Figure 4.
Figure 4.

Association of intervention completion in active group (n = 20) and change in practice integration and patient-reported outcomes.

PIP = Practice Integration Profile; PROMIS-29 = Patient-Reported Outcomes Measurement Information System.

Note: The center point of each bar is the mean adjusted effect (regression coefficient) of the completion of the intervention on a specific outcome, with the bars representing 95% CI. Values >0 indicate that completion of the intervention was associated with an increase in the outcome measured. Bars that do not cross 0 indicate statistical significance (P < .05).

a Greater score indicates improved outcome.

Patients who received care in practices that completed intervention workbooks reported significantly greater improvement in mental health summary (0.8; 95% CI, 0-1.6; P = .05) and social participation (1.4; 95% CI, 0.2-2.7; P = .02) (Figure 4) and nominally greater positive change on the remaining PROMIS-29 subscales than patients of active practices that did not complete workbooks, despite having access.

DISCUSSION

Practice access to the study intervention had no significant effect on the amount of change observed in patient-reported outcomes or PIP total score. Only the difference in PIP workflow improvement was large enough to attribute to active group assignment, which matches the structured focus of the toolkit on workflow redesign. However, study data allowed us to conduct additional analyses that could be important for future work attempting to understand how to evaluate and implement IBH.

Patients receiving care in practices that started the study with greater BH integration scores (before intervention) reported significantly better mental health function—both at baseline and at 2 years. Patients of practices that successfully completed intervention workbooks reported significantly greater improvements in mental health after 2 years. Although not proof of cause, these observations are consistent with the expectation that patients with MCCs benefit from efforts to enhance IBH. That total PIP score was associated with patient-reported outcomes and with implementation of IBH improvements supports the continued use of PIP as a measure of integration for future IBH enhancement activities.

Although the intervention was designed to support complex practice change and adapt to local context, only 13 of 20 active practices completed the intervention during the study period. Those 13 had significantly greater improvements in IBH and patient outcomes. Because confounding factors could affect both the ability to implement and the outcomes, these observations are not evidence of causation. As noted by Walunas and others, we need more studies to understand how much intervention is sufficient for practice change53-56 as well as how effective toolkits are in improving integration.

Limitations

The pragmatic design of the present study revealed real-world challenges to IBH. Participating practices in both groups experienced natural disasters, major work stoppages, loss of key personnel, and a pandemic. The research design allowed practices randomized to the control group to do QI work of their own (without access to the specific resources of the intervention) and allowed those in the active group to select the changes they perceived as most helpful. Too much flexibility of the intervention might have led to changes that were insufficiently strong to cause a clinic-wide effect on patient outcomes and integration. The resilience of practices in both groups, recruited based on their preexisting commitment to improve IBH, showed general improvement in integration but might have biased the outcomes and contributed to null results for the intervention. In particular, interpreting how the COVID-19 pandemic affected IBH is complex and requires further study.

A limitation of the present study is that patients who agreed to participate might be systematically different from those who declined. In addition, researchers did not have access to individual patient electronic health records after enrollment to know which patients were provided BH services and whether those who received additional services fared differently than others. Future studies distinguishing patients by service need and service use will provide additional insights. Continued research that includes the effect of BH services with meaningful clinical and service data, and relevant patient-reported outcomes for patients with MCCs, is needed to guide selection of the best IBH strategies and models (eg, Collaborative Care Model).7,57

CONCLUSIONS

Although the specific practice redesign intervention studied did significantly improve the workflow of integrated care, it had little effect on other practice or patient outcomes. Greater practice IBH levels at baseline were consistently associated with better patient mental and physical function. Practices that completed the intervention improved integration and patient outcomes more than those that did not. Future practice-based research will examine relations associated with practice engagement in the intervention steps and other factors to understand how to improve outcomes via integrated medical and behavioral health services for those who need it the most—patients with MCCs.

Acknowledgments

We would like to acknowledge the dedicated support and commitment of our 42 primary care active and control sites and 2 vanguard sites, their practice members, and their patients, all of whom generously gave their time and thought to fulfill the requirements of this study.

Footnotes

  • Conflicts of interest: authors report none.

  • Read or post commentaries in response to this article.

  • Funding support: This work was funded by a Patient-Centered Outcomes Research Institute (PCORI) Award (PCS-1409-24372).

  • Disclaimer: The statements in this report are solely the responsibility of the authors and do not necessarily represent the views of PCORI, its Board of Governors, or Methodology Committee. This manuscript has not been previously published and is not under consideration in the same or substantially similar form in any other peer-reviewed media.

  • Previous presentation: Early work was presented in a poster presentation at the North American Primary Care Research Group Annual Meeting; November 16-20, 2019; Toronto, Canada.

  • Supplemental materials

  • Received for publication December 21, 2022.
  • Revision received May 5, 2023.
  • Accepted for publication May 31, 2023.

References

  1. 1.↵
    1. Baird M,
    2. Blount A,
    3. Brungardt S, et al.
    The development of joint principles: integrating behavioral health care into the patient-centered medical home. Ann Fam Med. 2014; 12(2): 183-185. doi:10.1370/afm.1634
    OpenUrlFREE Full Text
  2. 2.↵
    1. Johnson NB,
    2. Hayes LD,
    3. Brown K,
    4. Hoo EC,
    5. Ethier KA; Centers for Disease Control and Prevention (CDC)
    . CDC National Health Report: leading causes of morbidity and mortality and associated behavioral risk and protective factors—United States, 2005-2013. MMWR Suppl. 2014; 63(4): 3-27.
    OpenUrl
  3. 3.↵
    1. Mokdad AH,
    2. Marks JS,
    3. Stroup DF,
    4. Gerberding JL.
    Actual causes of death in the United States, 2000. JAMA. 2004; 291(10): 1238-1245. doi:10.1001/jama.291.10.1238
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. Druss BG,
    2. Walker ER.
    Mental disorders and medical comorbidity. Synth Proj Res Synth Rep. 2011; (21): 1-26.
  5. 5.
    1. Kathol RG,
    2. McAlpine D,
    3. Kishi Y, et al.
    General medical and pharmacy claims expenditures in users of behavioral health services. J Gen Intern Med. 2005; 20(2): 160-167. doi:10.1111/j.1525-1497.2005.40099.x
    OpenUrlCrossRefPubMed
  6. 6.↵
    1. Melek S,
    2. Norris D.
    Chronic Conditions and Comorbid Psychological Disorders. Milliman; 2008. Accessed Aug 16, 2023. https://www.cbhc.org/wp-content/uploads/2015/11/chronic-conditions-and-comorbid-RR07-01-08.pdf
  7. 7.↵
    1. Kroenke K,
    2. Unutzer J.
    Closing the false divide: sustainable approaches to integrating mental health services into primary care. J Gen Intern Med. 2017; 32(4): 404-410. doi:10.1007/s11606-016-3967-9
    OpenUrlCrossRef
  8. 8.↵
    1. Balasubramanian BA,
    2. Cohen DJ,
    3. Jetelina KK, et al.
    Outcomes of integrated behavioral health with primary care. J Am Board Fam Med. 2017; 30(2): 130-139. doi:10.3122/jabfm.2017.02.160234
    OpenUrlAbstract/FREE Full Text
  9. 9.↵
    1. Reed SJ,
    2. Shore KK,
    3. Tice JA.
    Effectiveness and value of integrating behavioral health into primary care. JAMA Intern Med. 2016; 176(5): 691-692. doi:10.1001/jamainternmed.2016.0804
    OpenUrlCrossRef
  10. 10.↵
    1. Bierman AS.
    Preventing and managing multimorbidity by integrating behavioral health and primary care. Health Psychol. 2019; 38(9): 851-854. doi:10.1037/hea0000787
    OpenUrlCrossRef
  11. 11.
    1. Gilmer TP,
    2. Avery M,
    3. Siantz E, et al.
    Evaluation of the Behavioral Health Integration and Complex Care Initiative in Medi-Cal. Health Aff (Millwood). 2018; 37(9): 1442-1449. doi:10.1377/hlthaff.2018.0372
    OpenUrlCrossRef
  12. 12.
    1. Loeb DF,
    2. Bayliss EA,
    3. Candrian C,
    4. deGruy FV,
    5. Binswanger IA.
    Primary care providers’ experiences caring for complex patients in primary care: a qualitative study. BMC Fam Pract. 2016; 17: 34. doi:10.1186/s12875-016-0433-z
    OpenUrlCrossRef
  13. 13.
    1. Bogner HR,
    2. Morales KH,
    3. de Vries HF,
    4. Cappola AR.
    Integrated management of type 2 diabetes mellitus and depression treatment to improve medication adherence: a randomized controlled trial. Ann Fam Med. 2012; 10(1): 15-22. doi:10.1370/afm.1344
    OpenUrlAbstract/FREE Full Text
  14. 14.
    1. Katon WJ,
    2. Lin EHB,
    3. Von Korff M, et al.
    Collaborative care for patients with depression and chronic illnesses. N Engl J Med. 2010; 363(27): 2611-2620. doi:10.1056/NEJMoa1003955
    OpenUrlCrossRefPubMed
  15. 15.
    1. Lin EH,
    2. Von Korff M,
    3. Ciechanowski P, et al.
    Treatment adjustment and medication adherence for complex patients with diabetes, heart disease, and depression: a randomized controlled trial. Ann Fam Med. 2012; 10(1): 6-14. doi:10.1370/afm.1343
    OpenUrlAbstract/FREE Full Text
  16. 16.↵
    1. Stewart JC,
    2. Perkins AJ,
    3. Callahan CM.
    Effect of collaborative care for depression on risk of cardiovascular events: data from the IMPACT randomized controlled trial. Psychosom Med. 2014; 76(1): 29-37. doi:10.1097/psy.0000000000000022
    OpenUrlAbstract/FREE Full Text
  17. 17.↵
    1. Peek CJ and the National Integration Academy Council
    . Lexicon for Behavioral Health and Primary Care Integration: Concepts and Definitions Developed by Expert Concensus. AHRQ Publications No.13-IP001-EF. Agency for Healthcare Research and Quality; 2013. https://integrationacademy.ahrq.gov/sites/default/files/2020-06/Lexicon.pdf?_gl=1*1nqtq3h*_ga*NTU4NjQwODM5LjE2OTQ4OTQ2OTU.*_ga_45NDTD15CJ*MTY5NDkwNDMxMC4xLjAuMTY5NDkwNDMxMC42MC4wLjA
  18. 18.↵
    1. Macchi CR,
    2. Kessler R,
    3. Auxier A, et al.
    The Practice Integration Profile: rationale, development, method, and research. Fam Syst Health. 2016; 34(4): 334-341. doi:10.1037/fsh0000235
    OpenUrlCrossRef
  19. 19.
    1. Archer J,
    2. Bower P,
    3. Gilbody S, et al.
    Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev. 2012; 10: CD006525. doi:10.1002/14651858.CD006525.pub2
    OpenUrlCrossRefPubMed
  20. 20.
    1. Hunter CL,
    2. Funderburk JS,
    3. Polaha J,
    4. Bauman D,
    5. Goodie JL,
    6. Hunter CM.
    Primary Care Behavioral Health (PCBH) model research: current state of the science and a call to action. J Clin Psychol Med Settings. 2018; 25(2): 127-156. doi:10.1007/s10880-017-9512-0
    OpenUrlCrossRef
  21. 21.↵
    1. Moriarty AS,
    2. Coventry PA,
    3. Hudson JL, et al.
    The role of relapse prevention for depression in collaborative care: a systematic review. J Affect Disord. 2020; 265: 618-644. doi:10.1016/j.jad.2019.11.105
    OpenUrlCrossRefPubMed
  22. 22.↵
    1. Cohen D,
    2. McDaniel RR Jr.,
    3. Crabtree BF, et al.
    A practice change model for quality improvement in primary care practice. J Healthc Manag. 2004; 49(3): 155-168; discussion 169-170.
    OpenUrlPubMed
  23. 23.↵
    1. Davis MM,
    2. Howk S,
    3. Spurlock M,
    4. McGinnis PB,
    5. Cohen DJ,
    6. Fagnan LJ.
    A qualitative study of clinic and community member perspectives on intervention toolkits: “unless the toolkit is used it won’t help solve the problem”. BMC Health Serv Res. 2017; 17(1): 497. doi:10.1186/s12913-017-2413-y
    OpenUrlCrossRefPubMed
  24. 24.↵
    1. Hempel S,
    2. Miake-Lye I,
    3. Brega AG, et al.
    Quality improvement toolkits: recommendations for development. Am J Med Qual. 2019; 34(6): 538-544. doi:10.1177/1062860618822102
    OpenUrlCrossRef
  25. 25.↵
    1. van Eeghen C,
    2. Littenberg B,
    3. Holman MD,
    4. Kessler R.
    Integrating behavioral health in primary care using lean workflow analysis: a case study. J Am Board Fam Med. 2016; 29(3): 385-393. doi:10.3122/jabfm.2016.03.150186
    OpenUrlAbstract/FREE Full Text
  26. 26.↵
    1. Perry CK,
    2. Damschroder LJ,
    3. Hemler JR,
    4. Woodson TT,
    5. Ono SS,
    6. Cohen DJ.
    Specifying and comparing implementation strategies across seven large implementation interventions: a practical application of theory. Implement Sci. 2019; 14(1): 32. doi:10.1186/s13012-019-0876-4
    OpenUrlCrossRefPubMed
  27. 27.↵
    1. Powell BJ,
    2. Waltz TJ,
    3. Chinman MJ, et al.
    A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015; 10: 21. doi:10.1186/s13012-015-0209-1
    OpenUrlCrossRefPubMed
  28. 28.↵
    1. Kessler R,
    2. Glasgow RE.
    A proposal to speed translation of healthcare research into practice: dramatic change is needed. Am J Prev Med. 2011; 40(6): 637-644. doi:10.1016/j.amepre.2011.02.023
    OpenUrlCrossRefPubMed
  29. 29.↵
    1. Thorpe KE,
    2. Zwarenstein M,
    3. Oxman AD, et al.
    A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009; 62(5): 464-475. doi:10.1016/j.jclinepi.2008.12.011
    OpenUrlCrossRefPubMed
  30. 30.↵
    1. Crocker AM,
    2. Kessler R,
    3. van Eeghen C, et al.
    Integrating Behavioral Health and Primary Care (IBH-PC) to improve patient-centered outcomes in adults with multiple chronic medical and behavioral health conditions: study protocol for a pragmatic cluster-randomized control trial. Trials. 2021; 22(1): 200. doi:10.1186/s13063-021-05133-8
    OpenUrlCrossRefPubMed
  31. 31.↵
    1. Jewiss J,
    2. Natkin LW,
    3. Clark/Keefe K,
    4. Crocker A,
    5. Welkowitz JA.
    Integrating behavioural health and primary care: qualitative findings on contextual factors that influence integration. Fam Pract. 2023 Mar 21:cmad029. doi:10.1093/fampra/cmad029
    OpenUrlCrossRef
  32. 32.↵
    1. Hitt JR,
    2. Brennhofer SA,
    3. Martin MP, et al.
    Further experience with the Practice Integration Profile: a measure of behavioral health and primary care integration. J Clin Psychol Med Settings. 2022; 29(2): 274-284. doi:10.1007/s10880-021-09806-z
    OpenUrlCrossRef
  33. 33.↵
    1. Kessler RS,
    2. Auxier A,
    3. Hitt JR, et al.
    Development and validation of a measure of primary care behavioral health integration. Fam Syst Health. 2016; 34(4): 342-356. doi:10.1037/fsh0000227
    OpenUrlCrossRef
  34. 34.↵
    1. Mullin DJ,
    2. Hargreaves L,
    3. Auxier A, et al.
    Measuring the integration of primary care and behavioral health services. Health Serv Res. 2019; 54(2): 379-389. doi:10.1111/1475-6773.13117
    OpenUrlCrossRef
  35. 35.↵
    1. Cella D,
    2. Riley W,
    3. Stone A, et al; PROMIS Cooperative Group
    . The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010; 63(11): 1179-1194. doi:10.1016/j.jclinepi.2010.04.011
    OpenUrlCrossRefPubMed
  36. 36.↵
    1. Hays RD,
    2. Spritzer KL,
    3. Schalet BD,
    4. Cella D.
    PROMIS®-29 v2.0 profile physical and mental health summary scores. Qual Life Res. 2018; 27(7): 1885-1891. doi:10.1007/s11136-018-1842-3
    OpenUrlCrossRefPubMed
  37. 37.↵
    1. Mercer SW,
    2. Maxwell M,
    3. Heaney D,
    4. Watt GC.
    The consultation and relational empathy (CARE) measure: development and preliminary validation and reliability of an empathy-based consultation process measure. Fam Pract. 2004; 21(6): 699-705. doi:10.1093/fampra/cmh621
    OpenUrlCrossRefPubMed
  38. 38.↵
    1. Morisky DE,
    2. Green LW,
    3. Levine DM.
    Concurrent and predictive validity of a self-reported measure of medication adherence. Med Care. 1986; 24(1): 67-74. doi:10.1097/00005650-198601000-00007
    OpenUrlCrossRefPubMed
  39. 39.↵
    1. MacLean CD,
    2. Littenberg B,
    3. Gagnon M,
    4. Reardon M,
    5. Turner PD,
    6. Jordan C.
    The Vermont Diabetes Information System (VDIS): study design and subject recruitment for a cluster randomized trial of a decision support system in a regional sample of primary care practices. Clin Trials. 2004; 1(6): 532-544. doi:10.1191/1740774504cn051oa
    OpenUrlCrossRefPubMed
  40. 40.↵
    1. Adams PF,
    2. Hendershot GE,
    3. Marano MA; Centers for Disease Control and Prevention/National Center for Health Statistics
    . Current estimates from the National Health Interview Survey, 1996. Vital Health Stat 10. 1999; (200): 1-203.
    OpenUrlPubMed
  41. 41.↵
    1. Hlatky MA,
    2. Boineau RE,
    3. Higginbotham MB, et al.
    A brief self-administered questionnaire to determine functional capacity (the Duke Activity Status Index). Am J Cardiol. 1989; 64(10): 651-654. doi:10.1016/0002-9149(89)90496-7
    OpenUrlCrossRefPubMed
  42. 42.↵
    1. Rose GL,
    2. Bonnell LN,
    3. O’Rourke-Lavoie JB, et al.
    Development and validation of the patient centeredness index for primary care. J Clin Nurs. 2022; 31(23-24): 3485-3497. doi:10.1111/jocn.16177
    OpenUrlCrossRef
  43. 43.↵
    1. Spitzer RL,
    2. Kroenke K,
    3. Williams JB.
    Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA. 1999; 282(18): 1737-1744. doi:10.1001/jama.282.18.1737
    OpenUrlCrossRefPubMed
  44. 44.↵
    1. Spitzer RL,
    2. Kroenke K,
    3. Williams JBW,
    4. Löwe B.
    A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006; 166(10): 1092-1097. doi:10.1001/archinte.166.10.1092
    OpenUrlCrossRefPubMed
  45. 45.↵
    1. Bime C,
    2. Wei CY,
    3. Holbrook JT,
    4. Sockrider MM,
    5. Revicki DA,
    6. Wise RA.
    Asthma symptom utility index: reliability, validity, responsiveness, and the minimal important difference in adult asthmatic patients. J Allergy Clin Immunol. 2012; 130(5): 1078-1084. doi:10.1016/j.jaci.2012.07.058
    OpenUrlCrossRefPubMed
  46. 46.↵
    1. Revicki DA,
    2. Leidy NK,
    3. Brennan-Diemer F,
    4. Sorensen S,
    5. Togias A.
    Integrating patient preferences into health outcomes assessment: the multiattribute Asthma Symptom Utility Index. Chest. 1998; 114(4): 998-1007. doi:10.1378/chest.114.4.998
    OpenUrlCrossRefPubMed
  47. 47.↵
    1. Dennis ML,
    2. Chan YF,
    3. Funk RR.
    Development and validation of the GAIN Short Screener (GSS) for internalizing, externalizing and substance use disorders and crime/violence problems among adolescents and adults. Am J Addict. 2006; 15(Suppl 1): 80-91. doi:10.1080/10550490601006055
    OpenUrlCrossRef
  48. 48.↵
    1. Morean ME,
    2. DeMartini KS,
    3. Foster D, et al.
    The Self-Report Habit Index: assessing habitual marijuana, alcohol, e-cigarette, and cigarette use. Drug Alcohol Depend. 2018; 186: 207-214. doi:10.1016/j.drugalcdep.2018.01.014
    OpenUrlCrossRefPubMed
  49. 49.↵
    1. Hays RD,
    2. Spritzer KL,
    3. Fries JF,
    4. Krishnan E.
    Responsiveness and minimally important difference for the patient-reported outcomes measurement information system (PROMIS) 20-item physical functioning short form in a prospective observational study of rheumatoid arthritis. Ann Rheum Dis. 2015; 74(1): 104-107. doi:10.1136/annrheumdis-2013-204053
    OpenUrlAbstract/FREE Full Text
  50. 50.↵
    1. Yost KJ,
    2. Eton DT,
    3. Garcia SF,
    4. Cella D.
    Minimally important differences were estimated for six Patient-Reported Outcomes Measurement Information System-Cancer scales in advanced-stage cancer patients. J Clin Epidemiol. 2011; 64(5): 507-516. doi:10.1016/j.jclinepi.2010.11.018
    OpenUrlCrossRefPubMed
  51. 51.↵
    1. Salkind N
    1. Abdi H.
    The Bonferonni and Šidák corrections for multiple comparisons. In: Salkind N, ed. Encyclopedia of Measurement and Statistics. Sage; 2007.
  52. 52.↵
    1. Centers for Disease Control and Prevention
    . Ambulatory Health Care Data. Last reviewed Dec 30, 2021. Accessed Aug 16, 2023. https://www.cdc.gov/nchs/ahcd/about_ahcd.htm
  53. 53.↵
    1. Cohen DJ,
    2. Sweeney SM,
    3. Miller WL, et al.
    Improving smoking and blood pressure outcomes: the interplay between operational changes and local context. Ann Fam Med. 2021; 19(3): 240-248. doi:10.1370/afm.2668
    OpenUrlAbstract/FREE Full Text
  54. 54.
    1. Keddem S,
    2. Agha AZ,
    3. Long JA,
    4. Shasha B,
    5. Hausmann LRM,
    6. Shea JA.
    The gears of knowledge translation: process evaluation of the dissemination and implementation of a patient engagement toolkit. J Gen Intern Med. 2020; 35(Suppl 2): 808-814. doi:10.1007/s11606-020-06099-0
    OpenUrlCrossRefPubMed
  55. 55.
    1. Perry CK,
    2. Lindner S,
    3. Hall J,
    4. Solberg LI,
    5. Baron A,
    6. Cohen DJ.
    How type of practice ownership affects participation with quality improvement external facilitation: findings from EvidenceNOW. J Gen Intern Med. 2022; 37(4): 793-801. doi:10.1007/s11606-021-07204-7
    OpenUrlCrossRef
  56. 56.↵
    1. Walunas TL,
    2. Ye J,
    3. Bannon J, et al.
    Does coaching matter? Examining the impact of specific practice facilitation strategies on implementation of quality improvement interventions in the Healthy Hearts in the Heartland study. Implement Sci. 2021; 16(1): 33. doi:10.1186/s13012-021-01100-8
    OpenUrlCrossRef
  57. 57.↵
    1. Kathol RG,
    2. Hobbs Knutson K,
    3. Dehnel PJ.
    Physician’s Guide: Understanding and Working With Integrated Case Managers. 1st ed. Springer; 2016.
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Get Permissions

Jump to section

Subjects

Keywords