Addressing Common Challenges in the Implementation of Collaborative Care for Mental Health: The Penn Integrated Care Program
============================================================================================================================

* Courtney Benjamin Wolk
* Briana S. Last
* Cecilia Livesey
* Maria A. Oquendo
* Matthew J. Press
* David S. Mandell
* Erin Ingram
* Anne C. Futterer
* Grace P. Kinkler
* David W. Oslin

## Abstract

**PURPOSE** We developed and implemented a new model of collaborative care that includes a triage and referral management system. We present initial implementation metrics using the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework.

**METHODS** Primary care clinicians in 8 practices referred patients with any unmet mental health needs to the Penn Integrated Care program. Assessments were conducted using validated measures. Patients were primarily triaged to collaborative care (26%) or specialty mental health care with active referral management (70%). We conducted 50 qualitative interviews to understand the implementation process and inform program refinement. Our primary outcomes were reach and implementation metrics, including referral and encounter rates derived from the electronic health record.

**RESULTS** In 12 months, 6,124 unique patients were referred. Assessed patients reported symptoms consistent with a range of conditions from mild to moderate depression and anxiety to serious mental illnesses including psychosis and acute suicidal ideation. Among patients enrolled in collaborative care, treatment entailed a mean of 7.2 (SD 5.1) encounters over 78.1 (SD 51.3) days. Remission of symptoms was achieved by 32.6% of patients with depression and 39.5% of patients with anxiety. Stakeholders viewed the program favorably and had concrete suggestions to ensure sustainability.

**CONCLUSIONS** The Penn Integrated Care program demonstrated broad reach. Implementation was consistent with collaborative care as delivered in seminal studies of the model. Our results provide insight into a model for launching and implementing collaborative care to meet the needs of a diverse group of patients with the full range of mental health conditions seen in primary care.

Key words
*   delivery of health care, integrated
*   integrated behavioral health
*   mental health
*   collaborative care
*   continuity of care
*   coordination of care
*   health services
*   quality improvement
*   organizational change
*   primary care
*   practice-based research

## INTRODUCTION

The Collaborative Care Model (CoCM) has a robust research base and is the leading model for integrating mental health treatments in primary care, most notably for depression and anxiety.1-3 Key elements include a team composed of a primary care clinician, mental health professional (MHP), and supporting psychiatrist; use of evidence-based practices and measurement-based care; and a registry to track patients. Numerous meta-analyses and randomized controlled trials of CoCM have demonstrated that mild to moderate depression and anxiety can be effectively managed in primary care.2,3 Depression treatment in primary care improves access to care, is cost-effective, and achieves better quality of care.2,3

The CoCM focuses on patients with mild to moderate depression, anxiety,1 or alcohol misuse, or some combination thereof.4 Individuals with severe psychopathology typically require more care than this model provides5; however, navigating specialty mental health services is difficult for primary care clinicians and patients.6 Time spent finding services for complex patients with limited resources diminishes the MHP’s time available to implement psychological interventions (eg, behavioral activation7). This problem is important because delivery of psychological interventions is associated with improved outcomes in CoCM.8

As part of a staged implementation of CoCM, the University of Pennsylvania Health System (Penn Medicine), a large and diverse health system, developed an intake, triage, and referral management resource center and encouraged primary care clinicians to refer patients with any mental health symptom or condition, as opposed to only those with mild to moderate common issues. This resource center assesses patients by telephone using validated measures such as the Patient Health Questionnaire-9 (PHQ-9) scale,9 risk-stratifies them based on their assessment data, refers them to the appropriate level of care using decision-support software,10 and facilitates engagement in community-based specialty care. Our program’s CoCM team consists of the patient, primary care clinician, MHP, and consulting psychiatrist, and the mental health intake coordinators in the resource center (Figure 1).

![Figure 1.](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/19/2/148/F1.medium.gif)

[Figure 1.](http://www.annfammed.org/content/19/2/148/F1)

Figure 1. 
The Penn Integrated Care model.

MHP = mental health professional; PCC = primary care clinician.

In this article, we describe the elements of our program in detail. Then we present data from the first 12 months of its implementation using the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework for health behavior interventions.11,12

## METHODS

### Program Description

In January 2018, Penn’s Department of Psychiatry and the Primary Care Service Line launched a quality improvement program to increase access to and engagement with mental health care to improve mental health and physical health outcomes. The Penn Integrated Care (PIC) program has 2 main components: (1) a centralized intake, triage, and referral management center to facilitate referral to mental health services for patients in primary care, the PIC Resource Center, and (2) CoCM services in the primary care practice. Services are reimbursed through billing collaborative care Current Procedural Terminology codes.13

We elected to develop a resource center for 2 main reasons. First, there was concern that the MHPs in the practices would become overwhelmed with assessment and referral activities given their limited time (0.5 to 1.0 full-time equivalents per practice). Second, although CoCM programs manage mild to moderate mental health conditions, primary care clinicians face the full spectrum of conditions and are often challenged to access appropriate care for more severe conditions.6

Primary care clinicians initiate patient referrals to PIC in the electronic health record (EHR). A clinician can initiate a referral to the program for any reason, including elevated scores on routine depression screening, identification of a mental health concern during the visit, or patient request. The PIC Resource Center, staffed by trained bachelor’s level mental health intake coordinators supervised by a licensed MHP, calls the patient within 24 to 48 hours (or the patient may call the center directly) and conducts a structured assessment by telephone using Behavioral Health Laboratory proprietary software.10 The software uses validated measures that address depression, anxiety, suicidal ideation, substance use, trauma, mania, psychosis, and cognitive impairment. Based on screening results, an algorithm determines the appropriate level of care. The algorithm has been iteratively refined over several years and is used in the Department of Veterans Affairs.10,14 It can be tailored to the caseload capacity of the MHP. Patients with mild to moderate depression, anxiety, and alcohol use are typically referred directly to the MHPs embedded in the primary care practices for brief evidence-based treatment. Patients identified with a substance use disorder are referred to a community addictions program; those with possible psychosis or mania are referred to a community psychiatrist; and those with posttraumatic stress disorder or severe depression or anxiety are referred to a community specialist. Referrals also account for patient preference, insurance coverage, and information from the primary care clinician. A clinical staff member immediately assesses patients at high risk for suicide, and care is provided as needed. The PIC Resource Center maintains a detailed database that permits identification of community professionals who are in-network for the patient’s insurance. Once a community professional is identified, the center assists with scheduling an initial appointment and follows up to ensure patients have attended their appointments and successfully engaged in care. If not, the center facilitates additional referrals. Throughout this process, the center documents decisions and outcomes in the EHR for the primary care clinician.

A trained MHP, in this case a licensed social worker, is embedded within each primary care practice to deliver evidence-based CoCM services7,15-18 for patients with mild to moderate mental illness in collaboration with primary care clinicians and the consulting psychiatrist.1 We hired 5 full-time equivalent MHPs and engaged 0.4 full-time equivalents of a consulting psychiatrist to support implementation. The MHPs are available for warm handoffs when a patient is in crisis, but most referrals from primary care are handled by the PIC Resource Center through an electronic referral. The primary care clinician is the billing professional of record for all CoCM services, prescribes psychiatric pharmacotherapy as appropriate, continues to manage the patient’s other health needs, and collaborates in the patient’s mental health treatment.

### Implementation Framework

The RE-AIM framework for health behavior interventions11,12 guides the presentation of our results. This framework specifies 5 essential components for translating research into practice: (1) reach of the target population, (2) effectiveness or efficacy of the intervention, (3) adoption of the intervention by the organization and intervention agents, (4) implementation (eg, use of intervention, fidelity), and (5) maintenance, or the extent to which an intervention is sustained over time.

### Data Sources

#### Quantitative Data

We obtained data for unique patients referred to the PIC program between January 1, 2018 and December 31, 2018, the first year of the program’s implementation. For patients with multiple referrals to PIC or episodes of care with a PIC MHP during the year, we used the first referral in the analysis of reach and the first episode of care for the effectiveness and implementation analyses. Reach data, including intake assessment results, were derived from the Behavioral Health Laboratory software program. The EHR provided effectiveness data (eg, pretreatment and posttreatment scores) and implementation data.

The PHQ-9 questionnaire is a 9-item rating scale of depressive symptoms over the past 2 weeks.9 Each item is scored on a 0 to 3 Likert scale where 0 = not at all, 1 = several days, 2 = more than half the days, and 3 = nearly every day. Item 9 queries specifically about “thoughts that you would be better off dead, or of hurting yourself.” Good reliability and validity have been established.19 PHQ-9 data were obtained from the Behavioral Health Laboratory (ie, intake) and EHR (ie, pretreatment and posttreatment).

The Generalized Anxiety Disorder 7-item scale (GAD-7) assesses symptoms of generalized anxiety and uses the same format as the PHQ-9.20 It has been shown to be reliable and valid in primary care.20,21 We obtained GAD-7 data from the Behavioral Health Laboratory (ie, intake) and EHR (ie, pretreatment and posttreatment).

The PTSD Checklist for the *Diagnostic and Statistical Manual of Mental Disorders, 5th ed (DSM-5*) (PCL-5) is a 20-item self-report measure of posttraumatic stress disorder symptoms over the past month.22 Items are rated on a 0 to 4 Likert scale where 0 = not at all, 1 = a little bit, 2 = moderately, 3 = quite a bit, and 4 = extremely. Strong psychometric properties have been established.23-25 The PCL-5 was completed at intake, and data were obtained from the Behavioral Health Laboratory.

The Timeline Follow-back method26 was used to obtain estimates of daily drinking over the past 7 days. This method has established high test-retest reliability and evidence of criterion and construct validity.27-30

The Timeline Follow-back was completed at intake, and data were obtained from the Behavioral Health Laboratory.

The Drug Use, Mania, and Psychosis Screeners10 were developed by the Department of Veteran Affairs Integrated Care program and provide a basic screen for the presence of drug use, manic symptoms, and psychotic symptoms based on the *DSM-5*.31 The drug use items were adapted from the Alcohol, Smoking, and Substance Involvement Screening Test.32 These screens were completed at intake, and data were obtained from the Behavioral Health Laboratory.

#### Qualitative Data

We conducted qualitative interviews with 50 key stakeholders, including 15 health system leaders in psychiatry and primary care, 14 primary care clinicians, 14 mental health personnel (ie, MHPs, psychiatrists, and mental health intake coordinators), and 7 patients. Supplemental Table 1, available at [https://www.AnnFamMed.org/content/19/2/148/suppl/DC1/](https://www.AnnFamMed.org/content/19/2/148/suppl/DC1/), provides stakeholder demographics.

Adoption, implementation activities, and maintenance were documented during implementation by the research team through these stakeholder interviews, field observations, and reports from program leadership during biweekly team meetings; this information was also used to iteratively inform program refinement. We developed a semistructured interview guide for each stakeholder group that covered barriers to and facilitators of implementation, use of PIC services, and implementation strategies used and recommended.

Interviews were conducted in person or by telephone, depending on participant preference, and lasted an average of 30 minutes. For telephone interviews, oral informed consent was obtained. For in-person interviews, written informed consent was obtained. All procedures were approved by the University of Pennsylvania Institutional Review Board.

Interviews were digitally recorded, professionally transcribed, and deidentified with analyses supported by use of an NVivo (QSR International) database. A codebook was developed and applied to all data. Coding emphasized identification of barriers to and facilitators of implementation and suggestions for improvement. The coding team met weekly to discuss discrepancies and maintain consensus.

## RESULTS

### Reach

Reach data included rates of referral to PIC and results of initial intake screenings conducted by the resource center after referral. PIC began accepting referrals in January 2018. In the first year, 6,124 unique patients either had an initial referral to the PIC Resource Center (5,931 patients; 96.8%) or were connected with an MHP via a warm handoff because of an acute concern (193 patients; 3.2%) (Figure 2).33 The referring practices served 90,038 empaneled patients, indicating that primary care clinicians referred 6.8% of their patients to the program. This volume of referrals would have been difficult for the 5 MHPs to assess and provide with treatment or community resources.

![Figure 2.](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/19/2/148/F2.medium.gif)

[Figure 2.](http://www.annfammed.org/content/19/2/148/F2)

Figure 2. 
CONSORT diagram showing patient referral and disposition.

CONSORT = Consolidated Standards of Reporting Trials; PIC = Penn Integrated Care.

The PIC Resource Center allowed primary care clinicians to refer all patients with mental health concerns without detracting from the MHPs’ ability to deliver ongoing brief psychotherapy. According to stakeholders (Supplemental Table 1), the center was favorably received because it took the onus of navigating community mental health treatment off clinicians and patients, and created a streamlined referral process that enabled more patients to access needed care. Table 1 shows the sociodemographics and clinical characteristics of referred patients and screened patients, and Supplemental Table 2, available at [https://www.AnnFamMed.org/content/19/2/148/suppl/DC1/](https://www.AnnFamMed.org/content/19/2/148/suppl/DC1/), gives illustrative quotations from the stakeholder interviews.

View this table:
[Table 1](http://www.annfammed.org/content/19/2/148/T1)

Table 1 
Sociodemographic and Clinical Characteristics of Patients Referred to and Screened in the PIC Program

Of the 3,645 patients screened by the PIC Resource Center, 961 (26.4%) were scheduled with the PIC MHP in the primary care practice, 2,553 (70.0%) were referred to community-based mental health care, 17 (0.5%) were provided with self-directed resources because they reported very mild symptoms, and 114 (3.1%) declined referral to treatment (Figure 2). Additionally, 265 patients eligible for care with the PIC MHP were referred to community-based care because of ineligible insurance or because the MHP had a full caseload.

Patients connected with the PIC MHP mainly had mild to moderate depression or anxiety symptoms, and/or alcohol misuse. Patients referred to specialty care were referred primarily because of serious mental illness (eg, mania, psychosis, or active suicidal ideation or behavior) or because they were more appropriate for a specialty intervention such as substance use treatment. Compared with patients referred to PIC, patients referred to community-based mental health care had significantly higher mean total scores on the PHQ-9, GAD-7, and PCL-5 (Supplemental Table 3, available at [https://www.AnnFamMed.org/content/19/2/148/suppl/DC1/](https://www.AnnFamMed.org/content/19/2/148/suppl/DC1/)).

### Effectiveness

Our effectiveness analyses show the response to treatment for patients with anxiety and depression who had 2 or more PIC encounters. There were 181 patients with a PHQ-9 score of 5 or higher and a depressive disorder diagnosis before treatment. Depressed patients had a mean (SD) first (ie, pretreatment) total score of 13.1 (4.9) and last (ie, posttreatment) total score of 7.7 (5.4). Fifty-nine (32.6%) achieved depression remission, defined as a total score of less than 5. Among patients who had 2 or more PIC encounters, 263 had a GAD-7 score of 5 or higher before treatment. Patients with anxiety had a mean (SD) first total score of 11.6 (4.1) and last total score of 7.1 (5.3). Overall, 104 (39.5%) achieved anxiety remission, defined as a total score of less than 5.

Across the stakeholder groups interviewed, the PIC program was perceived as effective. The brief treatment model was viewed as logical and beneficial to patients. Primary care clinicians reported that introducing PIC led patients to perceive the clinicians are providing a higher quality of care. Patients reported that the program was “helpful” and a “great experience.” Patients also reported that “being able to talk to someone,” in addition to or instead of being prescribed psychotropic medication, was a key benefit.

### Implementation

Throughout implementation, program leadership closely monitored PIC program dashboards. When leadership observed deviations from the model, they addressed them in real time via consultation with the relevant PIC team members. Table 2 summarizes the quantitative program engagement and fidelity measures. Stakeholders provided concrete implementation suggestions to improve PIC, as described below.

View this table:
[Table 2](http://www.annfammed.org/content/19/2/148/T2)

Table 2 
PIC Engagement and Fidelity Measures

#### Staffing

Participants suggested PIC would benefit from additional full-time MHPs, resource center intake coordinators, supervising psychiatrists, and a dedicated PIC program manager to both meet current referral volume and support desired expansion. A program manager and additional resource center staff were ultimately hired, consistent with suggestions.

#### Expanded Training

The majority of primary care clinicians and MHPs desired additional training on psychiatric treatment more broadly and the PIC model in particular. Recommended trainings included psychiatry-led training for primary care on medication management; refresher trainings for all on the PIC model, roles, and structure; and trainings for MHPs on specific conditions (eg, insomnia, chronic pain) and treatment approaches.

#### Expanded Services

Stakeholders noted that there was sometimes a lack of specialty care referral sites for patients whose needs could not be met within primary care or when the MHP’s caseload was full. The primary care clinicians and MHPs expressed interest in both expanding the mental health services currently provided in the health system and expanding the model to offer patients with more serious psychiatric morbidities care as part of PIC.

#### Increasing Engagement and Collaboration

Clinicians and leadership initially expressed a desire for more interdisciplinary meetings (eg, between primary care clinicians and MHPs, opportunities for primary care clinicians to directly consult with psychiatrists). This collaboration was viewed as critical to ensuring fidelity to the model. Program leaders also articulated the importance of MHP adherence to the PIC model (eg, integration with the primary care practice, graduating patients from short-term treatment). Conversely, MHPs reported that some primary care clinicians used warm handoffs whenever patients displayed strong emotions (eg, crying), while others used them only in acute, high-risk situations (eg, when patients had risk for suicide). In response to these suggestions, opportunities for consultation and collaboration were arranged and encouraged. Primary care clinicians received guidance on the appropriate use of warm handoffs. MHPs’ caseloads were reviewed, and they were given feedback to support fidelity to the model. They also received additional coaching in how to support primary care clinicians in managing challenging (but not crisis) situations and ongoing support from leadership in integrating into the practices.

#### Addressing Workflow and EHR Challenges

Leaders and primary care clinicians emphasized the need to address initial inefficiencies within the EHR. Leaders noted the importance of these records to facilitate referrals, capture care, facilitate billing, and allow for program evaluation. They emphasized that building this infrastructure was time-consuming and tedious, and required ongoing adjustment and support. These EHR transformations were part of a suite of changes to primary care clinicians’ workflow, which presented challenges during program adoption, but were iteratively refined throughout implementation. The clinicians noted consulting with a psychiatrist electronically was helpful, although EHR messages and alerts regarding patients (ie, documenting all communication and encounters between mental health providers and patients) were sometimes overwhelming. Primary care clinicians acknowledged that this level of communication and collaboration was a hallmark of CoCM and necessary for billing purposes. They suggested improvements to the EHR alerts to avoid excessive notifications.

#### Financing and Making the Business Case

Leaders emphasized that PIC be self-sustaining through billing using CoCM codes. Initially, the program was operating at a loss, although this is no longer the case. Clinicians expressed concern that the program would be short lived or unsustainable long term because of low reimbursement rates. The importance of evaluating the program using varied metrics (including patient outcomes, clinician burnout, and satisfaction) were suggested as ways to demonstrate value that could potentially offset concerns about the program’s financial risk.

### Adoption and Maintenance

PIC implementation began in 8 primary care clinics located in Philadelphia having 4 to 26 primary care clinicians. These practices served 90,038 empaneled patients (approximately 3,900 to 18,000 per practice). Initially, the Primary Care Service Line offered PIC to these 8 practices. All adopted the program and began implementation in January 2018. The practices bill for PIC services using CoCM Current Procedural Terminology codes.13 Initial trainings (eg, training in the model, the referral process for primary care clinicians, and brief interventions for the MHPs) were well received and were refined based on feedback.

All of the 8 practices that initially implemented PIC continued to implement the program as of September 2020, demonstrating 100% sustainability. Now that the program has shown financial viability, expansion is in progress. PIC has expanded to 2 new primary care practices, and approval has been granted to hire staff to expand to an additional 4 practices, for a total of 14 PIC practices anticipated in the next year. The stakeholders interviewed expressed support for the program’s continuation and expansion.

## DISCUSSION

Penn Medicine, like many health systems, is implementing CoCM. The PIC program is different in its use of a centralized resource center to facilitate intake, triage, and referral management activities for all patients with mental health needs in primary care, including those who require specialty services in the community. Results of this quality improvement effort suggest that implementing collaborative care with the addition of a centralized resource center was associated with referral of 6.8% of patients in the primary care practices for mental health services. Effectiveness and implementation data suggest CoCM was implemented with fidelity in PIC. Patients with more severe mental health needs were connected to care in the community. Metrics of adoption and maintenance indicate primary care practices were eager to participate and have sustained the program.

Stakeholders viewed the PIC program as a “long overdue” improvement to the quality of services provided by the health system, meeting the previously unmet needs of patients with the full spectrum of mental health problems encountered in primary care. Primary care clinicians and patients were especially enthusiastic about the PIC program and the collaboration between primary care and MHPs. Program leaders and clinicians remarked that the changes in clinician workflows, transformations to the EHR, and new reimbursement codes presented challenges initially, and they provided concrete suggestions that were addressed during implementation.

PIC addresses a common challenge in CoCM implementation by leveraging the resource center. In the absence of identification, triage, and referral supports, such as those provided by the PIC Resource Center, individuals in need of mental health services may not receive them at all6 or they may receive suboptimal services (eg, subtherapeutic antidepressant dosages)34 in the primary care setting because of challenges navigating services in the community. By using structured, validated assessment tools paired with decision support software, the PIC Resource Center can connect patients to the most appropriate level of care while allowing MHPs to focus on intervention delivery. Stakeholders found the resource center to be an efficient and cost-effective way to coordinate and risk-stratify primary care patients’ mental health needs when compared with the traditional CoCM that uses a care manager to coordinate patients’ care. MHPs appreciated the thorough assessments the resource center conducted—data they used in their initial mental health visits with patients—and the ability to devote time to psychotherapy. Primary care clinicians, who may have limited time to screen for and treat mental health concerns, appreciated that they had a place to refer patients. Future research is needed to examine the impact of this model on clinician and patient burden (eg, time spent navigating through multiple clinicians to find one with availability and the appropriate expertise).

Rates of enrollment, symptom reduction, and remission evidenced in the PIC program’s first year of implementation are promising and generally consistent with those reported in seminal randomized controlled trials of CoCM such as the Improving Mood-Promoting Access to Collaborative Treatment (IMPACT) trial35 and the Prevention of Suicide in Primary Care Elderly: Collaborative Trial (PROSPECT),36 and randomized controlled trials of depression treatment such as the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial.37 The length and frequency of treatment observed suggest CoCM was implemented with fidelity to the model, which is particularly remarkable given that this was a quality improvement effort, and effectiveness often declines when programs move from efficacy trials to real-world implementation.38,39 Ongoing expansion efforts underscore the success of the initial implementation; the use of stakeholder feedback allowed implementation challenges to be addressed early, which may have contributed to the program’s success. The health system has committed to adding PIC to additional primary care practices, and these practices have accepted the opportunity.

Although this work provides an advance in CoCM, limitations exist. First, this study was conducted in a single health system as a quality improvement initiative, so our ability to comment on representativeness is limited. It is unknown, for example, how many patients in PIC practices had a visit during the year that included mental health screening (although mental health screening with the PHQ is mandated annually) or how many had existing mental health care and thus would not have been referred to the center even if they screened positive. Second, the PIC program is relatively new; long-term outcome data are not yet available. A higher proportion of patients were referred to community-based care in this first year because of ineligible insurance or because MHP caseloads were full.

Our results demonstrate the promise of the PIC model. It will be important to continue to evaluate the model as staffing increases and the program is expanded to additional clinics within Penn Medicine. Future studies implementing PIC in other health systems also are needed. As Penn Medicine and other health systems work to expand or validate the PIC model, or both, data about the generalizability of their findings will emerge.

## Footnotes

*   * These authors contributed equally as co–first authors.

*   Conflicts of interest: Dr Oquendo received royalties for the commercial use of the Columbia Suicide Severity Rating Scale and owns shares in Mantra, Inc. Her family owns stock in Bristol Myers Squibb. The other authors report no conflicts of interest.

*   To read or post commentaries in response to this article, go to [https://www.AnnFamMed.org/content/19/2/148/tab-e-letters](https://www.AnnFamMed.org/content/19/2/148/tab-e-letters).

*   **Funding support:** Funding for this research project was supported by a Center for Therapeutic Effectiveness Research PCOR pilot grant to Dr Wolk and the University of Pennsylvania’s Center for Mental Health (CMH) and Center for Health Incentives and Behavioral Economics (CHIBE) National Institute of Mental Health-funded ALACRITY (P50 MH113840; MPIs Beidas, Mandell, Volpp) Center pilot program. Ms Last was supported by the National Science Foundation Graduate Research Fellowship (DGE-1321851).

*   **Disclaimer:** The views expressed are solely those of the authors and do not necessarily represent official views of the authors’ affiliated institutions or funders.

*   **Previous presentations:** Parts of this work were previously presented at the annual meetings of the Association for Behavioral and Cognitive Therapies; November 21-24, 2019; Atlanta, Georgia, and the Academy Health Conference on the Science of Dissemination and Implementation in Health; December 4-6, 2019; Arlington, Virginia.

*   **Supplemental materials:** Available at [https://www.AnnFamMed.org/content/19/2/148/suppl/DC1/](https://www.AnnFamMed.org/content/19/2/148/suppl/DC1/).

*   Received for publication September 18, 2019.
*   Revision received September 28, 2020.
*   Accepted for publication October 5, 2020.


*   © 2021 Annals of Family Medicine, Inc.

## References

1.  1.Archer J, Bower P, Gilbody S, et al. Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev. 2012; 10: CD006525.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1002/14651858.CD006525.pub2&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=23076925&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 

2.  2.Miller CJ, Grogan-Kaylor A, Perron BE, Kilbourne AM, Woltmann E, Bauer MS. Collaborative chronic care models for mental health conditions: cumulative meta-analysis and metaregression to guide future research and implementation. Med Care. 2013; 51(10): 922-930.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1097/MLR.0b013e3182a3e4c4&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=23938600&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000324751000011&link_type=ISI) 

3.  3.Woltmann E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, Bauer MS. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am J Psychiatry. 2012; 169(8): 790-804.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1176/appi.ajp.2012.11111616&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=22772364&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000306888900007&link_type=ISI) 

4.  4.Oslin DW, Grantham S, Coakley E, et al. PRISM-E: comparison of integrated care and enhanced specialty referral in managing at-risk alcohol use. Psychiatr Serv. 2006; 57(7): 954-958.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1176/appi.ps.57.7.954&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16816279&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000238706800006&link_type=ISI) 

5.  5.Krahn DD, Bartels SJ, Coakley E, et al. PRISM-E: comparison of integrated care and enhanced specialty referral models in depression outcomes. Psychiatr Serv. 2006; 57(7): 946-953.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1176/appi.ps.57.7.946&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16816278&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000238706800005&link_type=ISI) 

6.  6.Cunningham PJ. Beyond parity: primary care physicians’ perspectives on access to mental health care. Health Aff (Millwood). 2009; 28(3)(Suppl 1): w490-w501.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjk6IjI4LzMvdzQ5MCI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTkvMi8xNDguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

7.  7.Mazzucchelli T, Kane R, Rees C. Behavioral activation treatments for depression in adults: a meta-analysis and review. Clin Psychol Sci Pract. 2009; 16(4): 383-411.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1002/cpp.620&link_type=DOI) 

8.  8.Coventry PA, Hudson JL, Kontopantelis E, et al. Characteristics of effective collaborative care for treatment of depression: a systematic review and meta-regression of 74 randomised controlled trials. PLoS One. 2014; 9(9): e108114.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1371/journal.pone.0108114&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=25264616&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 

9.  9.Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001; 16(9): 606-613.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1046/j.1525-1497.2001.016009606.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11556941&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000171184700005&link_type=ISI) 

10. 10.Tew J, Klaus J, Oslin DW. The Behavioral Health Laboratory: building a stronger foundation for the patient-centered medical home. Fam Syst Health. 2010; 28(2): 130-145.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1037/a0020249&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=20695671&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 

11. 11.Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999; 89(9): 1322-1327.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.2105/ajph.89.9.1322&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10474547&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000082214600005&link_type=ISI) 

12. 12.RE-AIM. Accessed Jan 13, 2020. [https://www.re-aim.org](https://www.re-aim.org)
    
    

13. 13.Press MJ, Howe R, Schoenbaum M, et al. Medicare payment for behavioral health integration. N Engl J Med. 2017; 376(5): 405-407.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 

14. 14.Oslin DW, Ross J, Sayers S, Murphy J, Kane V, Katz IR; The Behavioral Health Laboratory. Screening, assessment, and management of depression in VA primary care clinics. J Gen Intern Med. 2006; 21(1): 46-50.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1525-1497.2005.0267.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16423122&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000235163600007&link_type=ISI) 

15. 15.Oslin DW, Klaus J, Ingram E, et al. Foundations for integrated care. Accessed Mar 25, 2019. [https://www.mirecc.va.gov/visn4/BHL/BHLresources3.asp](https://www.mirecc.va.gov/visn4/BHL/BHLresources3.asp)
    
    

16. 16.Malouff JM, Thorsteinsson EB, Schutte NS. The efficacy of problem solving therapy in reducing mental and physical health problems: a meta-analysis. Psychol Rev. 2007; 27(1): 46-57.
    
    

17. 17.Mignogna J, Hundt NE, Kauth MR, et al. Implementing brief cognitive behavioral therapy in primary care: a pilot study. Transl Behav Med. 2014; 4(2): 175-183.
    
    

18. 18.Lundahl B, Moleni T, Burke BL, et al. Motivational interviewing in medical care settings: a systematic review and meta-analysis of randomized controlled trials. Patient Educ Couns. 2013; 93(2): 157-168.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.pec.2013.07.012&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24001658&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 

19. 19.Spitzer RL, Williams JB, Kroenke K, et al. Test review: Patient Health Questionnaire–9 (PHQ-9). Rehabil Couns Bull. 2014; 57(4): 246-248.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1177/0034355213515305&link_type=DOI) 

20. 20.Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006; 166(10): 1092-1097.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/archinte.166.10.1092&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16717171&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000237720500008&link_type=ISI) 

21. 21.Löwe B, Spitzer RL, Williams JB, Mussell M, Schellberg D, Kroenke K. Depression, anxiety and somatization in primary care: syndrome overlap and functional impairment. Gen Hosp Psychiatry. 2008; 30(3): 191-199.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.genhosppsych.2008.01.001&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=18433651&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000255428900001&link_type=ISI) 

22. 22.Weathers FW, Litz BT, Keane TM, Palmieri PA, Marx BP, Schnurr PP. The PTSD checklist for DSM-5 (PCL-5). Published 2013. Accessed Aug 1, 2019. [https://www.ptsd.va.gov](https://www.ptsd.va.gov)
    
    

23. 23.Blevins CA, Weathers FW, Davis MT, Witte TK, Domino JL. The Post-traumatic Stress Disorder checklist for DSM-5 (PCL-5): development and initial psychometric evaluation. J Trauma Stress. 2015; 28(6): 489-498.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1002/jts.22059&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=26606250&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 

24. 24.Bovin MJ, Marx BP, Weathers FW, et al. Psychometric properties of the PTSD Checklist for Diagnostic and Statistical Manual of Mental Disorders–Fifth Edition (PCL-5) in veterans. Psychol Assess. 2016; 28(11): 1379-1391.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1037/pas0000254&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 

25. 25.Wortmann JH, Jordan AH, Weathers FW, et al. Psychometric analysis of the PTSD Checklist-5 (PCL-5) among treatment-seeking military service members. Psychol Assess. 2016; 28(11): 1392-1403.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1037/pas0000260&link_type=DOI) 

26. 26.1.  Litten RZ, 
    2.  Allen JP
    
    Sobell L, Sobell MB. Timeline follow-back: a technique for assessing self-reported alcohol consumption. In: Litten RZ, Allen JP, eds. Measuring Alcohol Consumption: Psychosocial and Biochemical Methods. Totowa, NJ: Humana Press; 1992: 41-72.
    
    

27. 27.Breslin FC, Borsoi D, Cunningham JA, Koski-Jännes A. Help-seeking timeline followback for problem drinkers: preliminary comparison with agency records of treatment contacts. J Stud Alcohol. 2001; 62(2): 262-267.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11327193&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 

28. 28.Manwell LB, Fleming MF, Mundt MP, Stauffacher EA, Barry KL. Treatment of problem alcohol use in women of childbearing age: results of a brief intervention trial. Alcohol Clin Exp Res. 2000; 24(10): 1517-1524.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1530-0277.2000.tb04570.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11045860&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000089884000009&link_type=ISI) 

29. 29.Sobell LC, Maisto SA, Sobell MB, Cooper AM. Reliability of alcohol abusers’ self-reports of drinking behavior. Behav Res Ther. 1979; 17(2): 157-160.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/0005-7967(79)90025-1&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=426744&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1979GM75500010&link_type=ISI) 

30. 30.1.  Rush AJ, 
    2.  First MB, 
    3.  Blacker D
    
    Sobell LC, Sobell MB. Alcohol timeline followback (TFLB). In: Rush AJ, First MB, Blacker D, eds. Handbook of Psychiatric Measures. Washington, DC: American Psychiatric Association; 2000: 477-479.
    
    

31. 31.American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 5th edition – DSM 5. Washington, DC: American Psychiatric Association; 2013.
    
    

32. 32.WHO ASSIST Working Group. The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): development, reliability and feasibility. Addiction. 2002; 97(9): 1183-1194.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1046/j.1360-0443.2002.00185.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12199834&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000177981700016&link_type=ISI) 

33. 33.Eldridge SM, Chan CL, Campbell MJ, et al; PAFS consensus group. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot Feasibility Stud. 2016; 2: 64.
    
    

34. 34.Simon GE. Evidence review: efficacy and effectiveness of antidepressant treatment in primary care. Gen Hosp Psychiatry. 2002; 24(4): 213-224.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/S0163-8343(02)00198-6&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12100832&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000176679900004&link_type=ISI) 

35. 35.Unützer J, Katon W, Callahan CM, et al. Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA. 2002; 288(22): 2836-2845.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.288.22.2836&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12472325&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000179732600028&link_type=ISI) 

36. 36.Alexopoulos GS, Katz IR, Bruce ML, et al; PROSPECT Group. Remission in depressed geriatric primary care patients: a report from the PROSPECT study. Am J Psychiatry. 2005; 162(4): 718-724.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1176/appi.ajp.162.4.718&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15800144&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000228040600014&link_type=ISI) 

37. 37.Sinyor M, Schaffer A, Levitt A. The Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial: a review. Can J Psychiatry. 2010; 55(3): 126-135.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1177/070674371005500303&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=20370962&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 

38. 38.March JS, Silva SG, Compton S, Shapiro M, Califf R, Krishnan R. The case for practical clinical trials in psychiatry. Am J Psychiatry. 2005; 162(5): 836-846.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1176/appi.ajp.162.5.836&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15863782&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000228833100002&link_type=ISI) 

39. 39.Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health. 2003; 93(8): 1261-1267.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.2105/AJPH.93.8.1261&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12893608&link_type=MED&atom=%2Fannalsfm%2F19%2F2%2F148.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000184549100023&link_type=ISI)