Abstract
PURPOSE Many individuals with behavioral health challenges receive services in primary care, and integrated behavioral health (IBH) programs can help increase access to evidence-based interventions. IBH programs can benefit substantially from integrating standardized tracking databases that allow for the implementation of measurement-based care to evaluate patient-, clinician-, and practice-level outcomes. We describe the development and integration of Mayo Clinic’s pediatric and adult primary care psychotherapy tracking database.
METHODS IBH practice leaders directed the development of a large psychotherapy tracking database that continuously populates from Mayo Clinic’s electronic health record system. The database captures numerous patient variables including demographics, behavioral health and substance use issues, psychotherapy principles used, and self-reported symptoms. We retrieved current data for patients empaneled in Mayo Clinic’s pediatric and adult primary care psychotherapy programs from June 2014 to June 2022.
RESULTS The tracking database contained data for 16,923 adult patients and 6,298 pediatric patients. The mean age of adult patients was 43.2 years (SD 18.3), 88.1% were non-Latine White, and 66.7% identified as female. The mean age of pediatric patients was 11.6 years (SD 4.2), 82.5% were non-Latine White, and 56.9% identified as female. We provide examples of practical applications of the database across clinical, educational, research, and administrative domains.
CONCLUSIONS The development and integration of a psychotherapy tracking database supports clinician communication, examination of patient outcomes, practice quality improvement efforts, and clinically relevant research. Our description of Mayo Clinic’s IBH database may serve as a model for other IBH practices.
- integrated behavioral health
- measurement-based care
- electronic health records
- health informatics
- quality improvement
- health metrics
- mental health
- psychotherapy
- primary care
- multidisciplinary research
- patient care team
- population health
INTRODUCTION
Behavioral health disorders in both children and adults are prevalent1-4 and burdensome, resulting in a marked degree of functional impairment.5,6 Although evidence-based interventions exist, many individuals needing treatment do not receive services.1,2,4,7-9 For example, data from the 2020 US National Survey on Drug Use and Health indicated that although 21% of adults experienced mental illness in the past year, less than one-half of these individuals received services.1 Similarly, data from the 2016 National Survey of Children’s Health reported a 16.5% prevalence of at least 1 mental health disorder among US children and estimated that nearly one-half of those affected did not receive needed care.4 Compounding inequities in access to behavioral health treatment have been observed for marginalized racial and ethnic groups.10-12 The adverse behavioral health impact of the COVID-19 pandemic has further highlighted the need for increasing access to evidence-based services.13,14
For a variety of potential reasons, including mental health stigma and barriers to accessing referrals, many people struggling with behavioral health concerns seek support and receive services in a primary care setting.15-19 Consequently, primary care, family medicine, and pediatric clinics are ideal settings for identifying patients in need and providing evidence-based behavioral health services. Even so, formal screening for behavioral health concerns is not always routine practice in these settings.18,20,21 The adoption of integrated behavioral health (IBH) programs within these settings may help enhance screening consistency22 and increase access to services.23-27 Models integrating behavioral health services into primary care have been shown to be associated with important patient-, clinician-, and system-level outcomes such as patient engagement and satisfaction,28 improvements in emotional health,29-32 clinician ability to deliver patient care,33 and cost-effectiveness,34-36 although the literature examining direct links between specific psychotherapy services provided in primary care settings and various clinical outcomes is limited.30,37
Standardized approaches to screening and tracking a patient’s behavioral health progress and outcomes over time are critical to providing the highest quality of integrated services. Systems of measurement-based care (MBC) use standardized patient-reported outcomes to evaluate patient symptoms, inform treatment decisions, and monitor potential unwanted or disadvantageous effects over the course of care.38,39 Providing this type of feedback to patients and clinicians has been found to improve care quality and therapeutic outcomes.40-44
Although MBC has widely stated benefits, a recent review reported that less than 20% of behavioral health professionals use MBC in practice.39 Barriers to implementing MBC may include limited education for clinicians about the utility of these activities,45 time burdens for patients and clinicians,39 and limited organizational or system-level support.39 One important strategy to overcoming barriers to MBC implementation is the use of measurement-based feedback systems, health care technologies that aid in collecting and organizing data.39,46 Patients, clinicians, and health care practices likely receive the most benefit from the implementation of universal screening and MBC methods within integrated care settings when data are easily tracked and organized into registries linked to electronic health records (EHRs) that allow for efficient and routinized examination of service use, processes, and outcomes.39,46-48 This further underscores the importance of building infrastructure to effectively identify and manage behavioral health concerns at the point of primary care, aided by the implementation of MBC using a formal data-tracking infrastructure.
In this article, we describe the development and implementation of an adult and pediatric psychotherapy tracking database used by Mayo Clinic’s multistate, multisite IBH program in primary care. A tracking database specific to the psychotherapy program is an innovative approach to promoting continuous quality improvement in the delivery of evidence-based behavioral interventions at both the individual and population levels. Mayo Clinic IBH programs are embedded within several primary care clinics in Minnesota, Wisconsin, Iowa, Arizona, and Florida. Early publications from the practice have described Mayo Clinic’s IBH psychotherapy program, outlined initial efforts in the development and implementation of a psychotherapy tracking database, and reported on previous clinical practice outcomes.49-52 We provide an overview of the EHR process used to build an expansive and continuously populating psychotherapy tracking database, review the types of data available in the database, present descriptive statistics, and discuss specific ways in which the database functions to inform and improve the practice. We also offer examples of the practical utility of the database as applied across clinical, educational, research, and administrative domains.
METHODS
Population
We drew data from medical records for all patients who participated in either the adult or pediatric IBH psychotherapy program from June 2014 through June 2022. IBH programs are embedded within 22 total clinics across Minnesota (12 clinics), Wisconsin (7 clinics), Iowa (1 clinic), Arizona (1 clinic), and Florida (1 clinic), with each clinic serving as a hub for several other primary care practices in its respective catchment areas.
To be eligible for participation in the adult psychotherapy program, patients had to be aged 18 years or older and empaneled in primary care at one of the Mayo Clinic sites. To be eligible for the pediatric psychotherapy program, patients had to be aged younger than 18 years and empaneled to primary care at 1 of the Mayo Clinic sites. Patients who turned 18 during their course of care in pediatric IBH remained included in the pediatric division of the database.
Given that the IBH program is a real-world clinical service, there are no patient-level characteristics that act as exclusionary criteria for entry into the database. Minnesota patients who do not provide Minnesota Research Authorization for research participation are excluded from research. We included all medical records before and after referral to the psychotherapy program.
Data Source
The main data file includes 1 record for each period in which the patient was enrolled in the psychotherapy program. The data source was the Mayo Clinic EHR system. In July 2017, Mayo Clinic transitioned from multiple EHR systems to one (Epic). This transition was an essential development in the evolution of efficiently tracking outcomes within primary care. Before having a unified model, 3 different medical records existed that were partially linked to an external data-tracking system. Furthermore, patient self-reported and parent-reported measures were completed on paper. This required manual entry into the tracking database, thereby increasing risk for data variability, entry error, and incompleteness.
Approximately 18 months before the launch of the new EHR, practice leaders prioritized leveraging the database functionality within Epic and redesigned documentation templates to automatically capture relevant outcomes and populate the database. Likewise, a new, fully automated system of obtaining patient self-reported and parent-reported measures using tablets and patient access to the EHR online allowed for a dramatic increase in the completeness and fidelity of data collection efforts. Using this method, self- and parent-reported measures, such as the 9-item Patient Health Questionnaire (PHQ-9), are then available for IBH clinicians to access and view before the start of a psychotherapy visit. Considerable efforts were then made over the last 2 years to merge the psychotherapy data between the original legacy system and the new EHR, thereby creating a unified data platform.
The current data structure is continuously populating, with data extraction into the registry occurring after a visit is documented and billed and the visit encounter is closed. This process allows for real-time evaluation of program outcomes.
Measures Included
The main data set captures the key domains of demographics; behavioral health and substance use categories; other medical categories using the Agency for Healthcare Research and Quality Clinical Classifications Software Refined53; summary information about the patient’s interaction with the psychotherapy program (including psychotherapy principles used); and questionnaire results (Table 1). Self- and parent-reported questionnaires selected for inclusion in the database consist of measures that are frequently used in primary care settings and are an existing part of the program’s clinical workflow, allowing for easy communication between interdisciplinary clinicians. For example, the PHQ-954 and Generalized Anxiety Disorder 7-item scale (GAD-7)55 are brief measures originally developed and validated in primary care settings. Importantly, these measures are also commonly used in other primary care practices, allowing for comparisons across different outcome studies. Psychotherapy program variables selected for inclusion in the database include important practice metrics such as number of individuals who received a consultation only, duration of an episode of care in the psychotherapy program, and referrals made to external professionals. The database also tracks evidence-based psychotherapy principles commonly delivered in primary care (eg, behavioral activation, situational exposure) that are consistent with the practice’s cognitive behavioral therapy training model.
Overview of Data Fields in the Psychotherapy Tracking Database
Table 1 details the specific measures included in the database. Depending on specific questions of interest, supplementary data sets outside of the primary working database can be built to include detailed information on encounters, medication use, social determinants of health, and questionnaire results that could not be included in the main data file because of file size restrictions.
RESULTS
Patient Characteristics
The database included 18,148 unique adult patients and 6,813 unique pediatric patients across sites. Out of the total population, we had research authorization for 16,923 (93.2%) of the adults and 6,298 (92.4%) of the children. Among these adult patients, the mean age was 43.2 years (SD 18.3), 88.1% were non-Latine White, and 66.7% identified as female. Among these pediatric patients, the mean age was 11.6 years (SD 4.2), 82.5% were non-Latine White, and 56.9% identified as female.
Practical Applications
The development and implementation of the large and continuously populating psychotherapy database creates major opportunities for practice improvement across multiple domains. Table 2 provides the rationale for and examples of practical applications of the database in clinical, educational, research, and administrative domains.
Practical Applications of a Psychotherapy Tracking Database Across Domains
DISCUSSION
The prevalence and impact of behavioral health problems in primary care highlight the importance of exploring flexible methods to increase access to evidence-based behavioral health services.16 Concurrently, it is crucial to evaluate patient- and practice-level data to understand the demographic and clinical makeup of the patient population served, track fidelity to evidence-based practice, gain insight into the link between specific psychotherapy principles and patient outcomes, identify potential targets for program development, and leverage resources to improve population-based care.
The development and implementation of Mayo Clinic’s IBH psychotherapy tracking database allows us to better understand the types of patients we serve (eg, base rates of certain conditions)56 and to study what we do, giving us the opportunity to conduct continuous quality improvement efforts to increase access and improve outcomes. Evaluations of data collected using previous iterations of the database have been promising as they have provided information about the effectiveness of primary care–based adapted cognitive behavioral interventions for anxiety, depression, and adjustment disorders.49,50,57 For a psychotherapy program integrated within primary care, these data are especially useful given that these practices are accustomed to using MBC in the management of other chronic medical conditions such as asthma58,59 and diabetes.60 Having broad-scale data, such as those housed in Mayo Clinic’s database, allows us to more effectively communicate with primary care administration about the role of IBH and our impact on their patient population, helps our service advocate for continued support and resources, and aids us in targeting resources toward areas of high potential impact.
Practical Applications
Implementing a formal data-tracking system based on the principles of MBC offers benefits to both patients and clinicians. It facilitates patient communication of symptoms to the treatment team and serves as a roadmap of treatment progress. Research has demonstrated improved psychotherapy outcomes with MBC compared with usual care,40,41,44 and specifically among patients at risk of not responding to treatment.42,43 The availability of a range of variables within the data set enables us to evaluate patients who are responders and nonresponders to the psychotherapy program. It is important to understand these predictors because they may offer opportunities to modify the program to improve outcomes for those at risk for nonresponse, or identify areas of need, for which the development of additional models of population-based care would especially have an impact. Summary data from the tracking database can also facilitate communications among interdisciplinary team members (eg, primary care physicians, nurse care coordinators, social workers), providing efficient indicators of patient progress to help support ongoing clinical decision making across disciplines. The unified EHR also affords the opportunity to characterize similarities and differences in the primary care population in terms of service use (eg, patients who follow through on IBH referrals vs those who do not), treatment engagement (eg, those who complete a course of care vs those lost to follow-up), and referral patterns (eg, psychotherapy in IBH vs in the community). The databases may also allow for exploration of longitudinal outcomes of patients who have received psychotherapy services in primary care during childhood, adulthood, or both.
The psychotherapy tracking database has direct implications for the practice as well. For example, evaluation of outcomes allows for the identification of high-yield clinician trainings. Through examination of clinician-level data, we can determine whether clinicians are using specific evidence-based psychotherapy principles when indicated by a particular mental health issue (eg, exposure therapy for patients with anxiety disorders),49 which may suggest additional training needs in specific approaches. Importantly, the database may ultimately allow us to explore whether patients with diverse personal identities or cultural backgrounds (eg, transgender patients), those with specific presenting concerns (eg, trauma), or those carrying a substantial burden of social determinants of health (eg, housing instability) are consistently experiencing worse outcomes or higher dropout rates across the practice. This information could then inform appropriate trainings to help clinicians improve their ability to deliver care to underserved and disadvantaged patient populations. Finally, practice-level data allow us to examine clinician adherence to the brief course of care model in IBH, which is critical in the practice’s population health mission of maintaining access in a timely manner.
Examination of practice-level data can aid in identifying additional service needs and developing program efforts for quality improvement. For example, given the frequency of depression, anxiety, and sleep problems in primary care,61 we developed a series of group therapy programs to help improve access to services.62 Over the course of the pandemic, both individual therapy and group therapy programs were adapted for a virtual format.63 The database can support direct comparisons between in-person and virtual treatment approaches, including both treatment outcomes and frequency of service use. Reporting on patient and practice outcomes to leadership is of critical importance when advocating for program-level change such as the need for additional clinicians or organization of clinician schedules (eg, to include unscheduled time blocks for curbside consultations). With many competing interests in a primary care setting, programs that use metrics and data to demonstrate improvements in patient outcomes are best positioned to receive institutional support.
Finally, a psychotherapy tracking database helps to support and enhance a program’s collaborative research mission. A large, rich, and continuously populating database enables teams to ask and answer important clinical research questions that are most relevant to understanding population health and patient care. It also supports interdisciplinary clinical research, bringing together a variety of professional perspectives such as psychology, psychiatry, family medicine, social work, nursing, and public health, to collaboratively present, publish, and improve the overall quality of research produced.
Limitations
Although the processes for design and implementation of the Mayo Clinic psychotherapy tracking database are promising, several limitations should be acknowledged. First, we describe the development of the database within a highly resourced system that provides strong institutional support for research and quality improvement; thus, these implementation processes may not generalize to other clinics or health systems. Additionally, we have not yet formally examined database user acceptability with either IBH or primary care team members, nor have we examined whether specific practical applications of the database have resulted in benefit to clinicians, supervisors, or specific patient populations. We have also not yet examined longitudinal outcome data. These explorations are beyond the scope of this article but will be important areas for future research.
Regarding limitations of the database itself, our patient population is mostly non-Latine White, which will limit future opportunities to explore racial and ethnic inequities and subgroup analyses of marginalized groups. Research outcomes may therefore not generalize well to more diverse patient populations. Furthermore, the database captures only health care use recorded in the Mayo Clinic EHR. Patients may receive care elsewhere that would not be captured. As these patients are empaneled in primary care at Mayo Clinic, however, it is likely that they receive most of their care within this practice. To compensate for this limitation, we plan to link the database to the Rochester Epidemiology Project, which has medical records for patients living in 27 counties in Minnesota and Wisconsin since 2010. The project includes data from institutions other than Mayo Clinic and covers a large proportion of the care received in these areas.64,65 Additionally, the current data capture is largely based on assessing symptoms and service use, and lacks measures of quality of life and functional outcomes. Finally, evolutions in psychometric measures for adults and children will need to be evaluated and assessed for inclusion in future iterations of the psychotherapy tracking database.
Conclusions
The development, implementation, and dissemination of a large-scale psychotherapy tracking database in Mayo Clinic’s adult and pediatric IBH program has been both a major undertaking and a worthy initiative for the practice. The ability to report on behavioral health outcomes in real-world settings is extremely limited at present. The psychotherapy tracking database built within the infrastructure of the IBH program serves as an innovative blueprint for other practices to follow to improve quality outcomes.
Footnotes
Conflicts of interest: authors report none.
Funding support: This work was funded by the US Department of Health and Human Services (HHS), Health Resources and Services Administration (HRSA) under cooperative agreement UH133881. Dr Savitz’s contribution to this research was made possible in part by the Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery.
Disclaimer: The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by HRSA, HHS or the US Government.
- Received for publication May 27, 2022.
- Revision received September 12, 2022.
- Accepted for publication October 27, 2022.
- © 2023 Annals of Family Medicine, Inc.
