A Stepped-Wedge Evaluation of an Initiative to Spread the Collaborative Care Model for Depression in Primary Care

Following a large number of RCTs providing compelling evidence for the effectiveness of collaborative care for depression, Solberg and colleagues embarked on the mission to implement the intervention in primary care. For this effort alone they should be applauded, for this is not an endeavor undertaken lightly. Their choice of a stepped-wedge evaluation and the statewide approach to overcome implementation barriers (such as reimbursement problems), have resulted in a very informative article that provides food for thought for those who want to improve depression care.

Concerning the conclusion by the authors that the initiative was unable to produce results similar to clinical trials, a side remark is in order. The remission percentages are actually higher in all groups than in the original IMPACT-study (1) on which DIAMOND was based. We should note however that the study population was older in the IMPACT-trial. However, in the CC:DIP-trial (2) by our own research group with a more comparable age group remission percentages were actually lower than in the study by Solberg and colleagues. Therefore our conclusion would be that the intervention actually did produce results similar (or even exceeding) to those in RCTs. The lack of difference with the control groups is most likely due to exceptionally effective usual care (as the authors note). Another factor that might have diluted the effect of the intervention could have been the relative absence of brief psychotherapy in the current study, as this has been identified as one of the most powerful elements of collaborative care in those interventions where it was added on a structural basis (3).

The good news of the current study is that it proved possible to implement relatively effective care for depression on a statewide basis. The bad news is that even with state of the art care remission percentages remain low at about 35%. This then appears to be a key problem: although major depressive disorder can be treated effectively in a number of patients, an even larger number of patients benefits insufficiently. It is a major challenge to predict which patients will profit from (specific elements of) collaborative care. Much work in that field remains to be done, but there is evidence that effectiveness is most pronounced in patients with more severe depression, patients with comorbid medical conditions and high utilizers of healthcare (4). At least for these groups implementation efforts should be continued at full pace.

(1) Unutzer J, Katon W, Callahan CM, Williams JW Jr, Hunkeler E, Harpole L, et al. Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA. 2002;288:2836-45.

(2) Huijbregts KML, de Jong FJ, Van Marwijk HWJ, Beekman ATF, Ader HJ, Hakkaart-van Roijen L, et al. A target-driven Collaborative care model for Major Depressive Disorder is effective in primary care in the Netherlands. A Randomised Clinical Trial from the Depression Initiative. J Affect disord. 2013;146(3):328-337.

(3) Coventry PA, Hudson JL, Kontopantelis E, Archer J, Richards DA, Gilbody S et al. Characteristics of effective collaborative care for treatment of depression: a systematic review and meta-regression of 74 randomised controlled trials. PLoS One. 2014;9(9).

(4) Beekman ATF, Van der Feltz-Cornelis CM, Van Marwijk HWJ. Enhanced care for depression. Curr Opin Psychiatry. 2013;26:7-12.

Competing interests: None declared

As a member of the DIAMOND (Depression Improvement Across Minnesota- Offering a New Direction) collaborative, we read with interest the results of the recent publication by Solberg et al.(1) They found no difference in six month remission rates despite enhanced compensation from insurance carriers at involved clinics for depressed adult patients receiving collaborative care management (CCM) versus usual primary care (UC) across multiple sites. Although CCM has been extensively evaluated in randomized controlled trials and found to consistently be effective for management of depression,(2-4) implementation and dissemination of CCM into primary care practices is not without its challenges, as this study clearly demonstrated.

Solberg et al provided a candid review of potential limitations of this carefully designed study.(1) It is possible that the study made it more difficult for clinics using DIAMOND to differentiate from practice as usual. This could be due to delays in enrollment (assuming DIAMOND patients might show a faster improvement) or perhaps that more complex patients (for which DIAMOND might show a greater difference in outcomes) were not adequately represented. However, it is just as likely that this study was an excellent example of the challenges of real-world implementation, especially across multiple sites and clinical practices.

Moving from research to real life is a tough transition. In a standard randomized control trial, there are built- in measures of fidelity to monitor intervention components and enable the study team to identify and address variabilities that are commonly seen. Using the Plan- Do- Study- Act analogy, practices are generally more experienced with the Plan and the Do; but do they also excel in the Study and Act cycles so critical to ongoing improvement and sustained benefit? There will always be variability between sites and it is important to establish benchmarks and work with sites to achieve their goals. This study was not the only example of implementation of components of care coordination with minimal evidence of return on investment.(5)

The lessons from these studies suggest a need for greater attention to implementation science. How will practices respond? It is not easy to preserve the core change concepts while making the local adaptations needed for success. For example, during the time frame of the DIAMOND project, patient centered medical homes (PCMH) were being implemented at many of the DIAMOND clinics across the state and there was a tension to utilize DIAMOND care managers for comprehensive complex care coordination. There may also have been pressure to pull the care coordinators into acute care active activities.

Multiple factors impact implementation of new health care models, even those as effective as CCM. We would view this study as a testament to the significant challenges of multi-site implementation of health care process change rather than a statement about the potential impact of CCM on the care of patients with depression.

References
1. Solberg LI, Crain AL, Maciosek MV, et al. A Stepped-Wedge Evaluation of an Initiative to Spread the Collaborative Care Model for Depression in Primary Care. Ann Fam Med. Sep 2015;13(5):412-420.
2. Unutzer J, Katon W, Callahan CM, et al. Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. Jama. Dec 11 2002;288(22):2836-2845.
3. Thota AB, Sipe TA, Byard GJ, et al. Collaborative care to improve the management of depressive disorders: a community guide systematic review and meta-analysis. Am J Prev Med. May 2012;42(5):525-538.
4. Archer J, Bower P, Gilbody S, et al. Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev. 2012;10:CD006525.
5. Peikes D, Chen A, Schore J, Brown R. Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials. Jama. Feb 11 2009;301(6):603-618.

Competing interests: A member of the DIAMOND collaborative.

The aim of this study was to test whether a collaborative care model for depression treatment in primary care - the Depression Improvement Across Minnesota-Offering a New Direction (DIAMOND) - led to improved patient outcomes as compared to traditional, clinical depression management. The authors consider collaborative models of care to be a significant direction for future psychiatric care in the primary care setting, citing 79 randomized control trials that demonstrate collaborative care treatment as superior to traditional depression management - an impressive number of studies that our discussion group found intriguing as we have seen minimal collaborative care for psychiatry implemented in Rockford, Illinois (the third largest city in Illinois). It was unclear to the group whether similar large-scale studies have taken place, and if so how large a patient pool and was did those trials reach clinical significance. Having this information to set the scene would point us to the unique questions of this study.

A disappointing fact the authors cite in their introduction is that collaborative care, while shown in studies to be effective, is still not implemented even in the very clinics that participated in the randomized control trials. The authors reference financial coverage as a potential barrier to implementation of collaborative care, but the student group wondered whether this was the primary problem in making practical implementation difficult to achieve. The LP Johnson Family Health Center in Rockford, IL is lucky to have people on staff that can provide collaborative care, but most other clinics in the area would be unable to achieve this because of staffing. Our discussion of this study often came back to this constraint - so while the authors cited that collaborative care is "cost-effective and even cost-saving" it would not be practical for many primary care centers.

The dramatic need for psychiatrists is something our group of medical students has experienced first-hand, with each of us having patients who required a psychiatrist; yet, wait times are so extreme that the primary care physician takes over psychiatric care - care they are sometimes uncomfortable managing. Including statistics on numbers of psychiatrists, case managers, and councilors would be of benefit to this paper as this is certainly a barrier to implementation. So while this study did achieve novel financial payments for collaborative care, the lack of professionals who could receive training and the tools for this project would hinder implementation in many regions - particularly rural regions. Similarly, perhaps in more populated areas a case manager's load is already too heavy that adding more patients into the mix may actually hinder their patient care. Overall, the group as a whole was concerned with the shortage of healthcare workers in implementing such a collaborative care model. The paper does make mention of many barriers, but we don't particularly feel there was a potential solution presented in the end.

The DIAMOND initiative implementation was an impressive task including more than ten-thousand patients enrolled, a fact that our group believes elevated this study but may have led to its inability to achieve much statistical significance, as many unchecked variables likely were remaining during the study's data collection. We agreed that using the PHQ-9 was appropriate as it is widely used in primary clinics, and we thought the use of WPAI was important as well in determining depression's morbidity. Also discussed was the potential to use more objective measurements such as cortisol levels or sleep time improvements in future studies. A longer-term study could also examine suicide rates among participants, as a collaborative model may increase patient adherence to treatment and a better physician-patient relationship.

The "question about satisfaction" described in the methods could also be developed further to describe what aspect of care patients found better. Defining what aspect of satisfaction was useful would be advantageous for future studies. The 7-component model we considered was adequately explained and we appreciate the mention of treatment intensification for patients not improving - showing the study's ethical dedication to the patient above the rigors of keeping an unbiased study. By changing the treatments through the study the data was likely skewed, but creating a study without that rule would be a disservice to the patients and the reality of the true clinical practice of medicine. To conclude the methodology section, the group as a whole would have been interested in reading more about the role of the case manager in the collaborative care model as we have little to no experience with this in psychiatric care in Rockford, IL. Giving a brief overview of the responsibilities would have made it easier for us to understand some of the benefits of collaborative care and aspects of the model we could introduce into our own future practices.

The group would have liked a further description of the training each clinic received. In a concise paper such as this it may have been difficult, but part of the training of the clinics was felt to be a weak point in the study. It was hypothesized that the large scale of the study likely caused inconsistency of the training received by clinicians - an issue other smaller, randomized control trials would likely not have encountered to the same degree. That's not to say we are critiquing the training method but suggesting that the authors could highlight that education on this magnitude could be a considerable factor in widespread implementation. Ideally training would be consolidated to a few individual trainers who then observed clinics for correct implementation of collaborative care. If not, then we could see how this study's many simultaneously moving pieces would be overwhelming. What divided our group was the size of the study. Ten-thousand patients involved is a significant number that leads to questions of feasibility - seeing as how implementation was not simply prescribing a novel drug or attempting a new surgical procedure - as psychiatry is multifaceted, involving treatments that are heterogeneously administered. The 7 components of DIAMOND still involve individual clinicians' interpretations and implementations. So while the patient size is large perhaps the study was biting off more than it could manage, with too many variables at play.

The student discussion group argued that though there were many clinics involved (75 in total) it would have been interesting to know what barriers led to only 75 out of the possible 280 invited to participate while the others passed. A follow-up survey to those who dropped out could potentially be useful, as an important aspect of the collaborative care problem of implementation is why participation in such programs is so poor. Could the concerns be related to reimbursement procedures, difficulty implementing DIAMOND on top of an existing care structure, or do clinics think they are fine as is and don't need these questions answered?

As for participants, we wondered whether this collaborative care model may have reached statistical significance had the patients been more complicated and/or difficult to treat cases of major depressive disorder. Having no other information, we would believe that collaborative care would be more beneficial to refractory cases than depression that could be ameliorated with an antidepressant alone. The exclusion criteria for patients may have been better defined to generate a homogenized group of similar major depressive disorder. Our group also would want to know the time range for "recently" starting on antidepressants for the participant selection.

For the data, Figure 1 led to some confusion among the participants in the group. The patient flow diagram we followed until the end where we assumed for example 697 UCB correlated to 308 further down the chart having completed the study. However, this logic doesn't follow for 340 DCA becoming n=559 completing the study. Further clarity of this diagram would be useful.

The results of the study were somewhat disappointing as it demonstrated that the large scale implementation was not leading to statistically significant differences in patient care for depression treatment, leading only to clinical significance in patient satisfaction. We were all surprised by these results as 79 randomized control trials and 10 meta- analyses showed significance. The 4-week delay in obtaining PHQ-9 base levels could have been a problem, suggesting a later study could address this issue. However, the paper was felt by the group to be weakened by the mention that Minnesota is already too good at treatment to benefit from collaborative care. We hypothesized that a Medicaid/Medicare population may benefit more from collaborative care models as this patient population may have other barriers to care.

The group discussed how we might approach this study's questions. We think a more scaled down study would have been beneficial to the study results. Also, collecting the reasons for non-participation in the study could have been valuable information. There may also have been more exclusion criteria for participants such as the severity of their depression and other co-morbid conditions.

Overall, the group had mixed feelings about the takeaway from this study. Many in the group think the collaborative care model is going to be an invaluable tool for the future of medicine regardless of the results of this one study. Having an increased focus on individual attention to each patient - as supported by the satisfaction data - is a goal all of us should strive for. For this study what aspects specifically led to better satisfaction? With better satisfaction comes greater patient adherence to treatment plans in the long run and builds the physician-patient relationship, which when dealing with psychiatric diseases is of utmost importance. So while the statistics demonstrate problems with implementation on a large scale, individually our group unanimously considered collaborative care a goal for patient care we would all implement.

Competing interests: None declared

Thanks for your thoughtful and complimentary letter. Your point is an excellent one, but I am less confident about your solution. I don't think the problems of the RCT can be solved with another RCT. It seems to me that randomizing at the patient level will inevitably introduce similar practice disruptions and researchy needs for assuring consistent inclusion and randomization, even without consent. Instead, I think we need evaluations like ours of real life implementations, but with more intensive measures than we conducted of the depth of implementation, not just whether desired processes are in place. This would be even better if enough patients/site could be included that would allow the measurement of site-specific differences in process and outcomes, an advantage we did not have.

Competing interests: None declared