Implementation and Effectiveness of Integrating Palliative Care Into Ambulatory Care of Noncancer Serious Chronic Illness: Mixed Methods Review and Meta-Analysis ================================================================================================================================================================= * Linda C. Chyr * Lyndsay DeGroot * Julie M. Waldfogel * Susan M. Hannum * Danetta H. Sloan * Valerie T. Cotter * Allen Zhang * JaAlah-Ai Heughan * Renee F. Wilson * Karen A. Robinson * Sydney M. Dy ## Abstract **PURPOSE** To perform a mixed methods review to evaluate the effectiveness and implementation of models for integrating palliative care into ambulatory care for US adults with noncancer serious chronic illness. **METHODS** We searched 3 electronic databases from January 2000 to May 2020 and included qualitative, mixed methods studies and randomized and nonrandomized controlled trials. For each study, 2 reviewers abstracted data and independently assessed for quality. We conducted meta-analyses as appropriate and graded strength of evidence (SOE) for quantitative outcomes. **RESULTS** Quantitative analysis included 14 studies of 2,934 patients. Compared to usual care, models evaluated were not more effective for improving patient health-related quality of life (HRQOL) (standardized mean difference [SMD] of 4 of 8 studies, 0.19; 95% CI, ‒0.03 to 0.41) (SOE: moderate) or for patient depressive symptom scores (SMD of 3 of 9 studies, ‒0.09; 95% CI, ‒0.35 to 0.16) (SOE: moderate). Models might have little to no effect on patient satisfaction (SOE: low) but were more effective for increasing advance directive (AD) documentation (relative risk, 1.62; 95% CI, 1.35 to 1.94) (SOE: moderate). Qualitative analysis included 5 studies of 146 patients. Patient preferences for appropriate timing of palliative care varied; costs, additional visits, and travel were considered barriers to implementation. **CONCLUSION** Models might have little to no effect on decreasing overall symptom burden and were not more effective than usual care for improving HRQOL or depressive symptom scores but were more effective for increasing AD documentation. Additional research should focus on identifying and addressing characteristics and implementation factors critical to integrating models to improve ambulatory, patient-centered outcomes. Key words: * palliative care * ambulatory care * mixed methods review * meta-analysis ## INTRODUCTION In the United States, most care for adults with noncancer serious chronic illness occurs in ambulatory settings, particularly in clinicians’ offices. Populations with serious chronic illness and conditions include, but are not limited to, those with advanced heart failure, advanced chronic obstructive pulmonary disease, end-stage renal disease, frailty, or multiple serious chronic conditions.1 Care for these patients can be complex because they often experience high symptom burden and decreased health-related quality of life (HRQOL).2 Patients might benefit from increased integration of palliative care into ambulatory care either by incorporation of palliative care services or by training ambulatory care clinicians in palliative care. Care models integrating palliative care approaches with ambulatory care might further address complex care needs for serious illness.3 The key decisional dilemma facing health systems, clinicians, and patients is how individuals with serious chronic illness or conditions can best receive ambulatory care that integrates appropriate palliative care approaches for improving health outcomes. Given significant investments and competing needs for health systems and palliative care clinicians, along with costs and burdens for patients, evaluation of patient-centered outcomes and burden is important. Various types of interventions can be implemented, separately or together, to better integrate palliative care into ambulatory care for this population. Cancer has also been a key area for integrating ambulatory palliative care, already addressed by a recent systematic review.4 We conducted a mixed methods review to evaluate the effectiveness and implementation of models for integrating palliative care into ambulatory care for patients with noncancer serious chronic illnesses. ## METHODS We conducted this review as part of a larger review to evaluate the availability, effectiveness, and implementation of interventions for integrating palliative care into ambulatory care for US-based adults with serious, life-threatening, noncancer, chronic illness or conditions and their caregivers.5 We followed the Agency for Healthcare Research and Quality (AHRQ) Methods Guide for Effectiveness and Comparative Effectiveness Reviews6 (Supplemental Appendix 1). Integrative review methods are based on 2017 Cochrane guidance7 and Joanna Briggs Institute (JBI) methods for mixed methods systematic reviews.8 ### Study Selection We searched PubMed, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), and the Cochrane Central Register of Controlled Trials for articles published from January 2000 to May 2020 and applied eligibility criteria based on the Population, Intervention, Comparisons, Outcomes, Type of Study, Setting and Timing (PICOTS) framework (Supplemental Tables 1a and 1b). We included randomized controlled trials (RCTs) and nonrandomized studies (controlled trials [CTs] or prospective cohort studies) to evaluate the effectiveness of models or interventions. We explored patients’ perspectives of implementation of models using qualitative, mixed methods, and process evaluation studies. Key inclusion criteria for this review included (1) adults aged ≥18 years with serious life-threatening chronic illness, (2) models for integrating palliative care in ambulatory settings, (3) reported outcomes of interest, (4) published in English, and (5) US-based study. Details on search strategy and eligibility are in Supplemental Appendix 1 and Table 1c. ### Data Extraction and Risk of Bias Assessment Paired investigators abstracted data sequentially and recorded study characteristics, demographic variables, interventions, outcome measures and method of ascertainment, outcome results, and key qualitative themes. For quantitative studies, 2 reviewers independently assessed risk of bias using the Cochrane Risk of Bias Tool, version 2 for RCTs. For qualitative studies, 2 reviewers independently assessed study quality using the JBI Critical Appraisal Checklist.9,10 Discrepancies in reviewer assessments were resolved via discussion and revised on the basis of consensus. Two researchers independently reviewed each study to determine model type. ### Data Synthesis and Analysis We conducted meta-analyses for outcomes with at least 3 studies that were sufficiently homogeneous with respect to key variables. Statistical heterogeneity among studies was evaluated using an *I*2 statistic and anticipated statistical heterogeneity. For continuous outcomes, a meta-analyses standardized mean difference (SMD) was calculated using a random-effects model with the DerSimonian and Laird formula. Randomized controlled trials and nonrandomized studies were analyzed separately. For all analyses, statistical significance was set at a 2-sided alpha of .05. We determined clinically meaningful differences for outcome measures wherever possible (Supplemental Table 1d). All quantitative analyses were conducted using Stata version 14 (StataCorp LLC). Two investigators combined and synthesized findings from each qualitative study for the development of key themes related to the implementation of models from patients’ perspectives. ### Grading the Strength of the Body of Evidence We graded the strength of evidence (SOE) for quantitative studies using the grading scheme of the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews6 for patient-reported outcomes including HRQOL, symptom burden, depressive symptom scores, satisfaction, and advance directive (AD) documentation (see Supplemental Table 1e for definitions of grades). ## RESULTS We identified 5,065 unique records, of which 4,681 did not meet the inclusion criteria (Figure 1). Among the 384 full-text articles selected, 345 did not meet the inclusion criteria. In the full review, 39 articles were included. In the present review on implementation and evaluation of models for integrating palliative care, 22 articles were included (17 quantitative and 5 qualitative). ![Figure 1.](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/20/1/77/F1.medium.gif) [Figure 1.](http://www.annfammed.org/content/20/1/77/F1) Figure 1. PRISMA flow diagram. ### Effectiveness of Models #### Description of Included Quantitative Studies We identified 17 articles describing 14 quantitative studies assessing the effectiveness of models integrating palliative care in ambulatory settings11-27 including a total of 2,934 patients. Nine studies were RCTs, and 5 were CTs or prospective cohort studies. Two studies evaluated multimodal interventions not reported in this review. The remaining 12 studies assessed models for integrating palliative care; 4 were shared-care models, 4 involved care coordinators or social workers in care delivery, and 4 used a consultative-care model. A shared-care model was defined as care delivery in which there is joint participation of nonpalliative clinicians and palliative care clinicians working together in relation to an individual’s care and could include systematic cooperation in which different systems work together with various levels and disciplines of clinicians.28 A consultative-care model was defined as care delivery in which a clinician serves in a consultant role with provision of palliative advice and does not necessarily assume primary responsibility of care.3 Follow-up ranged from 2 weeks to 2 years, and 7 studies were multicenter (see Supplemental Appendix 2, Supplemental Table 2a for full study characteristics). #### Patient-Reported Outcomes We reported patient-reported outcomes graded for SOE that included satisfaction, HRQOL, overall symptom burden, and depressive symptom scores (see Supplemental Table 2b for full summary of findings). #### Satisfaction Two RCTs and 1 CT assessed the effect of models on patient satisfaction (n = 306).17,21,22,24 In the first RCT, patient satisfaction was assessed using an investigator-constructed, 5-point, Likert-type scale. Patients in the intervention group reported greater post-test satisfaction mean scores compared to those in the control group (control 3.98 [SD 0.67], intervention 4.07 [SD 0.68]; *P* = .03).24 It is not clear if this is a meaningful difference. In the second RCT, patient satisfaction was assessed with an unnamed scale.17 There were no reported differences in satisfaction between groups at 6 months. Assessing satisfaction using the Group Health Association of America Consumer Satisfaction Survey (range 20-100 points), the CT reported that the control group had a total satisfaction score of 72.4 compared to 70.1 in the intervention arm (*P* = .26).21,22 Therefore, models for integrating palliative care might have little to no effect on improving satisfaction compared to usual care (SOE: low). #### Health-Related Quality of Life Six RCTs and 2 CTs assessed the effect of models on HRQOL (n ≥ 987).14,16-19,21-23,25 In a meta-analysis of 4 RCTs,14,16,17,25 we found no difference in HRQOL for models compared to usual care (SMD, 0.19; 95% CI, ‒0.03 to 0.41; *I*2 = 0%) (Figure 2). The 2 additional RCTs did not report either follow-up data or variability measurements; therefore, we were unable to calculate a mean between-group difference. Consistent with our meta-analysis, neither of these studies reported clinically meaningful differences in HRQOL between groups. The results from the 2 CTs were also consistent with our meta-analysis. The results were consistently not statistically or clinically meaningful; models for integrating palliative care did not improve HRQOL (SOE: moderate). ![Figure 2.](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/20/1/77/F2.medium.gif) [Figure 2.](http://www.annfammed.org/content/20/1/77/F2) Figure 2. Meta-analysis of the effects of models for integrating palliative care on improving health-related quality of life in patients with noncancer serious chronic illness compared with usual care. #### Overall Symptom Burden Two RCTs assessed the effect of models on overall symptom burden (n=419).14,18 One study used the General Symptom Distress Scale (range 0-10 points).18 At 6 months, the reported mean between-group difference was 0.1 (95% CI, ‒0.5 to 0.7; *P* = .8). The second RCT used the Edmonton Symptom Assessment Scale–Parkinson Disease overall score (range 0-140 points). At 12 months, the reported mean between-group difference was –8.27 (95% CI, ‒13.9 to –2.6; *P* = .004). Given inconsistent results that are likely not clinically meaningful, models for integrating palliative care might have little to no effect compared to usual care for overall symptom burden (SOE: low). #### Depressive Symptom Score Six RCTs, 1 CT, and 2 prospective cohort studies assessed the effect of models on depression (n ≥ 729).14-22,25 In a meta-analysis of 3 RCTs,14,17,25 we found no difference in symptoms of depression for models for integrating palliative care compared to usual care (SMD, ‒0.09; 95% CI, ‒0.35 to 0.16; *I*2 = 0%) (Figure 3). Three RCTs, 1 CT, and 2 prospective cohort studies could not be included in the meta-analysis, owing to missing baseline and/or variability data. The 2 prospective cohort studies and CT reported results consistent with our meta-analysis.14,20-22 Two of the RCTs not included in the meta-analysis reported results that were significantly different, although neither was clinically meaningful.23,25 The other RCT assessed depression using the Hospital Anxiety and Depression Scale (range 0-21 points) and reported a difference of –1.94 (95% CI, –3.57 to –0.31; *P* = .02) between groups that was clinically meaningful.19 Models for integrating palliative care were therefore not more effective than usual care for depressive symptom scores (SOE: moderate). ![Figure 3.](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/20/1/77/F3.medium.gif) [Figure 3.](http://www.annfammed.org/content/20/1/77/F3) Figure 3. Meta-analysis of the effects of models for integrating palliative care on improving depressive symptom scores in patients with noncancer serious chronic illness compared with usual care. #### Advance Directive Documentation Four RCTs, 2 CTs, and 1 prospective cohort study assessed the effect of models on AD documentation (n = 966).14,17,20-25 We conducted a meta-analysis with all 4 RCTs using percent completion of AD documentation at 6 months.14,17,24,25 Patients in models had a 62% greater chance of having AD documentation compared to patients in standard care (relative risk, 1.62; 95% CI, 1.35 to 1.94; *I*2 = 0%) (Figure 4). The other 3 studies included 2 CTs reported in 3 articles and 1 prospective cohort study.20-23 The 2 CTs were consistent with the meta-analysis in reporting greater AD completion among patients in the intervention group compared to those in the usual care group, although the results were not statistically significant. Therefore, models for integrating palliative care were more effective than usual care for increasing AD documentation (SOE: moderate). ![Figure 4.](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/20/1/77/F4.medium.gif) [Figure 4.](http://www.annfammed.org/content/20/1/77/F4) Figure 4. Meta-analysis of the effects of models for integrating palliative care on increasing advance directive documentation in patients with noncancer serious chronic illness compared with usual care. ### Implementation of Models #### Description of Included Qualitative Studies We identified 5 qualitative studies exploring patients’ perspectives of important components contributing to effective implementation of models for integrating palliative care. Of these, 3 were intervention studies29-31; the remaining 2 studies did not evaluate interventions.32,33 The 5 studies conducted individual qualitative interviews with some variation of thematic analysis with a total of 146 patients (see Supplemental Table 2c for study characteristics). Two were rated as high quality,29,33 and 3 were rated as low quality.30-32 #### Components Contributing to Effective Implementation One study identified several simple and low-resource components of an intervention that were perceived by patients to be useful, which might be offered by nonmedical personnel: (1) obtaining social services that patients already qualify for under existing funding mechanisms, (2) facilitating better communication about difficult issues, and (3) offering patients the simple gift of listening with compassion.31 In another study, patients indicated that they want to feel heard and be “seen” which meant receiving care by a clinician who understands the context of their illness and its trajectory and who is able to offer solutions to individual concerns.32 The interventions delivered by registered nurses, advanced practice nurses, or social workers were feasible, and patients perceived them to be beneficial.29,30,33 Patients were satisfied with the telephone structure of the interventions, perceiving such structure as generally helpful. Among interventions led by registered nurses or advanced practice nurses, patients saw nurses as advocates.29,30 The studies also identified components of models that patients perceived as hindering implementation including burdensome, generic, or repetitive surveys.29 Patients identified several barriers to participation including lack of clinician time, difficulty scheduling sessions within busy schedules, and driving to clinic visits. Patients recommended that palliative care and regular clinic visits correspond to ease the burden of travel.30 In addition, patients’ concerns about taking opioids and the cost of paying for pulmonary rehabilitation and palliative care clinic visits influenced their decision to continue palliative care after the close of the study.30 Patients’ perceptions of appropriate timing to implement shared models or interventions varied. In 1 study, patients stated a preference for the intervention to be provided after diagnosis,29 yet another identified a preference for the beginning of the end of life.31 ## DISCUSSION In this mixed methods review and meta-analysis, the models evaluated were not more effective than usual care for improving HRQOL, had little to no effect on reducing overall symptom burden, and were not effective for improving depressive symptom scores. The models had little to no effect on increasing patient satisfaction but did increase AD documentation. This review highlights the variability in types of models used and components of interventions across palliative care studies. Interventions were often complex and included a variety of team members. Qualitative syntheses provided insight into important components for implementation of models from the patient perspective. Our mixed methods approach adds to recent systematic reviews of different noncancer patient populations and ambulatory care settings, which found similar results of small intervention effect for few patient-reported outcomes.34-36 For example, 1 systematic review evaluating the effectiveness of hospital-based, palliative care among adults with advanced illnesses including cancer found low- to very-low-quality evidence on the benefit of palliative care on patient HRQOL, symptom burden, and satisfaction.35 Another systematic review and meta-analysis examining the association of the receipt of palliative care among adults with chronic noncancer illness across settings found slightly lower acute health care use and modest improvement of symptom burden.36 Our findings highlight the need for innovative approaches to address the needs of at-risk populations that might benefit from palliative care approaches throughout serious chronic illness. Other US reviews of models for integrating palliative care with cancer populations and nonambulatory settings found that integrating palliative care specialists with primary and ambulatory care was key to model success.3 Although the present review was not able to identify factors associated with effective implementation, our qualitative findings highlight the importance of evaluating patient-perceived barriers and facilitators when integrating palliative care into existing ambulatory care. Patients valued care from clinicians, particularly nurses, who understood their unique needs and individual illness trajectory and often found separate palliative care appointments burdensome. Thus, integrating palliative care approaches into existing appointments with clinicians, nurses, care coordinators, or social workers who understand patients’ individual concerns and illness trajectory might promote uptake of palliative care. Patients with serious illness often have intense, varying, and complex care needs managed across settings and in the community. As noted in the National Academy of Medicine report *Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life*,37 improving care for those with serious, life-threatening illnesses and conditions requires comprehensive, individualized, and time-consuming care provided by needs-oriented systems and coordinated with community services.37,38 Such care can be facilitated by increased clinician-patient communication about palliative care, which the *Dying in America* report suggests might be encouraged by better systems support and require involvement from nonpalliative care clinicians. For policy, the report recommends that the health care system and payment structures need to better support palliative care domains such as symptom management and care coordination. In addition, little evidence exists on effective interventions for managing common symptoms in palliative care, and ongoing research and drug development are sparse; advances in symptom management science in these populations are needed.39 Our findings reinforce that models for integrating palliative care might have little to no effect on decreasing patient-reported outcomes but might be more effective in targeting care planning such as AD documentation. ### Limitations Our analyses do not provide insight on effectiveness by types of models, given the limited number of studies to conduct subgroup analyses. Comparisons across studies were challenging because studies lacked standard information on details of interventions and implementation. Outcomes were measured using heterogeneous assessment tools that might not be validated for palliative care populations. Owing to variations and missing outcomes, we could conduct only a few meta-analyses. Most of the quantitative studies were at high risk of bias, and qualitative studies often lacked rigorous reporting or methods. Long-term sustainability issues were not evaluated, which is particularly important because all interventions were supported by external funding. We used accepted standards for clinically meaningful differences, but many were not from palliative care populations, and some might consider these smaller differences meaningful. ### Conclusions The models evaluating integration of palliative care in ambulatory care for people with serious chronic illness or conditions other than cancer were not effective for improving HRQOL, had little to no effect on decreasing overall symptom burden, and were not effective for improving depressive symptom scores but were effective for increasing AD documentation. Patient preferences for appropriate timing of palliative care varied; costs, additional visits, and travel were considered barriers to implementing models. More research is warranted to advance the science of palliative care by identifying and addressing characteristic and implementation factors critical to integrating these models in ambulatory care to better improve patient-centered outcomes and incorporate patients’ perspectives on care delivery. ## Footnotes * * These authors contributed equally as co-first authors. * Conflicts of interest: authors report none. * **Read or post commentaries in response to this article**. * **Funding support:** This project was funded under Contract No. HHSA29020150 00061 from the Agency for Healthcare Research and Quality, US Department of Health and Human Services. * **Disclaimer:** The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the US Department of Health and Human Services. The Agency for Healthcare Research and Quality retains a license to display, reproduce, and distribute the data and the report from which this manuscript was derived under the terms of the agency’s contract with the author. * **[Supplemental materials](https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2754/-/DC1)** * Received for publication December 7, 2020. * Revision received April 29, 2021. * Accepted for publication June 3, 2021. * © 2022 Annals of Family Medicine, Inc. ## References 1. 1.Bernacki RE, Block SD; American College of Physicians High Value Care Task Force. Communication about serious illness care goals: a review and synthesis of best practices. JAMA Intern Med. 2014;174(12):1994-2003. doi:10.1001/jamainternmed.2014.5271 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jamainternmed.2014.5271&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=25330167&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 2. 2.Ahluwalia SC, Chen C, Raaen L, et al. A systematic review in support of the National Consensus Project Clinical Practice Guidelines for Quality Palliative Care, fourth edition. J Pain Symptom Manage. 2018;56(6):831-870. doi:10.1016/j.jpainsymman.2018.09.008 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.jpainsymman.2018.09.008&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 3. 3.Luckett T, Phillips J, Agar M, Virdun C, Green A, Davidson PM. Elements of effective palliative care models: a rapid review. BMC Health Serv Res. 2014; 14:136. doi:10.1186/1472-6963-14-136 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1186/1472-6963-14-136&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24670065&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 4. 4.Fulton JJ, LeBlanc TW, Cutson TM, et al. Integrated outpatient palliative care for patients with advanced cancer: a systematic review and meta-analysis. Palliat Med. 2019;33(2):123-134. doi:10.1177/0269216318812633 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1177/0269216318812633&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 5. 5.Dy SM, Waldfogel JM, Sloan DH, et al. Integrating palliative care in ambulatory care of noncancer serious chronic illness. Comparative Effectiveness Review No. 237. Prepared by the Johns Hopkins University Evidence-Based Practice Center under Contract No. 290-2015-00006-I. Agency for Healthcare Research and Quality; 2021. Accessed Oct 20, 2021. doi:10.23970/AHRQEPCCER237 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.23970/AHRQEPCCER237&link_type=DOI) 6. 6.Agency for Healthcare Research and Quality Methods for Effective Health Care. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Agency for Healthcare Research and Quality; 2008. Accessed Dec 20, 2021. [https://ncbi.nlm.nih.gov/books/NBK47095](https://ncbi.nlm.nih.gov/books/NBK47095) 7. 7.Harden A, Thomas J, Cargo M, et al. Cochrane Qualitative and Implementation Methods Group guidance series-paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018;97:70-78. doi:10.1016/j.jclinepi.2017.11.029 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.jclinepi.2017.11.029&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 8. 8.1. Aromataris E, 2. Munn Z Lizarondo L, Stern C, Carrier J, et al. Mixed methods systematic reviews. In: Aromataris E, Munn Z, eds. JBI Manual for Evidence Synthesis. Joanna Briggs Institute; 2020. 9. 9.Majid U, Vanstone M. Appraising qualitative research for evidence syntheses: a compendium of quality appraisal tools. Qual Health Res. 2018;28(13):2115-2131. doi:10.1177/1049732318785358 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1177/1049732318785358&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 10. 10.Lockwood C, Munn Z, Porritt K. Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int J Evid Based Healthc. 2015;13(3):179-187. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1097/XEB.0000000000000062&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 11. 11.Lakin JR, Robinson MG, Obermeyer Z, et al. Prioritizing primary care patients for a communication intervention using the “surprise question”: a prospective cohort study. J Gen Intern Med. 2019;34(8):1467-1474. doi:10.1007/s11606-019-05094-4 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-019-05094-4&link_type=DOI) 12. 12.Lakin JR, Koritsanszky LA, Cunningham R, et al. A systematic intervention to improve serious illness communication in primary care. Health Aff (Millwood). 2017;36(7):1258-1264. doi:10.1377/hlthaff.2017.0219 [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjk6IjM2LzcvMTI1OCI7czo0OiJhdG9tIjtzOjIyOiIvYW5uYWxzZm0vMjAvMS83Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 13. 13.Goldstein NE, Mather H, McKendrick K, et al. Improving communication in heart failure patient care. J Am Coll Cardiol. 2019;74(13):1682-1692. doi:10.1016/j.jacc.2019.07.058 [FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo0OiJhY2NqIjtzOjU6InJlc2lkIjtzOjEwOiI3NC8xMy8xNjgyIjtzOjQ6ImF0b20iO3M6MjI6Ii9hbm5hbHNmbS8yMC8xLzc3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 14. 14.Kluger BM, Miyasaki J, Katz M, et al. Comparison of integrated outpatient palliative care with standard care in patients with Parkinson disease and related disorders: a randomized clinical trial. JAMA Neurol. 2020;77(5):551-560. doi:10.1001/jamaneurol.2019.4992 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jamaneurol.2019.4992&link_type=DOI) 15. 15.Owens D, Eby K, Burson S, Green M, McGoodwin W, Isaac M. Primary palliative care clinic pilot project demonstrates benefits of a nurse practitioner-directed clinic providing primary and palliative care. J Am Acad Nurse Pract. 2012;24(1):52-58. doi:10.1111/j.1745-7599.2011.00664.x [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1745-7599.2011.00664.x&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=22243681&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 16. 16.Rogers JG, Patel CB, Mentz RJ, et al. Palliative care in heart failure: the PAL-HF randomized, controlled clinical trial. J Am Coll Cardiol. 2017;70(3):331-341. doi:10.1016/j.jacc.2017.05.030 [FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo0OiJhY2NqIjtzOjU6InJlc2lkIjtzOjg6IjcwLzMvMzMxIjtzOjQ6ImF0b20iO3M6MjI6Ii9hbm5hbHNmbS8yMC8xLzc3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 17. 17.O’Riordan DL, Rathfon MA, Joseph DM, et al. Feasibility of implementing a palliative care intervention for people with heart failure: learnings from a pilot randomized clinical trial. J Palliat Med. 2019;22(12):1583-1588. doi:10.1089/jpm.2018.0633 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1089/jpm.2018.0633&link_type=DOI) 18. 18.Bekelman DB, Allen LA, McBryde CF, et al. Effect of a collaborative care intervention vs usual care on health status of patients with chronic heart failure: the CASA randomized clinical trial. JAMA Intern Med. 2018;178(4):511-519. doi:10.1001/jamainternmed.2017.8667 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jamainternmed.2017.8667&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=29482218&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 19. 19.Bekelman DB, Plomondon ME, Carey EP, et al. Primary results of the Patient-Centered Disease Management (PCDM) for heart failure study: a randomized clinical trial. JAMA Intern Med. 2015;175(5):725-732. doi:10.1001/jamainternmed.2015.0315 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jamainternmed.2015.0315&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=25822284&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 20. 20.Feely MA, Swetz KM, Zavaleta K, Thorsteinsdottir B, Albright RC, Williams AW. Reengineering dialysis: the role of palliative medicine. J Palliat Med. 2016;19(6):652-655. doi:10.1089/jpm.2015.0181 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1089/jpm.2015.0181&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=26991732&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 21. 21.Rabow MW, Dibble SL, Pantilat SZ, McPhee SJ. The comprehensive care team: a controlled trial of outpatient palliative medicine consultation. Arch Intern Med. 2004;164(1):83-91. doi:10.1001/archinte.164.1.83 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/archinte.164.1.83&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=14718327&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000188045200011&link_type=ISI) 22. 22.Rabow MW, Petersen J, Schanche K, Dibble SL, McPhee SJ. The comprehensive care team: a description of a controlled trial of care at the beginning of the end of life. J Palliat Med. 2003;6(3):489-499. doi:10.1089/109662103322144862 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1089/109662103322144862&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=14509498&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 23. 23.Engelhardt JB, Rizzo VM, Della Penna RD, et al. Effectiveness of care coordination and health counseling in advancing illness. Am J Manag Care. 2009; 15(11):817-825. [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=19895186&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000271936800006&link_type=ISI) 24. 24.Engelhardt JB, McClive-Reed KP, Toseland RW, Smith TL, Larson DG, Tobin DR. Effects of a program for coordinated care of advanced illness on patients, surrogates, and healthcare costs: a randomized trial. Am J Manag Care. 2006; 12(2):93-100. [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16464138&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000235263700003&link_type=ISI) 25. 25.O’Donnell AE, Schaefer KG, Stevenson LW, et al. Social worker-aided palliative care intervention in high-risk patients with heart failure (SWAP-HF): a pilot randomized clinical trial. JAMA Cardiol. 2018;3(6):516-519. doi:10.1001/jamacardio.2018.0589 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jamacardio.2018.0589&link_type=DOI) 26. 26.Dionne-Odom JN, Ejem DB, Wells R, et al. Effects of a telehealth early palliative care intervention for family caregivers of Pprsons with advanced heart failure: the ENABLE CHF-PC randomized clinical trial. JAMA Netw Open. 2020;3(4):e202583. doi:10.1001/jamanetworkopen.2020.2583 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2020.2583&link_type=DOI) 27. 27.Lakin JR, Neal BJ, Maloney FL, et al. A systematic intervention to improve serious illness communication in primary care: effect on expenses at the end of life. Healthc (Amst). 2020;8(2):100431. doi:10.1016/j.hjdsi.2020.100431 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.hjdsi.2020.100431&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=32553522&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 28. 28.Paquette-Warren J, Vingilis E, Greenslade J, Newnam S. What do practitioners think? A qualitative study of a shared care mental health and nutrition primary care program. Int J Integr Care. 2006;6:e18. doi:10.5334/ijic.164 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.5334/ijic.164&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17041680&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 29. 29.Bekelman DB, Hooker S, Nowels CT, et al. Feasibility and acceptability of a collaborative care intervention to improve symptoms and quality of life in chronic heart failure: mixed methods pilot trial. J Palliat Med. 2014;17(2):145-151. doi:10.1089/jpm.2013.0143 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1089/jpm.2013.0143&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24329424&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 30. 30.Long MB, Bekelman DB, Make B. Improving quality of life in chronic obstructive pulmonary disease by integrating palliative approaches to dyspnea, anxiety, and depression. J Hosp Palliat Nurs. 2014;16(8):514-520. doi:10.1097/NJH.0000000000000111 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1097/NJH.0000000000000111&link_type=DOI) 31. 31.Rabow MW, Schanche K, Petersen J, Dibble SL, McPhee SJ. Patient perceptions of an outpatient palliative care intervention: “It had been on my mind before, but I did not know how to start talking about death…”. J Pain Symptom Manage. 2003;26(5):1010-1015. doi:10.1016/j.jpainsymman.2003.03.002 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.jpainsymman.2003.03.002&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=14585552&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000186426400004&link_type=ISI) 32. 32.Hobler MR, Engelberg RA, Curtis JR, et al. Exploring opportunities for primary outpatient palliative care for adults with cystic fibrosis: a mixed-methods study of patients’ needs. J Palliat Med. 2018;21(4):513-521. doi:10.1089/jpm.2017.0259 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1089/jpm.2017.0259&link_type=DOI) 33. 33.Bekelman DB, Nowels CT, Retrum JH, et al. Giving voice to patients’ and family caregivers’ needs in chronic heart failure: implications for palliative care programs. J Palliat Med. 2011;14(12):1317-1324. doi:10.1089/jpm.2011.0179 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1089/jpm.2011.0179&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=22107107&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 34. 34.Kavalieratos D, Corbelli J, Zhang D, et al. Association between palliative care and patient and caregiver outcomes: a systematic review and meta-analysis. JAMA. 2016;316(20):2104-2114. doi:10.1001/jama.2016.16840 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.2016.16840&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=27893131&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 35. 35.Bajwah S, Oluyase AO, Yi D, et al. The effectiveness and cost-effectiveness of hospital-based specialist palliative care for adults with advanced illness and their caregivers. Cochrane Database Syst Rev. 2020;9(9):CD012780. doi:10.1002/14651858.CD012780.pub2 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1002/14651858.CD012780.pub2&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 36. 36.Quinn KL, Shurrab M, Gitau K, et al. Association of receipt of palliative care interventions with health care use, quality of life, and symptom burden among adults with chronic noncancer illness: a systematic review and meta-analysis. JAMA. 2020;324(14):1439-1450. doi:10.1001/jama.2020.14205 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.2020.14205&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F20%2F1%2F77.atom) 37. 37.Dying in America: improving quality and honoring individual preferences near the end of life. Mil Med. 2015;180(4):365-357. doi:10.7205/MILMED-D-15-00005 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.7205/MILMED-D-15-00005&link_type=DOI) 38. 38.Cohn J, Corrigan J, Lynn J, et al. Community-based models of care delivery for people with serious illness. National Academy of Medicine. Published Apr 13, 2017. Accessed Oct 20, 2021. [https://nam.edu/community-based-models-of-care-delivery-for-people-with-serious-illness](https://nam.edu/community-based-models-of-care-delivery-for-people-with-serious-illness) 39. 39.Currow DC, Abernethy AP, Fallon M, Portenoy RK. Repurposing medications for hospice/palliative care symptom control is no longer sufficient: a manifesto for change. J Pain Symptom Manage. 2017;53(3):533-539. doi:10.1016/j.jpainsymman.2016.10.358 [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.jpainsymman.2016.10.358&link_type=DOI)