Effects of Facilitated Team Meetings and Learning Collaboratives on Colorectal Cancer Screening Rates in Primary Care Practices: A Cluster Randomized Trial

INTRODUCTION

Quality improvement (QI) approaches vary in the extent to which specific objectives, tools, resources, and change processes are provided and orchestrated by health systems or researchers. On one end of the spectrum, these features are externally imposed on participating organizations/subjects, such as providing physicians with flow sheets,^1,2 checklists,^3,4 or computer-based reminders,^5–8 or distributing patient educational materials.^6,9,10 Although such approaches can provide straightforward change mechanisms that ensure generalizability and treatment fidelity, they can pose problems when contextual variables contradict intervention fidelity¹¹ or when motivation to sustain changes wanes once the researchers leave.¹²

On the other end of the spectrum are approaches where organizations/subjects engage in their own problem identification, and the processes for change emerge internally. These approaches move beyond filling a knowledge deficit on the part of patients or clinicians to enhancing the organization’s capacity and resources for change.^13–17 Research on stakeholders—those individuals and groups who have an interest in and are influenced by the organization¹⁸—suggests that when stakeholders identify problems and generate their own solutions, they are more likely to engage in and sustain change processes.¹⁹ Without the engagement, motivation, and commitment of key stakeholders within an organization, even meritorious innovations may be abandoned before they have had the chance to be effective.²⁰

We report the results of the Supporting Colorectal Cancer Outcomes through Participatory Enhancements (SCOPE) study, which combined features on both ends of this spectrum. The study imposed on participating primary care practices a specific goal—to improve colorectal cancer screening (CRC) rates—and a change process—a series of facilitated team meetings and learning collaboratives. The use of practice facilitators to guide QI efforts²¹ and learning collaboratives to stimulate cross-practice learning^22–26 has received growing attention as robust methods for translating evidence-based guidelines into practice. Within these parameters, the study tailored the change process, allowing practice members to generate their own QI objectives and strategies in hopes of enhancing practices’ capacity for change.

METHODS

SCOPE was a cluster randomized trial designed to evaluate the effectiveness of a tailored intervention on CRC screening rates in primary care practices. The study design incorporated a mixed-methods evaluation to assess practice-level variation in intervention fidelity and experiences.²⁷ CRC screening was selected because of its documented benefits for reducing morbidity and mortality, and its proven cost-effectiveness.^28–31

The unit of randomization and intervention was the practice, whereas the unit of observation of outcomes was patients within each practice. SQUIRE^32,33 and CONSORT³⁴ guidelines served as a framework for implementing the intervention and reporting findings. The study was approved by the University of Medicine and Dentistry of New Jersey Institutional Review Board, and informed consent was obtained from participating practice members and patients.

RESULTS

Twenty-five practices consented to participate and were randomized to either the intervention arm (n = 12) or control arm (n = 13) (Supplemental Figure 1, available at http://annfammed.org/content/11/3/220/suppl/DC1). Early on, 2 practices closed (1 intervention, 1 control). To ensure an adequate intervention group sample, 1 control practice was randomly selected to be in the intervention group, thus providing a final sample size of 23 practices (12 intervention, 11 control). Of the 23 practices, the average number of physicians was 4 (min/max = 1 to 11). All were family or internal medicine practices, and only 1 was a residency practice (P16); 83% of practices were located in suburban settings. The average length of practice existence was 11.7 years.

Of the 12 intervention practices, 7 fully engaged in the intervention, 2 practices (P17 and P21) failed to participate in the intervention, and 3 others never fully engaged in developing collaborative processes as intended by the study (P7, P11, and P15) (Supplemental Table 1, available at http://annfammed.org/content/11/3/220/suppl/DC1).

At baseline, 80% (N = 791) of eligible patients consented to participate in the study; 67% (n = 723) of eligible patients participated at the 12-month follow-up (Supplemental Figure 2, available at http://annfammed.org/content/11/3/220/suppl/DC1). On average, 37% of patients had Medicare or Medicaid insurance. A total of 1,315 charts were audited for this study. Patient characteristics are presented in Table 1.

DISCUSSION

Project SCOPE tested an intervention model that used a facilitated, team-based approach to improve CRC screening rates in primary care settings. Facilitators tailored the change efforts according to the particular culture and perceived needs of each practice. Although CRC screening rates were emphasized as the focus of the intervention, specific QI objectives and plans rested with the practice members. A central assumption was that getting multiple stakeholder buy-in through this approach would enhance motivation and commitment to the change process. An explicit goal of the study was to develop a practice change model (using CRC screening as an initial focus) that could then be replicated for ongoing change efforts.

Most SCOPE practices were successful in several QI implementation characteristics, including team structure, team member engagement, and intrateam communication. Except for 2 practices that opted not to participate in the intervention, all others formed a RAP team, sent representatives to the learning collaboratives, and worked on 1 or more QI plans, suggesting that this type of intervention model is viable in primary care settings of varying size and structure. Variation between high- and low-performing practices, however, was evident in how well leaders fostered team development and the extent to which team members felt psychologically safe to take risks during the change process. Most teams were not adept at communicating their QI plans with the rest of the practice regardless of size. Moreover, only a few practices adapted the RAP model for use as an ongoing method to identify and work on continuous QI efforts. Organizational disruptions likely affected several practices’ progression of their change capacity. Previous analyses have explored, in depth, additional aspects of QI implementation from the SCOPE trial.^25,49

Despite certain successes regarding practices’ QI implementation, overall SCOPE did not yield statistically significant improvements in CRC screening rates. Importantly, the integration of qualitative methods into the study design allowed us to answer recent calls to explore the implementation context of null trials.²¹ Several lessons learned from SCOPE are important to consider for future interventions.

One lesson was that allowing RAP teams to choose their own QI objectives and plans meant that some practices chose issues that were not directly related to CRC screening. RAP teams that focused on poor communication or chaos in the practice viewed these issues to be of sufficient priority that they needed to be addressed before the teams could delve into concrete clinical improvements. Facilitators prompted teams to keep CRC screening in the foreground, but their discussions often maintained a broader focus on practice dynamics and operations. Although potentially beneficial for the organization in other ways, this aspect of tailoring likely diminished their CRC screening improvements in the time frame of the study.

Another lesson pertained to the notion of spread. RAP teams typically included 1 clinician, and there was variability in how well these leaders fostered a climate of change for the entire practice. Other clinicians in a practice tended not to be aware of or engaged in the CRC improvement efforts, and RAP teams tended to communicate poorly with the rest of the practice regarding QI plans. Even though facilitators emphasized the importance of practice-wide communication regarding their QI efforts, ultimately this responsibility rested with the teams. As a result, segments of a practice improved their capacity for change and CRC screening efforts, but most practices were unsuccessful in effecting organization-wide improvements.

Additionally, because RAP teams were made up of diverse practice members where differing levels of administrative power were evident, teams needed a sense of psychological safety and trust^50,51 that supported critical reflection of the change process.⁴⁹ Facilitators helped foster a safe environment, but it often hinged on the role of practice leaders. High-performing practices had strong to moderate leadership and psychological safety, whereas low-performing practices were weak in both of these areas. Future interventions must pay attention to the role of practice leaders given their influence on team dynamics and the change process. Interventions using a team-based approach may benefit from incorporating instructional components for practice leaders to enhance their knowledge and skills in leading QI teams.

Lastly, SCOPE employed generalist facilitators who had expertise in organizational change and group process but not in cancer screening. As such, facilitators were not relied upon for giving practices CRC solutions. Instead, practices were encouraged to develop their organizational learning capacity to identify and implement their own solutions. Although the facilitators were well-suited to concentrate on the change process—eg, prompting teams to confront and deal with barriers to change—having facilitators who also had expertise in the target condition likely could have had more direct benefits on practices’ CRC screening efforts.

We recognize several limitations to our study. Power to detect an intervention effect was limited by the small number of participating practices, which was compounded by the failure of 2 practices to participate in the intervention as prescribed. This lack of fidelity would lead to attenuation of the intervention effect and, thus, reduced power. Moreover, the higher than expected CRC screening rates on average affected our ability to detect a significant increase in CRC screening rates. We also would expect that the volunteer practices in our sample were sufficiently motivated to improve CRC screening rates, which may differ from practices in general. Lastly, based on previous work showing that the level of uncertainty associated with a disease is a critical factor for intervention design,⁵² we acknowledge that CRC screening may preclude the need for an intensive, team-based model of practice improvement^53–55 and, therefore, may have affected the change process and, consequently, the intended goal of extrapolating the change model to other diseases or areas of improvement.

Various QI approaches and methods can be effective in achieving targeted outcomes. Yet practice transformation, such as that envisioned by the patient-centered medical home, cannot be realized through only a series of incremental QI projects. Developing greater organizational learning and change capacities is required. The SCOPE intervention sought to bridge the gap between an externally orchestrated, single-target intervention and full-scale, emergent practice transformation. The response of practices to the SCOPE intervention suggests that this QI approach (ie, MAP/RAP, including facilitated team meetings and learning collaboratives) may not be an optimal strategy for single-target interventions, particularly if short-term guideline concordant numerical gains are the goal. The MAP/RAP approach provides considerable flexibility in the improvement focus a practice can take, as well as the strategies to get there. If improving performance measures for a preselected target, such as CRC screening rates, is the focus, perhaps a more traditional targeted continuous QI approach would be more appropriate. Nevertheless, because there are so many potential disease-specific and patient-centered targets in need of improvement in primary care, relying on a series of single-target QI interventions may not be realistic.

Methodologically, the SCOPE study shows that quantitative and qualitative findings should not be seen as a way to merely confirm or disconfirm each other. In some cases, SCOPE results reveal discordance in the 2 types of data, which might tempt us to think one or the other is wrong. Rather, integrating both views into an overarching analysis of the study provides a richer understanding of the intervention. Advancing the knowledge base of QI interventions requires future reports to address how and why QI interventions work rather than simply measuring whether they work.