Engaging Primary Care Practices in Studies of Improvement: Did You Budget Enough for Practice Recruitment?

INTRODUCTION

Small practices comprise nearly one-half of primary care practices in the United States.¹ Although declining in number, they remain a critical part of the health care ecosystem,² particularly in rural areas and in densely populated urban areas. These practices often lack the necessary infrastructure, skills, knowledge, and support required to sustain engagement in quality improvement activities essential to the strived-for improvement in quality of patient care.^3–7

EvidenceNOW: Advancing Heart Health in Primary Care is an Agency for Healthcare and Research (AHRQ)-funded grant initiative dedicated to examining strategies to build quality improvement capacity in small and medium-size primary care practices with a focus on implementing evidence-based cardiovascular disease risk reduction strategies.⁸ The participating practices, located in 7 regional cooperatives, were provided 5 core services: on-site practice facilitation and coaching, health information technology support, shared learning collaboratives, expert consultation, and data feedback and benchmarking.⁹ Further details about this large national initiative are published in a companion article in this journal supplement.¹⁰ Although a few studies have described the costs of technical support and practice facilitation for quality improvement,¹¹ very little is known about the cost of recruiting and enrolling large numbers of small practices as subjects in formal studies of primary care practice redesign. Here we report on the experiences of 2 of the 7 EvidenceNOW studies funded to address this gap by describing and evaluating the various approaches these studies used for practice recruitment and by estimating the cost of recruiting more than 500 practices across 6 states participating in EvidenceNOW.

METHODS

The EvidenceNOW Northwest cooperative, Healthy Hearts Northwest, led by the MacColl Center for Health Care Innovation within the Kaiser Permanente Washington Health Research Institute, recruited practices in Oregon, Washington, and Idaho.¹² The EvidenceNOW Midwest cooperative, Healthy Hearts in the Heartland, based at the Center for Health Information Partnerships, Institute of Public Health & Medicine in Northwestern University’s Feinberg School of Medicine, recruited practices in Illinois, Indiana, and Wisconsin. Both cooperatives focused on small and medium-size (fewer than 10 clinicians per site) primary care practices with existing electronic health record (EHR) systems that lacked the necessary resources for data management and quality improvement. Each cooperative comprised multiple organizations with existing relationships and considerable experience engaging and recruiting small practices, including Centers for Medicare & Medicaid Services (CMS) Quality Improvement Network/Quality Improvement Organizations (QIN/QIOs), health information technology regional extension centers (HITRECs), and a practice-based research network.

RESULTS

A total of 3,669 practices were contacted during the 18-month recruitment period, resulting in 484 enrolled practices across the 6 states served by the 2 cooperatives (Table 2). Across the 2 cooperatives, it took 8.5 months to reach the halfway mark for enrollment—6 months for Healthy Hearts Northwest and 10 months for Healthy Hearts in the Heartland. For each practice enrolled, there were 5 practices recruited in Healthy Hearts Northwest and 10 practices in Healthy Hearts in the Heartland, for an overall average recruitment ratio of 8 practices recruited for each 1 enrolled. The average number of interactions per enrolled practice was 7 (Table 3). Across these 2 regional cooperatives, 29,100 hours were involved in the recruitment of 484 practices at a total cost of $2.675 million or $5,529 per enrolled practice (Table 4).

Practices with a prior relationship were significantly more likely to enroll than those without. A prior relationship was present for 214 (44%) of the enrolled practices (Table 2). The cooperatives had no prior connections with 145 (30%) of the practices that eventually enrolled, while the balance involved referrals or warm handoffs (Table 2). Each recruited practice had 7 contacts on average, with an estimated cost of $730 per practice or $113 per contact (Table 3 and Table 4). If recruitment had been limited to practices with no prior relationship, the cooperatives would have needed to reach out to 8,501 practices with 8 contacts per practice to enroll 484 practices, significantly increasing the cost of recruitment from $2.7 million ($5,529 per practice) to $7.7 million ($15,878 per enrolled practice).

DISCUSSION

Recruitment and engagement of large numbers of smaller primary care practices across large geographic regions to participate in studies of strategies to build quality improvement capacity is time-consuming and expensive. An average of 7 contacts during 14 months of recruitment resulted in an average cost of $5,529 per enrolled practice. The total recruitment costs were remarkably similar between both cooperatives, one in the Pacific Northwest and the other in the Midwest.

Prior relationships are crucial, with stakeholder engagement playing an important role. In previous work, Solberg describes the 7 Rs comprising the key principles for recruiting medical groups for this type of research: relationships, reputation, requirements, reward, reciprocity, resolution, and respect.¹⁶ Enrollment and engagement of the recruited primary care practices were built on trust from established relationships developed by both the Northwest and Midwest cooperatives, but the large numbers of practices to be recruited required both cooperatives to also approach practices lacking such relationships.

The cooperatives initially targeted practices with a prior relationship, then reached out to other organizations and leaders with whom they had a relationship seeking introductions, and finally, extended an invitation to those on a list of eligible practices. Not captured in our data is the role of physician leadership in promoting and disseminating the EvidenceNOW quality improvement study opportunity across multiple organizations. Indeed, both cooperatives experienced higher rates of successful enrollment among practices where a previous relationship existed or when a warm handoff or recommendation was made from one practice to another. Almost 1 in 3 (30%) of these practices enrolled, compared with approximately 1 in 20 (6%) practices with no existing relationship or warm handoff. These handoffs required time and extended the period of recruitment. As one practice facilitator commented, “…recruitment moves at the ‘speed of trust.’”

Recruitment of the required 500 practices in a year’s time would not have been possible in the absence of prior relationships and strong connections to local organizations (ie, state primary care associations, independent physician associations, etc). Although the combined cooperatives required 8.5 months to reach the recruitment halfway point, our experience strongly suggests that the relationship between total cost of recruitment and the total number of practices recruited is not linear. We did not measure cost at a fine enough detail to capture the relation between cost per practice (including marginal cost) and total practices recruited, but it is reasonable to assume that we encountered diseconomies of scale as we began recruiting practices with little or no prior relationship.

Each recruited practice had 7 contacts on average, with an estimated cost of $730 per practice ($113 per contact). If recruitment had been limited to practices with no prior relationship or warm handoffs or referrals, the cooperatives would have needed to reach out to 8,501 practices to enroll 484 practices, increasing the cost of recruitment from $2.7 million ($5,529 per-practice) to $7.7 million ($15,878 per enrolled practice). Such organizations as practice-based research networks, QIOs, and regional extension centers have little ongoing infrastructure support to maintain these relationships; instead, they rely on an ongoing series of project-specific contracts and grants to support staff and operations. Health care organizations tasked with practice improvement and developing optimal models of health care delivery should consider investing in organizations that have developed trusted relationships with primary care practices. As seen in this study, the return on investment in developing relationships is substantial.

It is important to note that no direct monetary incentive was offered for participation. This lack of incentive may have affected recruitment efforts, as practices often expressed skepticism about potential future financial rewards for enrolling in this project even when offered the time of a skilled practice facilitator. Asch et al reported that “modest” financial incentives did not appear to affect participation rates of community-based physicians into health services research studies,¹⁷ but our experience suggests financial incentives would have increased enrollment, especially when described as a financial offset for the information-reporting burden. Although the practices were aware that value-based reimbursement was on the horizon, several described their previous disappointment with the amount of work and resources required for practices to become patient-centered medical homes—but without receiving the promised enhanced reimbursements or other financial incentives.^18,19 For example, one of the Healthy Hearts Northwest recruiting organizations, ORPRN, provided faculty support and technical assistance for the CMS Comprehensive Primary Care Initiative in which practices submitted an application to CMS instead of being actively recruited. That initiative required practices to make changes in care delivery to improve quality and reduce costs but also provided support with enhanced payments, data feedback, and learning support. Comprehensive Primary Care Initiative practices also participated in the CMS shared savings program and were reimbursed for their quality improvement efforts, receiving a median of $175,775 per practice ($51,286 per clinician) in the first year of the program.^14,20 In Oregon, health systems that owned practices submitted applications for multiple clinics with most (76%) of the 65 Comprehensive Primary Care Initiative practices affiliated with 7 large health systems. The federally qualified health centers and federally designated rural health clinics were not eligible to participate. In contrast, EvidenceNOW emphasizes practice support through at-the-elbow practice facilitation, learning support, and using data for improvement, but it was not linked to a specific shared savings program where the practice might have received increased payments for improvements in care quality. To our knowledge EvidenceNOW is the first large, federally funded study to recruit primary care practices for quality improvement.

Practices that declined to participate frequently described competing projects or health system initiatives or simply being too overwhelmed by existing federal reporting requirements to participate. In earlier studies, it was often possible to engage with smaller primary care practices by establishing a relationship with the physicians or clinicians in the practice, gaining their trust, empathizing with the needs of their practices, and delivering on promises of support.^17,21 In 2017 engagement and recruitment required relationship building at multiple levels, with health information technology staff, health system leaders, administrative staff, and practice staff involved in reporting quality measures and implementing tailored practice facilitation that aligns quality improvement initiatives with federal and local practice strategic goals.^12,22,23 The cost of recruiting practices without a existing partnership is expensive, costing 7 times more than engaging familiar practices. Supporting the infrastructure of organizations that have a sustained, trusted relationship with practices is needed in large-scale practice improvement efforts.

There are limitations to our study. For any individual practice, multiple individuals may have made contact during recruitment efforts; therefore, attributing recruitment success to specific characteristics or roles of recruiters is not possible. Although study teams in Healthy Hearts Northwest and Healthy Hearts in the Heartland had similar types of team members in recruiting activities, the recruiting process for each team varied as did the distribution of senior and junior team members and the team members with clinical or practice facilitation experience. The depth of prior partnerships was not defined beyond having completed a project with the recruiting organization. Our study team did not collect data on the number of interactions with practices that declined participation and are not able to report whether the decision not to participate occurred after a single contact or after multiple contacts.

It is important to recognize that recruitment of a diverse set of practices for large-scale quality improvement or redesign studies requires an unanticipated or unplanned expenditure of resources, thus, extending the recruitment period. We did not anticipate the time and effort required to enroll practices which delayed the implementation of our planned 15-month practice facilitation intervention by 6 months for Healthy Hearts Northwest and by 2 months for Healthy Hearts in the Heartland.

The Northwest (Healthy Hearts Northwest) and Midwest (Healthy Hearts in the Heartland) recruitment experiences highlight 3 key messages: (1) existing relationships are important; (2) even with an existing relationship, recruitment requires multiple contacts with each practice to enroll them; and, (3) the high cost of recruitment at $5,529 per practice. The return on investment in existing relationships is substantial, with a cost of $2,366 per enrolled practice as opposed to the cost of practices with no prior connections, $15,878 per enrolled practice. Investigators planning to study implementation and improvement strategies or implement pragmatic clinical trials across large numbers of primary care practices should build on existing relationships, ensure that adequate funds are available for high-touch strategies, and plan for the long time frame needed to meet recruitment targets. Our findings also highlight the need for further research focused on developing and comparing the costs and effectiveness of different strategies for recruitment of primary care practices as subjects in future improvement studies, disseminating the findings, and using them to design more cost-efficient studies in the future.