Declining Participation in Primary Care Quality Improvement Research: A Qualitative Study

Participant Recruitment

This qualitative study drew on data from 4 QI research projects funded under the Agency for Healthcare Research and Quality’s EvidenceNOW initiative, which tests the use of a supportive model to help primary care practices implement evidence into practice and improve QI capacity. Two projects were aimed at helping practices implement screening, brief interventions, and treatment for unhealthy alcohol use, and 2 were aimed at improving heart health, focused on blood pressure and tobacco use (Table 1). All projects were based in the Midwest (Illinois, Indiana, Michigan, Ohio, and Wisconsin) and included health information technology support, data feedback, shared learning collaboratives, and expert consultation. All 4 projects used practice facilitation, meaning each participating practice had a dedicated coach to help facilitate meaningful change. Time commitments for participation ranged from 1 to 3 hours per month. Some projects required the whole practice to participate, whereas others focused on a subset of clinical or office staff (ie, only practice managers and/or health care professionals). All participating practices were required to extract performance measure data from their electronic health records (EHRs), and all offered continuing medical education credits to participants.

We asked recruiters from all 4 projects to provide us with contact information for individuals who declined to participate in their studies. Using a purposive approach, the recruiters prioritized contacts with whom they had some interaction (eg, a conversation during the recruitment phase or participation in a previous project).¹⁵ Those individuals, primarily physicians or practice managers, were contacted by e-mail and invited to participate in a 15-minute interview. We followed up with reminders approximately 1 week later; if we still received no response, we reached out a third and final time, asking the potential respondents to complete an interview or a brief questionnaire.

Data Collection

Telephone interviews were held between July and September 2022, and followed a semistructured interview guide tailored for each project (Supplemental Appendix). The guide included questions about why the practice declined participation, elements of the project that posed a challenge for the practice (eg, data collection requirements, time commitment), and strategies for making the project more feasible or attractive in the future. All interviews were led by a team member with substantial experience conducting interviews with primary care physicians and staff (M.M.), and interviews were digitally recorded and transcribed verbatim.

The questionnaire contained 5 open-ended questions that aligned with the interview guide (Supplemental Appendix). Some potential respondents declined to participate in an interview or to complete the questionnaire, but e-mailed a response describing why they declined to participate in the QI project. We stored data from all 3 response modalities (interview transcripts, questionnaire responses, and e-mailed responses) in Dedoose version 9.0.17 (SocioCultural Research Consultants), a qualitative analysis software.

This study was given a “not regulated” determination by Northwestern University’s Institutional Review Board. Verbal informed consent, including permission to record, was nonetheless obtained from all interview participants. The study followed the Consolidated Criteria for Reporting Qualitative Research (COREQ).¹⁶

Analysis

We performed thematic analysis to characterize perspectives of respondents using the 3 response modalities.¹⁷ The data were first read by 2 team members (M.M., J.H.) for overall clarity, and they met to discuss initial observations and redundancy of the data, concluding that data saturation was reached.¹⁸ Next, team members developed 10 deductive codes, based on the interview and survey questions.¹⁹ Two team members (M.M., J.H.) independently coded 4 transcripts, and after discussion of discrepancies, the remaining data were coded by a single team member (M.M.). A third team member (S.P.) participated in identifying broader patterns of meaning across the coded data.

The 3 team members independently wrote analytic memos, which aided in data reduction by tying together pieces of data into conceptual clusters and producing candidate themes.^20,21 They shared their individual memos, and through discussion, agreed on primary and secondary themes. The themes were then checked to ensure they captured the most important features of the coded data and were coherent and substantial with clear boundaries, resulting in a final set of primary and secondary themes. Lastly, the team selected verbatim quotations to illustrate the themes.²² Throughout the process, they kept coding definitions, analysis decisions, and ideas about emerging themes in a log so that the process could be replicated.

Sample Characteristics

Of the 109 practices contacted across the 4 projects, 31 (28.4%) provided responses (Table 2). Eleven respondents had titles of vice president, director, or chief clinical officer; 10 were physicians or nurse practitioners; 6 were practice managers; and 4 were nurses. Sixteen respondents (51.6%) participated in an interview, 12 sent an e-mailed response, and 3 completed the questionnaire.

Reasons for Declining Participation in a QI Project

Table 3 provides key primary and secondary themes and representative quotes related to reasons why the practice declined participation. Overwhelmingly, staff turnover and shortages, spanning both physician and support staff roles, were identified as primary reasons for declining, and many noted that staffing challenges were exacerbated during the pandemic. These were general staff shortages, not shortages of individuals with QI or practice transformation experience, as no specific QI expertise was needed to participate in the projects. Because of staffing shortages, several practices were struggling to maintain normal operations, and many noted that it was contributing to burnout among remaining staff. Staff were leaving practices unexpectedly, and restaffing was increasingly difficult. Numerous respondents also described general time constraints that prevented them from participating in the QI studies. Although the reasons why practices did not have time to join the QI project varied (eg, increased documentation burdens, site visits from federal sponsors), respondents often reported that clinicians were “stretched too thin” and simply “could not be asked to do one more thing.” Importantly, respondents frequently reported that the projects’ time commitment requirements were reasonable; they simply did not have that time to spare. A few skeptically noted, however, that the commitment may have been understated in the recruitment materials.

The secondary themes in Table 3 show the less common reasons why respondents declined participation. Some respondents noted that they did not feel compelled to participate because they were confident in their ability to provide good care for patients with unhealthy alcohol use or those at risk for cardiovascular health issues. These practices noted having previously participated in QI projects on those topics or having reviewed relevant performance measures and believing they had little opportunity to further improve. Some respondents also mentioned challenges with EHRs. In some cases, the transition to a new EHR system was causing disruptions and contributing to time constraints. In other instances, respondents reported that they did not believe their EHRs could easily report the performance data needed for participation in the QI project. Some respondents also expressed concern about the time and staff effort involved in extracting the data from the EHRs, especially for small practices without an information technology department. Even if a practice opted to allow a practice facilitator from the project to extract the data, someone from the practice would still have to allocate time to grant the practice facilitator access to the system. Finally, some respondents noted that they would have to be compensated for participation in a QI project that would reduce the time clinicians spent on patient care, or that the compensation offered by the project was insufficient.

Recommendations for Improving the Feasibility or Attractiveness of QI Projects

When asked how the recruitment process could be changed to improve participation, many respondents struggled to make recommendations (Table 4). The primary theme was that staffing shortages and general time constraints were not something that could be addressed by an external QI project. Nevertheless, several respondents offered relatively minor recommendations, for example, calling practices in advance to identify the “right” contact within the practice before sending a recruitment e-mail, explaining how the QI project will benefit the practices’ reimbursement in pay-for-performance programs, and designing the QI project so that staff, rather than physicians, are the key participants.

Additionally, all respondents were asked whether financial compensation (or a higher level of financial compensation) would have influenced the practices’ decision to participate. Although some respondents thought compensating the practice for lost patient care revenue might make a difference, they struggled to identify an appropriate level of compensation. Most respondents noted that compensation was not a factor influencing their participation decision.

Implications

To engage in QI, primary care practices need sufficient time and staff capacity. Currently, practices are tightly staffed, with many practices struggling to maintain normal clinical operations.²⁶ The high turnover rates experienced during the pandemic resulted in lost institutional knowledge of practice operations and EHR functions, making it even more challenging for practices to engage in new QI projects.

Over the past 2 decades, policy makers have encouraged QI through quality-reporting programs, support for meaningful use of EHRs, and value-based payment reform. These efforts, while laudable in goals and outcomes, have also increased burden on practices. There have been several calls for policy makers to undertake fundamental reforms that would create positive work and learning environments for clinicians, reduce administrative burdens, and provide supports for clinician well-being.^10,27 These initiatives could help to alleviate burnout leaving clinicians better able to engage in QI projects. We also suggest that policy makers consider more basic supports for primary care QI, namely, direct support to expand the primary care workforce and provide additional training to improve QI capacity. One option might be to establish a loan repayment program and/or scholarships for the primary care workforce, such as those offered by the National Health Service Corps. There appears to be movement toward these supports with the recently announced $80 million Nursing Extension Grant Program to expand and diversify the nursing workforce.²⁸ Programs like this could be expanded beyond nursing. The Centers for Medicare & Medicaid Services (CMS) could also consider adding additional financial incentives for practices engaged in QI to support protected time for staff to work on QI projects. Such action could be key to the CMS Innovation Center achieving its goal of having all Medicare fee-for-service beneficiaries in practices accountable for quality and total cost of care by 2030.²⁹ Small and independent practices are likely to need additional assistance.³⁰

It is also notable that EHRs still posed a barrier for some practices to participate in QI projects, either because of inability to extract relevant performance measures or because of disruptions resulting from EHR transitions. The Health Information Technology for Economic and Clinical Health (HITECH) Act and its Meaningful Use Program were successful in using incentives to encourage adoption of EHRs and transfer of information electronically.³¹ Today, the incentives have been replaced by financial penalties for practices that do not use certified EHR technology and demonstrate meaningful use. Our results suggest that practices may need more support to meet this standard and adopt clinical decision support, connect to health information exchanges, conduct electronic referrals, and use EHRs to their full potential to enhance quality.

Leaders of QI projects also need to reconsider recruitment goals, particularly as primary care physicians and staff are reporting high levels of burnout. Because of the need for statistical rigor, the EvidenceNOW projects aimed to enroll tens or hundreds of practices; however, ambitious enrollment requirements may not be realistic for QI studies that take time away from patient care. More flexible evaluation designs may be needed. Further, project leaders should embrace more modest strategies to boost recruitment suggested by respondents in this study, including compensation for practices, which may make a difference for some practices. QI project leaders, however, could also consider more substantial enhancements that address the larger staffing and time constraints, for example, embedding a project representative into the practice or compensating practices for every hour of participation, making enrollment cost neutral for the practice.²⁶ EvidenceNOW permitted compensation to participating practices only for efforts related to data collection; however, more substantial payments to compensate for lost patient revenue may be warranted.³² Ultimately, QI projects may have to “meet practices where they are,” through tailored support based on practice constraints. These recommendations, like those directed toward policy makers, would require substantial additional cost. There appears to be consensus among leading QI and primary care organizations, however, that in the absence of substantial investment in primary care, there are likely to be wide-ranging, adverse consequences for individual clinicians, health care organizations, and patient care.³³

Leaders of QI projects should also evaluate the true time commitment needed by practices to participate successfully. Some of our respondents were skeptical about the low time commitments described by the recruiters (eg, 1 to 2 hours per month). One approach is to survey clinicians who completed participation in a QI research project about the amount of time spent on the project and compare it with the expected time commitment.

Limitations

There are several limitations of this study to consider. Some respondents declined participation in the QI project more than a year before our interviews, introducing the possibility of recall bias among respondents; however, the recollection of why the practices declined was high, and there was frequent consistency of responses. Second, the response rate to our interview invitations was low, although this was expected as our focus was on practices that declined a QI project. To address the expected low response rate, we offered nonrespondents the option of completing a questionnaire and accepted e-mailed responses. The interviews yielded greater richness of data in terms of details and examples, but again, there was consistency across response modalities in terms of reasons why the practices declined participation and suggestions for improving the feasibility and attractiveness of future QI research projects. Third, the study may give an overly pessimistic impression of practices’ ability to participate in QI projects as all 4 of the EvidenceNOW projects were able to recruit practices to participate. Finally, although a qualitative approach provides unique value to research topics such as the one explored here, 2 potential sources of bias should be acknowledged. Recognizing that recruitment would likely be difficult for this follow-up study, we contacted only practices that had some prior interaction with recruiters or the study team. Practices that never responded to invitations to participate in 1 of the 4 QI projects were not contacted for this follow-up study. This purposive sampling may have introduced selection bias. We do not have information on the characteristics of practices that did not participate in the 4 QI projects or those that did not participate in this follow-up study, and therefore cannot compare them with participating practices to identify possible differences. Finally, the investigators’ close familiarity with EvidenceNOW projects may have introduced confirmation bias.