Randomized Trials in Primary Care: Becoming Pragmatic

• primary care
• randomized trials
• cluster randomized trials
• pragmatic trials
• stepped-wedge cluster randomized design
• COVID-19

The COVID-19 pandemic has tested many aspects of society, but has also reminded us anew of the limitations and challenges of our scientific approaches. Take the example of randomized trials. While multiple randomized trials were demonstrating the efficacy and safety of SARS-CoV-2 vaccines,¹ it became clear that other interventions against SARS-CoV-2 (eg, community masking, physical distancing, school closures, national lockdowns, etc) required research paradigms outside of the classic randomized trial design to which many scientists are accustomed.^2,3 This again reminds us that randomized trials may have significant practical limitations to their generalizability because they are in tightly controlled settings with narrow eligibility, and therefore often in settings divorced from the real world.⁴ Whereas classic randomized trials evaluate interventions in ideal settings, pragmatic trials evaluate interventions against real-world alternatives provided in routine care (especially in primary care). Typically, pragmatic trials also relax eligibility criteria which may allow for greater generalizability of study findings. With the benefit of generalizability, however, comes challenges that are unique to pragmatic trials. To balance the relative risks and benefits of both of these designs, investigators employ strategies that often hybridize the 2 designs to maximize benefit and minimize limitation. In this issue, 3 studies demonstrate increasingly used approaches to construct trials that are pragmatic, but retain features and benefits of classic trial design.

First, a randomized controlled trial led by Mitchell et al⁵ sought to evaluate the relative effectiveness of additions to a nationally disseminated readmission reduction program (called Re-Engineered Discharge [RED]) to reduce hospital readmission rates and emergency department visits among depressed patients. In intent-to-treat (ITT) analyses, the study found no difference in all-cause hospitalization between the study arms. Intent-to-treat analyses are used in trials to account for real-word deviation from treatment, and include all randomized study participants in prespecified analyses regardless of events after they are randomized (eg, noncompliance, study withdrawal, protocol deviation, etc). Intent-to-treat analyses are thought to produce less bias than when the randomized participants who were entirely adherent to their assigned intervention are included in this analysis.⁶ An alternative to an intent-to-treat analysis is to consider as-treated analyses which compares intervention groups that only include patients who actually received the treatment(s) without regard to their randomized assignment.⁶ In addition to intent-to-treat analyses, Mitchell et al⁵ also performed as-treated analyses and found that with sufficient uptake of the adapted RED intervention, patients saw a larger decrease in hospital readmission compared with RED alone. While it is tempting to consider the as-treated analysis a definitive analysis, it is known that as-treated analyses are more likely to be biased and exaggerate treatment effects.⁶ In real-world settings, complete adherence to any intervention is a challenge. Reporting ITT analyses and as-treated analyses present a full picture for primary care clinicians and researchers to put findings into context.

Next, Orrego et al⁷ present a cluster randomized trial which evaluates the effectiveness of a virtual community of practice on improving primary health care professionals’ attitudes toward empowering patients with chronic diseases. “Cluster randomizing” is an approach to make a trial more pragmatic in nature. In this approach, participants are randomized at the group level (eg, primary care clinic, health care professionals, etc), which has several benefits, especially when the target of the intervention is at the practice or health system level. Along with logistical conveniences for intervention delivery, a major reason to consider a cluster randomized trial is to avoid contamination bias (eg, intervention is adopted by health care professionals who were randomized to the control arm).⁸ Instead of randomizing patients to the intervention or control arms, this study randomized 63 primary care practices to study groups. Researchers considering this design should be aware that those benefits must be evaluated against potential limitations, including possible imbalance in clinic/system size, and wide provider and patient characteristic variation between study groups. Most notably, cluster randomized trials may have substantially reduced statistical efficiency as a result of variance inflation due to intracluster correlation.⁹ This reduced efficiency requires more patients to reach the same power as a classic randomized trial. More sophisticated statistical methods are required to properly address the data in cluster randomized trials.

Finally, another significant drawback to some cluster randomized designs in real-world practice is recruiting interested but busy practices to join a study where they may get randomized into the control arm—which still includes many tasks—but yet never receive the intervention (and therefore derive no short-term benefit from their extensive energy and cost output). This may understandably result in low recruitment and a lack of generalizability when certain kinds of practices cannot participate due to these barriers. However, a special case of cluster randomized design, which may mitigate these issues, is the stepped-wedge cluster randomized design. A stepped-wedge trial randomizes clusters (eg, groups of practices) to a sequence which determines when (not if) they receive the intervention.¹⁰ All practices contribute data pre- and post-intervention which allows the researcher to use the control period data to adjust for secular trends. It is an appealing pragmatic design because all practices eventually receive the intervention. This produces better statistical efficiency and thus greater power to detect an intervention effect compared with a parallel intervention/control cluster randomized trial. For stepped-wedge designs, however, several challenges need to be considered, as highlighted by Nguyen et al.¹⁰ These include: (1) retention challenges for practices who are randomized to receive the intervention later in the study period, (2) potential for contamination and Hawthorne effect, and (3) concurrent improvement activities or temporal trends in quality improvement may confound results, as well as others.

The pandemic has highlighted the persistent need to employ numerous trial designs to better include diverse primary care practices and produce consistent and better science. Primary care researchers should continue to embrace pragmatic trials; this requires careful thought at the protocol development stage in collaboration with a host of multilevel partners (patients, health care professionals, health systems, communities, researchers, and statisticians). This careful thought will produce more rigorous and applicable evidence, will engage a greater proportion of our workforce in the creation of science, and will facilitate healthier patients and communities.

• Received for publication April 6, 2022.
• Accepted for publication April 8, 2022.

• © 2022 Annals of Family Medicine, Inc.