Randomized Controlled Trials: Do They Have External Validity for Patients With Multiple Comorbidities?

INTRODUCTION

Clinical practice guidelines are one widespread response to the modern dilemma of combining quality with efficiency to meet patients’ needs for and expectations of evidence-based treatment.¹ Guide-lines are devoted to helping clinicians with clinical decision making.² An important part of guidelines, which are based on a detailed review of the relevant literature about a given subject, are the published reports of randomized controlled trials (RCTs), widely accepted as the criterion standard of rigorous research design for clinical studies.³ The external validity and generalizability of many RCTs have been questioned, however.³ To which group of patients can the results of these RCTs be applied? It is not always easy to track down that information from the RCT reports.

To ensure the internal validity of their findings, many RCTs exclude patients with multiple comorbid conditions. In other cases, comorbidities of patients actually enrolled in the RCTs are not reported. These trials, however, provide the data that inform the justification for use of new treatments and interventions for all patients. Excluding a subset of the population from such trials or from the final reports means important information about the proper use of a treatment or intervention for that subset is not available. Numerous pharmacological treatments and interventions dealing with isolated chronic conditions take little account of the multiple morbidities experienced by the majority of patients in general practice.^4–6 The resulting guidelines may offer a simplified, potentially inadequate approach to the treatment because of inadequate attention to the comorbid illnesses.⁷

Hypertension, a chronic condition often associated with multiple comorbidities, is the frequent subject of such updates and of new guideline publications.^8–12 Almost 25% of Canadian adults have this condition.¹² The aim of the current study was to illustrate the magnitude of the prevalence of comorbidities among primary care patients with hypertension who met the inclusion criteria of selected RCTs focusing on the treatment of hypertension.

METHODS

For this research, we used a data set of patients collected from a study¹³ conducted in the Saguenay region of Québec, Canada, from January to July 2003. Details of the methods and sampling strategies used to obtain the database are described in a previous issue of this journal.¹³ In brief, of the 1,085 adult patients solicited during consecutive consultation periods from 21 family physicians’ practices, 980 (90.3%; 660 women) agreed to participate. We included all consecutive patients attending appointments for a period of several weeks who gave their informed consent. This database of a representative sample of a primary care population contained information about all their chronic medical conditions, compiled and counted in a chart review. The prevalence of hypertension in the data set is 43.3%. A list of comorbidities associated with hypertension and their prevalence in the group of patients with hypertension of the data set is shown in Table 1⇓.

We randomly selected articles from the reference lists of the 2004 Canadian practice guidelines for management of hypertension.^11,12 Only those articles reporting RCTs of patients with hypertension were eligible for inclusion in our study. To be eligible, articles had to be easily accessible and to have treatment of hypertension as the main topic. There were 25 RCTs that met our inclusion criteria. Using a random number generator, we arbitrarily selected 5 articles for inclusion in our study.

A list of inclusion and exclusion criteria was extracted from each RCT selected, and these criteria were then applied (1 study at a time) to all the patients from our database. Patients meeting the specified age for a given RCT and the diagnosis of hypertension, if required by the RCT, were included in a potential sample for that RCT. Those who remained after applying the inclusion and exclusion criteria were considered eligible for that RCT. For each RCT we identified the number of eligible patients with comorbidities and computed the mean number of chronic conditions.

RESULTS

Inclusion and exclusion criteria of the 5 RCTs randomly selected for this study are shown in Table 2⇓. Four^14–17 of the RCTs reported only a few comorbid conditions relevant to hypertension for participants at the time of randomization. In the fifth study,¹⁸ those affected by some chronic diseases (Table 2⇓) were excluded, but the presence of other chronic diseases was not reported, which precluded any comparison of the total number of comorbidities between eligible patients from our data set and the number of comorbidities of patients actually enrolled in RCTs.

Results of the analysis of our data set after applying the inclusion and exclusion criteria for each RCT are summarized in Table 3⇓. For each RCT, we identified a number of eligible patients by applying those criteria (Table 2⇑). The percentage of eligible patients for a given RCT who also had comorbidity ranged from 89% to 100%. The mean number (± SD) of chronic conditions among eligible patients with comorbidities for each study ranged from 5.5 ± 3.3 to 11.7 ± 5.3.

DISCUSSION

Many issues potentially affect the external validity of RCTs.³ With our study, we wanted to call attention to an issue of increasing relevance—comorbidity. Results from our study suggest that RCTs targeting a chronic medical condition such as hypertension would most likely find a great many patients with comorbid conditions during the screening process. Whether these patients are sampled or excluded should be reported. This situation applies mainly to those RCTs that enroll patients from primary care practices, because it is known that patients with multiple morbidities are not a subset but the majority of patients seeking medical assistance in primary care.¹³

None of the RCTs we analyzed reported how many patients with multiple morbidities were excluded or sampled after meeting the inclusion criteria. Four RCTs^14–17 reported the presence of only some comorbidities limited to those relevant to hypertension in participants. The absence of information on whether patients with comorbidities (other than the few mentioned in some trials) were excluded from the RCTs prevented us from having the evidence to address deeper in this discussion the important issue of patients with comorbidities who are excluded from RCTs. This problem limits the usability of many RCTs in family medicine. The importance of this issue is clearly expressed in the conceptual analysis for developing efficacious interventions published under the name of RE-AIM (reach, efficacy, adoption, implementation, maintenance) framework.¹⁹ Several dimensions are proposed to be examined in this framework to evaluate the development of efficacious interventions. The one that refers to the “representativeness” of individuals participating in an intervention is of particular relevance for RCTs. Representativeness is defined as the similarities or differences between those who participate in an intervention and those who are eligible but do not.¹⁹ If participants and eligible individuals are very different, then the efficacy of the intervention is compromised. When RCTs are conducted with samples in which patients with comorbidities are excluded from the study or not mentioned in the final report, the impact of these studies is similarly compromised.

Addition of other variables, such as comorbidity, to a subgroup analysis necessitates the recruitment of substantially more participants into a trial to obtain the same statistical power, which, in turn, increases the trial’s cost. Exclusion of patients with comorbidity from RCTs, however, challenges the external validity of trial findings by limiting their widespread applicability. Indeed, not only the exclusion of patients with comorbidity but also excessive exclusion criteria in general lead to exclusion of a large proportion of patients and makes results not applicable in primary care.

All coexistent diseases are potentially important when treatments target specific chronic diseases. For example, depression is a psychiatric comorbidity that RCTs targeting hypertension may consider irrelevant; however, depression in patients with hypertension can result in a difficult clinical course because depression may adversely affect the patients’ adherence to medication and self-care regimens. A greater number of multiple chronic conditions generally results in an increased number of medications. Given that nonadherence to prescribed treatments is common,²⁰ the likelihood is high that the results of RCTs would be different by the inclusion of patients suffering from multiple comorbidities. Furthermore, increased numbers of medications, as a consequence of the number of medical conditions, are associated with a higher risk of adverse effects,²¹ and some adverse effects may not be identified if patients who consume large numbers of drugs are not under study

None of the selected RCTs for this study addressed the issue of the competing demands raised by treating patients with multiple comorbidities. In family practice many visits are complex, and physicians are addressing more than 3 problems more than one third of the time.²² These competing demands involve such aspects as the time for patient management,²³ cognitive strategies of family physicians,²⁴ and interactions among medical conditions.⁷ The aspect of interactions is particularly important in RCTs focusing on a single disease. The treatment for 1 disease can also adversely affect another disease, and it may be necessary to prioritize the diseases requiring treatments. In some cases, it may not be feasible to apply all indicated treatments simultaneously in a patient.⁷ RCTs should not ignore this real-world environment of patients with comorbidities in primary care.

Collecting data about all comorbidities during RCTs can be a difficult task. Indeed, to take into account and analyze all the individual diseases participants may have is not feasible. Comprehensive comorbidity measures that include many comorbid conditions in 1 valid variable and take this potential confounder into account do exist, however.²⁵ Application of such methods to the evaluation of comorbidity could strengthen both the internal and the external validity of RCTs.

Results of this research are particularly relevant to family practice, where the prevalence of patients with multiple chronic conditions has been reported to be higher¹³ than in the general population.²⁶ In our data set, the prevalence of having 2 or more medical conditions in the 18- to 44-year, 45- to 64-year, and 65-year and older age-groups was, respectively, 68%, 95%, and 99% among women and 72%, 89%, and 97% among men.¹³ According to the general health survey conducted by Statistics Canada in 1998–1999, the prevalence of having 2 or more medical conditions in the 20- to 39-year, 40- to 59-year, 60- to 79-year, and 80-year and older age-groups was, respectively, 11%, 26%, 55%, and 64%,²⁶ which must have had an influence on the high proportion of eligible patients with multiple chronic conditions that we found (Table 3⇑). The percentage of patients with multiple chronic conditions in the general population is not negligible, however, and our point that results of RCTs concerning these patients should be reported is equally valid for studies in the general population.

A limitation of this study is that the inclusion and exclusion criteria of the different trials were applied to an existing data set, and we were forced to adjust criteria used to define inclusions and exclusions. Even so, we always applied criteria that were stricter than those of the trials. For example, when an exclusion criterion was poorly controlled hyperlipidemia or diabetes, we excluded all patients with hyperlipidemia or diabetes; or if patients with plasma creatinine concentrations of more than 2.5 mg/dL had to be excluded, then we excluded all patients with renal problems. This way, we should have actually contributed to lower the number of comorbidities in the eligible population.

A balance between external and internal validity is urgently needed to inform the justification of application of new treatments and interventions to patients with multiple chronic diseases. The increasing health problem of comorbidity should not be underestimated. Results from this study suggest that RCTs targeting a chronic medical condition such as hypertension could find that, in a sample taken from family practice, most eligible patients have comorbidities. In this study, we chose the example of hypertension as an index disease to illustrate the potential implications for the majoity of patients with hypertension whose treatment could be inadequate because of the evidence it is based on. Research devoted to generating knowledge to be applied in medical practice should take into consideration the complex reality of the situation.