Randomized Controlled Trials: Do They Have External Validity for Patients With Multiple Comorbidities?

Evidence-based medicine (EBM) is, at first, a paradigm. It is based on a subset of conditions that are now taken for granted. Fortin and al (1) are remembering us that this paradigm is all but perfect, especially in the realm of the primary care practice. We already knew it... But Fortin and al are putting another brick to the wall of its limitations, as did Mant and al earlier this year(2).

The previous comments by Soubbhi, Hudon, Hogg, Wright, Bayliss (among others) are also reflecting a very practical point of view of the Fortin and al article : how can we care about patients that are everything except controlled-randomized individuals ? Our guidelines call for measurable results, as efficiency is often based on quantitative data, but this is all but easy to transpose to our practice.

In fact, what Fortin and al are reflecting in their article is that we just don’t know. Starfield (3) hypothesis that the number needed to threat in some studies might be defavorably biased by the heterogeneity of their population. However, we just don’t know. Are RCT valid, on an external basis ? How complex should be the usual table 1 given in published research ? And could this lack of information be detrimental for our patients ? We don’t know either. Is Bayliss suggestion workable ? Probably, but, again, there is no real answer available. We still have work to do.

Fortin and al suggest that qualitative research might be one of the answers. Maybe, but it will call for a change of paradigm. Each of our primary care setting is different : in population (rural, urban, young, aged, with or without multiple co-morbidities), in type (family practice group, private clinic, hospital, emergency room), in culture, etc. How can we study that ? That is the real question.

Also, as Starfield (3) pointed out, the burden of disease does not equal the disease in itself, and often the research endpoints are not reflecting the real burden of the disease for our patients or their experience of it. In fact, rare are the studies that are considering health and wellbeingness as endpoints.

EBM is not the “one size fit all” answer for our everyday challenging practice of medicine. Nevertheless, and hopefully, we might never be able to get a “one size fit all” guideline anyway (4). Family medicine is almost all about uncertainty... As Hudon wrote : that is the essence of our practice : an Art.

1. Fortin M, Dionne J, Pinho G, Gignac J, Almirall J, Lapointe L. Randomized controlled trials: Do they have external validity for patients with multiple comorbidities? Ann Fam Med 2006; 4 (4): 104-8.

2. Mant J, McManus RJ, Hare R, Applicability to primary care of national clinical guidelines on blood pressure lowering for people with stroke: cross sectional study, BMJ 2006;332 (7542) :635-637

3. Starfield B. Threads and yarns: weaving the tapestry of comorbidity. Ann Fam Med 2006;4(2):101-3.

4. Mickan S, Askew D. What sort of evidence do we need in primary care ? BMJ 2006; 332 (7542) : 635-637.

Competing interests:   None declared

I would like to thank Fortin et al. (1) for their paper on the generalizability of clinical trials to our patients with multiple chronic diseases. Knowing that patients with multimorbidity are excluded from clinical trials makes me question my prescriptions: are they based on proven facts? Do their potential advantages sufficiently outweigh the risks of secondary effects, drug interactions or iatrogenic complications?

Without arguing against the importance of evidence based medicine, this problem calls our attention towards other dimensions that make the practice of medicine an art, as Soubhi highlights in his comment. Do we concede enough importance to other non-pharmacologic actions such as changes in lifestyles or the development of strategies to encourage patients to take a more active role in their health? (2). Even in front of most convincing data, there is a complex patient who has to "manage" his/her comorbidity daily.

(1) Fortin M, Dionne J, Pinho G, Gignac J, Almirall J, Lapointe L. Randomized controlled trials: Do they have external validity for patients with multiple comorbidities? Annals of Family Medicine 2006; 4 (4): 104-8.

(2) Young LE, Hayes V. Transforming Health Promotion Practice. Concepts, Issues, and Applications. Philadelphia: F. A. Davis; 2002.

Competing interests:   None declared

Thinking of multiple morbidities in terms of complementary or interacting components is one good way of thinking about the problems that they challenge us with. But it is not the only one. I believe that, as van Weel and Schellevis suggest(1), we also need to think harder into whether standardization and prefabricated structures (such as RCT based management guidelines) is the way to go in the first place. Ultimately the problem that multiple maladies in the same patient raises for clinicians is a problem of illness management, a problem of how to make decisions in the context of ill-defined problems and fragmentary evidence, a problem of how to design adaptive responses to unpredictable aspects of the illnesses, a problem of how to organize multiple resources to achieve specific health goals. Information in this context, even of the best kind (RCTs for example), is only one of the ingredients in several possible designs. The others belong to the realms of experience, practice, intuition, creativity. At the end of the line are different designs each fit to respond to specific purposes, their ultimate test being their relevance to the patient's specific needs, goals, and wants.

Maybe it is worth our efforts to try to understand how clinicians can tune into the changing experience of illness and take in the information they need to achieve their goals as the illness unfolds. This kind of thinking calls for understanding how clinicians expand their competence, how they respond to the emergent, how they learn and innovate, how they create. This is in fact what clinicians in their early training start doing when they take responsibility for their first patients; when they realize that textbooks do not contain all the information they need.

(1) van Weel C, Schellevis FG. Comorbidity and guidelines: conflicting interests. www.thelancet.com 2006; 367:550-551.

Competing interests:   None declared

The ongoing discussion about the external validity of RCTs is very interesting and I would like to thank all participants for their contributions. Something that struck me was the fact that comorbidity (or multimorbidity when we do not refer to an index condition) really needs to be redefined. The comment by Bayliss is especially enlightening in this respect. She categorized comorbidities by their effect on the main condition (positive, negative or neutral effect). One implication that I see, is that a definition of comorbidity or multimorbidity should include the dimension of “competing demand”, which would represent a major change in the definitions originally published by Feinstein for comorbidity (1) and by van den Akker for multimorbidity (2). This addition to the definition would make it more relevant for clinical practice. There are probably some other dimensions that have to be considered in the definition. This suggests that a qualitative approach of the question may help identify other dimensions of the concept as it is experienced by primary care workers. I strongly think that this kind of work would contribute to developing a comprehensive definition that would be equally understood by everybody.

An advantage to sharing a comprehensive definition of comorbidity or multimorbidity is that we might end up proposing new ways to measure it. The well known, frequently used and cited Charlson index (3) may not be capturing all the dimensions pertaining to the concept. We have shown that this index does not perform well when used to study health-related quality of life (4). Adapting clinical trials for multimorbidity may in fact require an index based on a solid conceptualization to help improve the interpretation of the findings and their generalizability. Of course, there will be a cost associated with that, but it would be worth taking for the wellbeing of our patients.

Feinstein AR: The pre-therapeutic classification of co-morbidity in chronic disease. J Chron Dis 1970;23:455-469.

Akker M vd, Buntinx F, Ross S, Knottnerus JA. Comorbidity or multimorbidity: what’s in a name? A review of the literature. Eur J Gen Pract 1996;2:65-70.

Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373-383.

Fortin M, Hudon C, Dubois MF, Almirall J, Lapointe L, Soubhi H. Comparative assessment of three different indices of multimorbidity for studies on health-related quality of life. Health and Quality of Life Outcomes. 2005;3:74.

Competing interests:   None declared

Fortin and colleagues deserve a rousing hand for their work on the impact of co-morbidities as potentially limiting the generalizability of RCT's for patients in primary care -- and the need to define the extent to which the exclusion of patients occured due to co-morbidities. As one very wise country doc from Northern Wisconsin once told me when I was the director of WReN: "Never do research on diabetes -- do research on PATIENTS with diabetes!"

Competing interests:   None declared

There are two sides to the equation for measuring care of persons with multiple medical conditions: the generation of recommendations for care and the implementation of these recommendations. Neither side of this equation currently exists for persons with multimorbidities. In this issue of Annals, Fortin et al. nicely illustrate one of the missing pieces in generating recommendations for care: that the randomized controlled trials on which most care guidelines are based do not necessarily either include or adjust for a the comorbidities of the participants. (1) This calls into question the generalizability of any care recommendations based on these RCTs and adds to the many reasons it is difficult to assess the quality of care delivered to this population.

Comorbid conditions (i.e. those present that are in addition to the condition of interest) in RCTs fall into three categories: Those clusters of comorbidities that are pathophysiologically related to the condition of interest and require congruent treatment plans. (These conditions might be called positive effect modifiers of the disease-specific outcome of interest.) Comorbid conditions that have discordant (and potentially competing) treatment recommendations (negative effect modifiers of the outcome of interest). And comorbid conditions that do not modify the outcome of interest at all.

Results from disease-specific RCTs that do not account for comorbid conditions apply to only the subset of the population with comorbidities that are non-modifiers of the outcome of interest. When generalizing to patients with negatively-modifying comorbidities, disease-specific findings are at best difficult to apply, and at worst harmful. (2)

As an example, consider the case of an RCT participant with diabetes, hypertension, osteoarthritis, dyspepsia and asthma enrolled in a trial of exercise for blood glucose control. The comorbidity of hypertension would potentially improve the chances of achieving the outcomes. The comorbidities of arthritis and asthma might decrease the likelihood of meeting the same outcomes; and the comorbidity of dyspepsia might not modify the outcome at all. For investigations of different conditions of interest (for example the effectiveness of dyspepsia treatment) the same list of multimorbidities would cluster differently.

With these categories in mind, it would not be difficult to use existing analytic methodologies to manage comorbid conditions in disease- specific investigations. Once comorbid conditions are categorized as potentially positive, negative or neutral effect modifiers of the disease- specific outcome of interest, analyses could be stratified by these disease clusters or the relevant interaction terms included in the multivariate analysis. This would avoid the loss of external validity that results with the most commonly used approach to potentially-interacting comorbidities: lengthy exclusion criteria.

The availability of validated comorbidity measures that utilize administrative data make these analyses possible and could improve recommendations for care for persons with multiple medical conditions.

(1) Fortin M, Dionne J, Pinho G, Gignac J, Almirall J, Lapointe L. Randomized controlled trials: Do they have external validity for patients with multiple comorbidities? Annals of Family Medicine 2006 Mar;4(4):104- 8.

(2) Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA 2005 Aug 10;294(6):716-24.

Competing interests:   None declared

Fortin M., et al are to be congratulated for their elegant study. Their recommendation that randomized controlled trials report the co morbid conditions of enrolled patients is an important contribution to research methods.

Determining the prevalence of patients with co morbidities in primary care is influenced by the patient sampling strategy. Fortin M., et al sampled all consenting consecutive patients attending appointments in a primary care clinic over a period of several weeks. A consecutive sampling strategy of patients in primary care offices will bias selection towards patients who attend frequently and patients with multiple morbidities attend frequently. Fortin M., et al may have overestimated the prevalence of co morbid conditions in the practice but they succeeded in making the case that randomized controlled trials should report the co morbid conditions of enrolled patients.

Competing interests:   None declared

Pragmatism is key to the practice of effective primary care medicine. Fortin et al allude to combining quality with efficiency to meet patients’ multiple complex health needs within a limited time-scale.(1) Such practice requires a pragmatic approach and arguably primary care, more than any other branch of medicine, requires a pragmatic approach to the provision of health care. Interestingly pragmatism is also a feature of real-world randomised controlled trial methodology. Many researchers whilst conducting RCTs in the health care setting take great pride in claiming a pragmatic rather than explanatory approach. However the claim to pragmatism is compromised when such RCTs have strict inclusion and exclusion criteria such that large numbers of people with multiple morbidity are deemed ineligible to participate. I have spent several years conducting RCTs amongst homeless people and drug users residing in prison with multiple morbidity. Therefore I found Fortin et al’s research considering the dilemmas of conducting valid RCTs in the complex real world primary care setting a compelling read. They have exhorted us to consider afresh how to consider the common clinical research conundrum – the need to balance research rigour with an approach that is feasible. They raise the possibility of subgroup analysis as a means of accounting for co-morbidity yet rightly acknowledge that this necessitates increased recruitment for the same statistical power. However this area merits further consideration by the primary care scientific community. The authors inform us that valid tools exist to comprehensively measure co- morbidity.(2) Perhaps one possibility could be to use such tools to identify those patients with comorbidities. Following identification, a process of randomisation stratified for comorbidity would ensure equal allocation across the intervention groups.

Fortin et al have provided those of us working in primary care with an excellent article. What is clear is that the issue of co-morbidities will not go away and we should do well to heed their recommendation that it should not go under-reported. The article has genuinely changed the way I practice medicine. The next time I receive the reductionist, simplistic “how to” protocol to manage a single disease in primary care I shall appraise it just a little more critically for the external validity of the supporting evidence base. For years I’ve heard family doctors dismiss research as not applicable to primary care because it was not conducted in the primary care setting. Often such comments are based on prejudice and ideology rather than lucid rational thought. This article provides us with a framework to critically appraise our received primary care wisdom on the basis of scientific rigour rather than primary care professional territorialism.

References

1. Fortin M, Dionne J, Pinho G, Gignac J, Almirall J, Lapointe L. Randomised Controlled Trials: Do they have external validity for patients with multiple comorbidities? Annals of Family Medicine 4(2): 104-108

2. de Groot V, Beckerman H, Lankhorst GJ, Bouter LM. How to measure comorbidity, a critical review of available methods. J Clin Epidemiol 2003; 56: 221-229

Competing interests:   None declared