Measures of Multimorbidity and Morbidity Burden for Use in Primary Care and Community Settings: A Systematic Review and Guide

Inclusion Criteria

We included studies with empirical data that enabled us to assess the validity and/or reliability of measures of multimorbidity when used in generalist primary care or population settings.

Assessment of validity depends on determining whether a measure is able to demonstrate associations that support an underlying theory about the relationship between the construct being measured and other variables.¹⁰ Because the nature of these anticipated relationships will vary in different settings, rather than addressing the validation of a measure, it is appropriate to assess the validity of a measure in a specific group of people and a specific context.¹¹

For this review, we included studies that provided data about associations between measures of multimorbidity and (1) patient sociodemographic characteristics, such as age, sex, and deprivation; (2) worse health outcomes; and (3) process measures, such as utilization of health care, costs, and quality of care. It was anticipated that a valid measure of multimorbidity would demonstrate associations with these variables. We also sought to identify articles comparing one measure of multimorbidity against another. Finally, we sought to identify articles that demonstrated the test-retest, intrarater or interrater reliability of these measures when used in a primary care or community context.

We included quantitative studies of any design that were predominantly based on adults. Participants had to be identified either from a generalist primary care setting or a population sample. We did not restrict searches by country or language, although we did require an English abstract.

Exclusion Criteria

We excluded studies in which participants were identified through their contact with specialist services or hospital admission. We also excluded studies of measures in which the presence of an index disease was integral to the measure (for example, measures specific to diabetes); studies of comorbidity (an additional disease in patients with a specified index disease); studies in which the multimorbidity measure was only used to show associations with variables related to secondary care (for example, in-patient mortality); and studies that described the prevalence of multimorbidity without studying associations with other variables.

Searches

We conducted a systematic review through searches in MEDLINE and EMBASE from inception to December 2009. Searches were undertaken in 3 stages, which were then combined. MeSH headings and free text were used to identify terms relating to (1) multimorbidity or comorbidity; (2) measures or indexes and terms for measures that we had already identified; and (3) ambulatory, outpatient, primary, or community care or general/community population.

The searches were developed iteratively to identify the combinations of terms that achieved an acceptable level of sensitivity and specificity. We repeatedly checked articles identified through different strategies against relevant articles already identified and articles identified through existing bibliographies.^12–14 We also selected other articles from our personal files, contacted other researchers, and checked reference lists from relevant articles. The final search strategy is shown in the Supplemental Appendix, available at http://annfammed.org/content/10/2/134/suppl/DC1.

Data Management and Extraction

One author (A.H.) conducted a preliminary screen of titles and abstracts to exclude articles that were clearly irrelevant. Abstracts from the remaining studies were screened independently by 2 authors to identify potentially relevant articles that were then reviewed independently in full text. Disagreements were resolved between the 2 authors, with discussion with a third author as necessary.

We extracted data about the characteristics of the study population, setting, outcome variables, study design, and main results into a Microsoft Access database.

We describe the measures identified below. Supporting tables provide details about the information needed to calculate each measure, along with details of which measures have shown evidence of validity by demonstrating associations with the specified patient, process, or outcome variables.

Disease Counts: 98 Studies

Disease counts were defined as a simple unweighted enumeration of the number of diseases. Disease counts specify whether the person has 1 or more of a limited list of conditions, but the conditions included in this list varied in different studies from 9 to 35 different items. These items may have been individual conditions, diseases, health problems, or categories of conditions or diseases. Disease counts may be self-rated, clinician-rated, or extracted from records. Disease counts are the most commonly used measure of multimorbidity and have been used mainly in relation to patient demographic characteristics and health outcomes and to a lesser extent process measures.

Chronic Disease Score (9 Studies)/RxRisk (8 Studies)

The Chronic Disease Score (CDS) uses pharmacy dispensing data to identify classes of medication that are taken as proxies for the existence of chronic disease (Table 1). The CDS has shown anticipated relationships with self-rated health status, functional status, hospitalization rates, and mortality.^{15,16,43,47–52} The original version¹⁵ considered 17 disease states with weights predefined by an expert panel. Notable subsequent versions include Clark et al’s revised CDS¹⁶ and RxRisk.¹⁷ Clark and colleagues considered an expanded number of diseases using weights for health utilization and costs derived empirically using health maintenance organization data. The RxRisk score, developed by Fishman et al, further expanded and revised the CDS, focusing on the estimation of future health care costs and increasing applicability to a wider range of pharmacy data sets and to children. Studies using the RxRisk model have shown anticipated associations with a wide range of variables (Supplemental Tables 1, 2, and 3).

Charlson Index and Variations: 38 Studies

Charlson et al developed this score for evaluating prognosis based on age and weightings for specific comorbid conditions.¹⁸ The validity of the Charlson index has been studied more extensively than other measures, particularly in hospital and specialist settings. Although it was developed and validated in hospitalized patients, it has since been adapted and validated in primary care and community populations.^19,50,51,53 There are several variations of the Charlson index, but studies comparing these variations suggest they produce similar results.^{21,51,54–56} The majority of studies using the Charlson index described the effect of multimorbidity on health outcomes, particularly mortality.

Adjusted Clinical Groups System: 25 Studies

The Adjusted Clinical Groups (ACG) System, a population/patient case-mix adjustment system based on medical records or insurance claims, measures health status by grouping diagnoses into clinically cogent groups. The ACG System was originally designed to predict future morbidity and use of health care resources.²⁵ Most studies of the ACG System described predictive models for a range of cost or process outcomes associated with multimorbidity.

Cumulative Index Illness Rating Scale: 10 Studies

The Cumulative Index Illness Rating Scale (CIRS) index uses a scoring system that includes 14 body system domains and a severity scale for each domain. The CIRS can be applied directly in consultations or from medical records (Table 1). Studies of the CIRS have found associations with a range of patient demographic characteristics, measures of process and health care utilization, and health outcomes. One study compared CIRS scored through direct patient observation or chart review and also assessed interrater and intrarater reliability. All methods produced comparable results. ⁵⁷

Duke Severity of Illness: 6 Studies

Duke Severity of Illness (DUSOI) is a tool for measuring a person’s illness severity that comprises 4 parameters of each diagnosis, namely, symptoms, complications, prognosis without treatment, and treatment potential. DUSOI can be completed at a consultation or from chart review. A few studies of the DUSOI demonstrated associations with age and sex, health care utilization, and quality of life. Parkerson et al found good interrater reliability for the DUSOI when rated by a physician or an auditor.^28,29

Other Measures: 21 Studies

Eleven other types of multimorbidity measure were used in studies, often in comparison with other measures (Appendix). These studies all described associations also found by more commonly used multimorbidity measures.

Comparison Studies: 15 Studies

Several studies have directly compared how different measures of multimorbidity were associated with relevant variables in generalist primary care or community settings.* Most of these articles suggested that the performance of the different measures studied was similar.^{17,29,44,47,50,51,58,61} The Charlson index and the ACG System appeared to be the strongest predictors of mortality,^47,50 whereas the ACG System and measures based on medication prescribed (Appendix) were strongest at predicting health care utilization.^17,50,51,53 Measures that include an assessment of functional status or subjective disease burden appear to be stronger predictors of a range of health outcomes than those that count diseases without adjustment for their severity or impact.^41,58–60 Some studies have shown that combining different types of measures improves the overall predictive performance of models.^29,41,51,60 Two studies have suggested that simple measures perform almost as well as more complex measures, for example, using a count of prescribed medications to predict health care utilization or a simple count from a list of major chronic diseases to predict mortality.^47,50

Summary of Main Findings

This review provides an index of previous literature for investigators seeking to use a multimorbidity measure in relation to a particular outcome. Researchers interested in the relationship between multimorbidity and health care utilization will find most evidence for the validity of the Charlson index, the ACG System, and disease counts. Evidence of validity in relation to patient or health service costs is strongest for the ACG System. For studies of the relationship between multimorbidity and mortality, the evidence is strongest for the Charlson index. The most commonly used measures of multimorbidity in relation to patient functioning or quality of life are disease counts and the Charlson index, but some studies have suggested that the CIRS is actually superior,^58,62 as are measures that incorporate self-reported disease impact and severity.⁴¹ That other measures have been used less often in relation to these outcomes does not necessarily mean that they are less valid, but their performance has been less well established.

Choice of Measure

The choice of measure is likely to be based on the suitability of the measure for the data available as well as the outcome of interest. The Charlson index, ACG System, disease counts, and prescription counts can all be calculated from patient records, and these measures are particularly suitable for cross-sectional studies based on electronic records or administrative data. Both the CIRS and DUSOI, however, require judgment about individual patients (also requiring manuals and training to ensure reliability) and cannot be automated for use with large volumes of data.

Measures based on routine data may be easy to use, but ease of use needs to be balanced against the quality of the data. All measures are dependent on the range of conditions recorded, how accurately and recently these conditions were recorded, and whether there is information about the severity and impact of conditions. Measures using clinician ratings or patient self-report will be up-to-date and may be more accurate at predicting functional outcomes if they include assessment of severity or disability. These measures, however, are often based on a more restricted list of diseases than measures based on records.

There are limitations to measures that use complex scoring. Changes to disease coding systems may mean that weights need to be reestimated, and relevant drugs used in medication-based measures are constantly changing, so scoring algorithms need regular updating. Proprietary risk adjustment systems, such as the ACG System, tend to use scoring systems that are not transparent and often have considerable costs to end-users.

The most common approach to measuring multi-morbidity is disease counts. Even so, it is hard to compare findings between studies, as different authors have included very different numbers of diseases, sometimes providing no details about which diseases are included or the criteria for inclusion.⁶³ Most studies are based on counting so-called chronic diseases, but chronicity is rarely defined. The number of diseases is also related to the level of disease abstraction—for example, some measures count cancer as one condition, whereas others count each malignancy separately.⁶³

It might be anticipated that such measures as the Charlson index, the ACG System, and the DUSOI, which weight different conditions, would be more effective at predicting outcomes than simple counts, which weight all conditions equally. Some studies, however, have concluded that simple measures, such as a simple count of chronic diseases or of prescribed medications, are almost as effective at predicting mortality and health care utilization as more sophisticated methods and may be much simpler (and also less expensive) to use despite the reservations outlined above.^47,50

Part of the problem in choosing an appropriate measure is due to the abstract nature of the concept of multimorbidity and how it relates to other concepts, such as disease burden and patient complexity.¹ It is important that measures are based on an underlying conceptualization of why and how multimorbidity is expected to have an impact on other variables. For example, the impact of multimorbidity on quality of life is likely to be most appropriately assessed using a self-report measure that takes account of functional ability,⁴¹ whereas the impact on health care utilization is likely to be best assessed using a measure that was derived using empirical weights to predict this outcome.^64,65

Relatively few studies have directly compared the performance of different measures in a primary care context, and the findings do not show the clear superiority of one measure over another. Evidence about the reliability of these measures when used in a primary care or population setting is also limited. Evidence about the reliability of measures when used in hospitalized patients and specialist secondary care settings⁹ may not necessarily pertain to primary care settings, where patient characteristics, disease classifications, record systems, and staffing are very different.

Strengths and Limitations

This article builds on previous reviews of comorbidity measures in the context of specific index diseases^9,66,67 by assessing the use of multimorbidity measures in generalist primary care and population settings. Multimorbidity is not well indexed in the literature, so it is unlikely that we have found all studies that would fit our inclusion criteria. We are aware that a number of risk adjustment models have been developed within the US health insurance system which have not been used frequently within academic research.⁶⁸ Included studies used a variety of methods, and we have not set out to assess individual study quality. The methods used to derive each measure also differ considerably; therefore, comparing measures directly is fraught with both the inherent biases in the original studies plus the potential biases introduced by a systematic review, especially one of observational studies that have used different study designs. In some cases it was debatable whether the setting of a study should be considered as primary care; we resolved such issues through discussion. We are confident that our review reflects the range and application of multimorbidity measures in the primary care and population context.

Implications

Different measures are needed to assess associations with different outcomes and the choice of measure will also depend on the type of data available. The measures that have been most widely used and for which there is greatest evidence of validity are the Charlson index, disease counts, and the ACG System. Other measures such as the CIRS and the DUSOI are more complex to administer, and their advantages over easier methods have not been well established. Measures based on counts of prescribed medication appear promising but need further research.