Original Article
Cumulative Illness Rating Scale was a reliable and valid index in a family practice context

https://doi.org/10.1016/j.jclinepi.2004.10.017Get rights and content

Abstract

Background and Objective

The goal of this study was to validate an instrument measuring the clinical burden of several medical problems in the same patient (multimorbidity), in a family practice context and, more specifically, to verify if trained nurses can score the Cumulative Illness Rating Scale (CIRS) from chart review.

Study Design and Setting

A convenience sample of 40 patients was selected. The attending physicians scored the CIRS during clinical interview (CIRS-MD/I), then three nurses scored the CIRS during clinical interview (CIRS-NUR/I) and three other nurses scored the CIRS from chart review (CIRS-NUR/C) (interrater reliability). Two of these nurses scored the CIRS-NUR/C again 2 months later (intrarater reliability).

Results

For interrater reliability, the intraclass correlation coefficients were 0.81 (0.70–0.89) for the CIRS-NUR/I and 0.78 (0.66–0.87) for the CIRS-NUR/C. The intrarater reliability of the CIRS-NUR/C was 0.89 (0.80–0.94) for one of the nurses and 0.80 (0.65–0.89) for the other. Concomitant validity of these two forms of CIRS with the CIRS-MD/I ranged from 0.73 to 0.84.

Conclusion

The CIRS appears to be a reliable and valid instrument in a primary care context and trained nurses can score the CIRS from chart review.

Introduction

One of the family physician's main roles is to follow patients over the long term [1]. He or she often treats patients with multiple but not necessarily related medical conditions, termed multimorbidity. This is a frequent problem in primary care [2], [3], [4], and one that has a major impact on both the health care system and the patients themselves, such as an increase in the length and frequency of office visits and hospitalizations [5], [6], [7], [8], [9], [10] and in mortality [7], [11], [12].

Most authors have defined multimorbidity as the presence of more than one chronic condition [2], [13]. It differs from comorbidity in that it views health problems from an overall perspective, rather than considering them in relation to an index disease. A multimorbidity scale that takes into account the number of medical problems and weights them according to their severity would generate a more accurate assessment of the patient's health. The importance of identifying and validating such a tool is that it could be used for quantitative assessments of multimorbidity in family practice research. It could also be useful in a clinical setting (e.g., to plan the duration of appointments based on the clinical condition of these types of patients).

One of the existing tools used to measure multimorbidity is the Cumulative Illness Rating Scale (CIRS), which considers all medical problems encountered in primary care. In the original version developed by Linn et al. [14], the gravity of co-occurring medical conditions was weighted from 0 to 4 for the 13 main systems.

We used the 14-system modified version of Miller et al. [15], with the hematopoietic system considered separately from the vascular system, because of the availability of a scoring manual [16] developed for a geriatric population but appropriate for a family practice clientele. The organs or systems in this modified version of the CIRS are (1) cardiac; (2) vascular; (3) hematological; (4) respiratory; (5) otorhinolaryngological, ophthalmological; (6) upper gastrointestinal; (7) lower gastrointestinal; (8) hepatic and pancreatic; (9) renal; (10); genitourinary; (11) musculoskeletal, tegumental; (12) neurological; (13) endocrine, metabolic, breast; and (14) psychiatric. The total theoretical score ranges from 0 to 56, based on scoring from 0 to 4 as follows: 0, no problem; 1, minor current problem or significant history; 2, morbidity or moderate discomfort, requiring primary care treatment; 3, severe problem: constant significant discomfort, chronic problem difficult to control; 4, extremely severe problem, requiring immediate treatment: organ failure or severe functional impairment.

The CIRS was initially developed to measure that chronic medical illness burden which is, in fact, multimorbidity [14], [15]. Some authors have used the CIRS to assess comorbidity [9], [17]; these studies were also included in the literature review of the metrological properties of the CIRS.

A number of studies [9], [14], [15], [17], [18] have documented good interrater reliability of the CIRS, with correlation coefficients ranging from 0.55 to 0.91. One study [17] done with a group of geriatric patients suffering from neoplasia documented very good intrarater reliability (intraclass correlation coefficient ICC = 0.91–0.99). Some studies also documented good predictive validity for mortality [9], [18], [19], [20] and autopsy results [21]. One study showed a significant association with the risk of hospitalization in the following year [20]. Another study reported good discriminant validity [15]. Convergent validity has been studied by several authors, some of whom documented a moderate correlation with functional status [15], [18], [22]. Other authors found weaker correlations between these two entities [17], [20].

All the studies we found on the reliability and validity of the CIRS were conducted in geriatric or specialized care contexts (psychiatry, neurosurgery, oncology, etc.), none in a primary care ambulatory context. Also, although Linn's and Miller's versions were designed to be based on a clinical assessment, some studies were based exclusively on chart reviews [9], [17], [18], because this approach is more practical. Therefore, our primary objectives with the present study were to verify the interrater and intrarater reliability of the CIRS completed by nurses from chart review (CIRS-NUR/C) and to compare the interrater reliability of the CIRS-NUR/C with that of the CIRS completed by nurses during clinical interview (CIRS-NUR/I) in an ambulatory family practice context. The secondary objective was to document the concomitant validity of the two forms of CIRS completed by nurses (CIRS-NUR/C and CIRS-NUR/I) by comparing them with the CIRS completed by the attending physician during clinical interview (CIRS-MD/I), which we considered the gold standard measure in this study.

Section snippets

Study participants

Three family physicians practicing at the Family Medicine Unit of the Sagamie Hospital Center in Chicoutimi, Province of Quebec, Canada, were selected conveniently by the principal investigator to participate in the study. They were clinicians of more than 15 years of experience who followed different patients of any age and who took a relatively standard approach to maintaining their medical records accordingly to the judgment of the main investigator (e.g., legibility, note's length). The

Results

The subjects who agreed and refused to participate in the study were comparable in terms of age and gender (Table 1). Other variables could not be compared because the subjects who did not participate in the study did not sign the consent form. The three physicians who participated in the study followed 11, 15, and 14, respectively, of the 40 patients in the study. Figure 2 shows the distribution of the subjects' scores on the CIRS-MD/I, which followed a normal curve (P = .86).

The ICC was 0.81

Discussion

In this study, the interrater reliability of the CIRS-NUR/C was acceptable and comparable to that of the CIRS-NUR/I. These results corroborate the coefficients obtained in specialized contexts (range: 0.55–0.91) [9], [14], [15], [17], [18]. Training and standardization were deliberately minimized to ensure that the training time required was reasonable. The reliability of the CIRS-NUR/C might be improved, however, by lengthening the period during which nurses can practice to complete CIRS and

Conclusion

This study indicates that the CIRS-NUR/C has acceptable interrater reliability that is comparable to that for the CIRS-NUR/I, and good intrarater reliability 2 months later in an ambulatory family practice context. These results could be improved with better prior training. Concomitant validity compared to the CIRS-MD/I is acceptable. Thus, trained nurses can complete the CIRS acceptably from the medical record without interviewing the subject. A study is being conducted to evaluate other types

Acknowledgments

This study was made possible by grants from the Canadian College of Family Physicians and the Family Medicine Department at the University of Sherbrooke.

References (25)

  • P.A. Rochon et al.

    Comorbid illness is associated with survival and length of hospital stay in patients with chronic disability: a prospective comparison of three comorbidity indices

    Med Care

    (1996)
  • L.I. Iezoni et al.

    Comorbidities, complications and coding bias: does the number of diagnosis codes matter in predicting in-hospital mortality?

    JAMA

    (1992)
  • Cited by (171)

    View all citing articles on Scopus
    View full text