Original articles
Further Evidence Supporting an SEM-Based Criterion for Identifying Meaningful Intra-Individual Changes in Health-Related Quality of Life

https://doi.org/10.1016/S0895-4356(99)00071-2Get rights and content

Abstract

This study used the standard error of measurement (SEM) to evaluate intra-individual change on both the Chronic Respiratory Disease Questionnaire (CRQ) and the SF-36. After analyzing the reliability and validity of both instruments at baseline among 471 COPD outpatients, the SEM was compared to established minimal clinically important difference (MCID) standards for three CRQ dimensions. A value of one SEM closely approximated the MCID standards for all CRQ dimensions. This SEM-based criterion was then validated by cross-classifying the change status (improved, stable, or declined) of 393 follow-up outpatients using the one-SEM criterion and the MCID standard. Excellent agreement was achieved for all three CRQ dimensions. Although MCID standards have not been established for the SF-36, the one-SEM criterion was explored in these change scores. Among SF-36 scales demonstrating acceptable reliability and reasonable variance, the percent of individuals within each change category was consistent with those seen in the CRQ dimensions. These results replicate previous findings where a value of one SEM also closely approximated MCIDs for all dimensions of the Chronic Heart Disease Questionnaire among cardiovascular outpatients. The one-SEM criterion should be explored in other health-related quality of life instruments with established MCIDs.

Introduction

Health-related quality of life (HRQoL) instruments, both disease-specific and generic, must be reliable, valid, and sensitive to change [1]. Ideally, an HRQoL instrument also needs established standards for identifying clinically important change for each patient population in which it is used [2]. A minimal clinically important difference (MCID) is defined by Jaeschke et al. [3] as “the smallest difference in a score of a domain of interest that patients perceive to be beneficial and that would mandate, in the absence of troublesome side effects and excessive costs, a change in the patient's management.” Such clinically important change, which recognizes the patient's perceptions as well as the clinical aspects of case management and outcomes, can only be assessed by practitioners having extensive experience with both the patient population and the assessment instrument [4], and individual patients. It is such clinically important change that reliable, valid, and sensitive health status measures must be able to detect 5, 6.

Using a longitudinal study of 563 outpatients with a history of cardiac illness who were interviewed on both a disease-specific HRQoL measure, the Chronic Heart Disease Questionnaire (CHQ) [7], and the Medical Outcomes Study 36-Item Short Form Health Survey (SF-36) [8], we recently found that a change equivalent to one standard error of measurement (SEM) approximated the established MCID standards for each CHQ dimension [9]. Furthermore, when applied to the SF-36 change scores of these same outpatients, the one-SEM criterion showed promise in scales with acceptable reliability and variability by reflecting similar percentages of individuals who improved, remained stable, or declined as indicated by the CHQ dimensions.

The SEM is the standard error in an observed score related to measuring with a particular test that obscures the true score. It is estimated by the standard deviation of the instrument multiplied by the square root of one minus its reliability coefficient [10], or σx1−rxx.

Through distributional (σx) and reliability (rxx) components, the SEM takes into consideration the possibility that some of the observed change may be do to random measurement error [11]. Furthermore, according to classical test theory, the SEM possesses the unique attribute of being sample-independent. That is, the SEM is relatively constant across all but the extremes (top and bottom 10%) of a given population's ability levels [12]. Ironically, this is due to the simultaneous incorporation of the two sample-dependent statistics, σx and rxx in the SEM's estimation formula. In samples drawn from a given population, these components vary together around a fixed SEM as reflected in this rearrangement of the SEM formula: rxx=1−SEM2σx2.

In addition to its population invariance, a second valuable quality of the SEM is that it is expressed in the original metric of a measure, providing ease in interpretation [10].

In the present study, we explored a SEM-based criterion for evaluating clinically important individual change on the Chronic Respiratory Disease Questionnaire (CRQ) [13] and the SF-36 among adult outpatients with a history of chronic obstructive pulmonary disease (COPD). The study population completed baseline and follow-up interviews on both instruments. Because we wanted to identify a statistical marker of clinically relevant change in reliable and valid dimensions, we appraised the psychometric properties of each of these instruments in our patient population at baseline. By using MCID standards that have been established for the CRQ [3], the number of SEMs that an individual's score must change to be deemed clinically relevant could be determined. The SEM criterion was validated by cross-classifying trichotomous categorizations of the patient's change scores (improved, stable, or declined) on the SEM criterion and the MCID standard for 393 outpatients with follow-up interviews. This SEM criterion was then applied to the eight scales of the SF-36 to identify the percentage of individual outpatients who improved, remained stable, or declined. These SF-36 change percentages were compared to those of the CRQ to explore the potential for a SEM-based change criterion for the SF-36, although MCID standards have not been established for those scales [14].

Section snippets

Sampling

This study is a secondary data analysis of baseline and follow-up telephone interviews of outpatients who attended the general medicine clinics of a large academic medical center and participated in a randomized controlled trial (RCT) assessing the effects of computerized reminders to physicians and pharmacists regarding drug utilization review (DUR). The Regenstrief Medical Record System (RMRS) 15, 16, an electronic medical chart, identified outpatients as eligible for inclusion if they had a

Baseline Descriptive Data

As indicated, 487 COPD outpatients were enrolled at baseline. Although the interviews were considered complete because all items were asked of each outpatient, not every item was answered. There were 391 outpatients who had no missing items on the CRQ, 392 who had no missing items on the SF-36, and 316 (65%) who had completely answered all items on both instruments at baseline. The number of usable interviews was increased by applying to both instruments the standard prorated imputation methods

The SEM and MCID Linkage

Previous research on the MCID in each CRQ dimension has concluded that an average per item change of 0.5 in each dimension is a sufficient and straightforward criterion for indicating clinically meaningful change [3]. In these data, the CRQ baseline dimensional raw score analog of a one-SEM change divided by the number of items in the target dimension was 0.50 for dyspnea, 0.54 for fatigue, and 0.46 for emotional function. These results are equivalent to those found in our previous study,

Discussion

This study adds to the evidence supporting the contention that a one-SEM criterion can be used to identify meaningful intra-individual change in HRQoL instruments. In all three CRQ dimensions with established MCID standards, one SEM identified clinically relevant change. Furthermore, when we applied the one-SEM criterion to the SF-36, those scales with acceptable psychometric properties (minimal floor to ceiling effects, reasonable variance, and acceptable internal consistency) showed similar

Acknowledgements

This research was supported by grants to Dr. Wolinsky from the National Institutes of Health (R37 AG-09692) and to Dr. Tierney from the Agency for Health Care Policy Research (R01 HS-07763). An earlier version of results in this paper were presented at the International Society for Quality of Life Research Annual Conference on November 17, 1998. The opinions expressed here are those of the authors and do not necessarily reflect those of the funding agencies, academic, research, or governmental

References (37)

  • A. Feinstein

    Benefits and obstacles for development of health status assessment measures in clinical settings

    Med Care

    (1992)
  • S. Their

    Forces motivating the use of health status assessment measures in clinical settings and related clinical research

    Med Care

    (1992)
  • G. Guyatt et al.

    Development and testing a new measure of health status for clinical trials in heart failure

    J Gen Intern Med

    (1989)
  • J. Ware et al.

    The MOS 36-item short-form health survey (SF-36)I. Conceptual framework and item selection

    Med Care

    (1992)
  • K. Wyrwich et al.

    Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life

    Med Care

    (1999)
  • A. Anastasi et al.

    Psychological Testing

    (1997)
  • J. Nunnally et al.

    Psychometric Theory

    (1994)
  • G. Guyatt et al.

    A measure of quality of life for clinical trials in chronic lung disease

    Thorax

    (1987)
  • Cited by (0)

    View full text