Original articlesFurther Evidence Supporting an SEM-Based Criterion for Identifying Meaningful Intra-Individual Changes in Health-Related Quality of Life
Introduction
Health-related quality of life (HRQoL) instruments, both disease-specific and generic, must be reliable, valid, and sensitive to change [1]. Ideally, an HRQoL instrument also needs established standards for identifying clinically important change for each patient population in which it is used [2]. A minimal clinically important difference (MCID) is defined by Jaeschke et al. [3] as “the smallest difference in a score of a domain of interest that patients perceive to be beneficial and that would mandate, in the absence of troublesome side effects and excessive costs, a change in the patient's management.” Such clinically important change, which recognizes the patient's perceptions as well as the clinical aspects of case management and outcomes, can only be assessed by practitioners having extensive experience with both the patient population and the assessment instrument [4], and individual patients. It is such clinically important change that reliable, valid, and sensitive health status measures must be able to detect 5, 6.
Using a longitudinal study of 563 outpatients with a history of cardiac illness who were interviewed on both a disease-specific HRQoL measure, the Chronic Heart Disease Questionnaire (CHQ) [7], and the Medical Outcomes Study 36-Item Short Form Health Survey (SF-36) [8], we recently found that a change equivalent to one standard error of measurement (SEM) approximated the established MCID standards for each CHQ dimension [9]. Furthermore, when applied to the SF-36 change scores of these same outpatients, the one-SEM criterion showed promise in scales with acceptable reliability and variability by reflecting similar percentages of individuals who improved, remained stable, or declined as indicated by the CHQ dimensions.
The SEM is the standard error in an observed score related to measuring with a particular test that obscures the true score. It is estimated by the standard deviation of the instrument multiplied by the square root of one minus its reliability coefficient [10], or
Through distributional (σx) and reliability (rxx) components, the SEM takes into consideration the possibility that some of the observed change may be do to random measurement error [11]. Furthermore, according to classical test theory, the SEM possesses the unique attribute of being sample-independent. That is, the SEM is relatively constant across all but the extremes (top and bottom 10%) of a given population's ability levels [12]. Ironically, this is due to the simultaneous incorporation of the two sample-dependent statistics, σx and rxx in the SEM's estimation formula. In samples drawn from a given population, these components vary together around a fixed SEM as reflected in this rearrangement of the SEM formula:
In addition to its population invariance, a second valuable quality of the SEM is that it is expressed in the original metric of a measure, providing ease in interpretation [10].
In the present study, we explored a SEM-based criterion for evaluating clinically important individual change on the Chronic Respiratory Disease Questionnaire (CRQ) [13] and the SF-36 among adult outpatients with a history of chronic obstructive pulmonary disease (COPD). The study population completed baseline and follow-up interviews on both instruments. Because we wanted to identify a statistical marker of clinically relevant change in reliable and valid dimensions, we appraised the psychometric properties of each of these instruments in our patient population at baseline. By using MCID standards that have been established for the CRQ [3], the number of SEMs that an individual's score must change to be deemed clinically relevant could be determined. The SEM criterion was validated by cross-classifying trichotomous categorizations of the patient's change scores (improved, stable, or declined) on the SEM criterion and the MCID standard for 393 outpatients with follow-up interviews. This SEM criterion was then applied to the eight scales of the SF-36 to identify the percentage of individual outpatients who improved, remained stable, or declined. These SF-36 change percentages were compared to those of the CRQ to explore the potential for a SEM-based change criterion for the SF-36, although MCID standards have not been established for those scales [14].
Section snippets
Sampling
This study is a secondary data analysis of baseline and follow-up telephone interviews of outpatients who attended the general medicine clinics of a large academic medical center and participated in a randomized controlled trial (RCT) assessing the effects of computerized reminders to physicians and pharmacists regarding drug utilization review (DUR). The Regenstrief Medical Record System (RMRS) 15, 16, an electronic medical chart, identified outpatients as eligible for inclusion if they had a
Baseline Descriptive Data
As indicated, 487 COPD outpatients were enrolled at baseline. Although the interviews were considered complete because all items were asked of each outpatient, not every item was answered. There were 391 outpatients who had no missing items on the CRQ, 392 who had no missing items on the SF-36, and 316 (65%) who had completely answered all items on both instruments at baseline. The number of usable interviews was increased by applying to both instruments the standard prorated imputation methods
The SEM and MCID Linkage
Previous research on the MCID in each CRQ dimension has concluded that an average per item change of 0.5 in each dimension is a sufficient and straightforward criterion for indicating clinically meaningful change [3]. In these data, the CRQ baseline dimensional raw score analog of a one-SEM change divided by the number of items in the target dimension was 0.50 for dyspnea, 0.54 for fatigue, and 0.46 for emotional function. These results are equivalent to those found in our previous study,
Discussion
This study adds to the evidence supporting the contention that a one-SEM criterion can be used to identify meaningful intra-individual change in HRQoL instruments. In all three CRQ dimensions with established MCID standards, one SEM identified clinically relevant change. Furthermore, when we applied the one-SEM criterion to the SF-36, those scales with acceptable psychometric properties (minimal floor to ceiling effects, reasonable variance, and acceptable internal consistency) showed similar
Acknowledgements
This research was supported by grants to Dr. Wolinsky from the National Institutes of Health (R37 AG-09692) and to Dr. Tierney from the Agency for Health Care Policy Research (R01 HS-07763). An earlier version of results in this paper were presented at the International Society for Quality of Life Research Annual Conference on November 17, 1998. The opinions expressed here are those of the authors and do not necessarily reflect those of the funding agencies, academic, research, or governmental
References (37)
- et al.
Measuring health statusWhat are the necessary measurement properties?
J Clin Epidemiol
(1992) - et al.
Determining a minimal important change in a disease-specific quality of life questionnaire
J Clin Epidemiol
(1994) - et al.
Psychotherapy outcome researchMethods for reporting variability and evaluating clinical significance
Behav Ther
(1984) - et al.
Quality of life in patients with chronic airflow limitations
Br J Dis Chest
(1987) - et al.
Assessing the minimal important difference in symptomsA comparison of two techniques
J Clin Epidemiol
(1996) The minimal important differenceWho's to say what is important?
J Clin Epidemiol
(1996)- et al.
Evaluation of the MOS SF-36 Physical Functioning Scale (PF-10)II. Comparison of relative precision using Likert and Rasch scoring methods
J Clin Epidemiol
(1997) - et al.
Methodological problems in the retrospective computation of responsiveness to changeThe lessons of Cronbach
J Clin Epidemiol
(1997) - et al.
Measurement of health statusAscertaining the minimal clinically important difference
Con Clin Trial
(1989) Evaluating measurement responsiveness
J Rheum
(1995)