Abstract
PURPOSE The Assessment of Burden of Chronic Conditions (ABCC) tool was developed to improve care by facilitating shared decision making and self-management. It assesses and visualizes the experienced burden of 1 or multiple chronic conditions and integrates it in daily care. The aim of this study is to evaluate whether the ABCC scale is valid and reliable in people with chronic obstructive pulmonary disease (COPD), asthma, or type 2 diabetes (T2D).
METHODS The Saint George Respiratory Questionnaire (SGRQ), the Standardized Asthma Quality of Life Questionnaire (AQLQ-S), and the Audit of Diabetes Dependent Quality of Life Questionnaire (ADDQoL19) were compared with the ABCC scale to assess convergent validity. The internal consistency was evaluated using Cronbach’s α. Test-retest reliability was evaluated at a 2-week interval.
RESULTS A total of 65 people with COPD, 62 with asthma, and 60 with T2D were included. The ABCC scale correlated, in accordance with hypotheses, with the SGRQ (75% of correlations ≥0.7), AQLQ-S (100%), and ADDQoL19 (75%). The ABCC scale was internally consistent with a Cronbach’s α of 0.90, 0.92, and 0.91 for the total score for people with COPD, asthma, and T2D, respectively. The ABCC scale had a good test-retest reliability with an intraclass correlation coefficient of 0.95, 0.93, and 0.95 for people with COPD, asthma, and T2D, respectively.
CONCLUSIONS The ABCC scale is a valid and reliable questionnaire that can be used within the ABCC tool for people with COPD, asthma, or T2D. Future research should indicate whether this applies to people with multimorbidity, and what the effects and experiences are upon clinical use.
- asthma
- diabetes mellitus
- patient reported outcome measures
- pulmonary disease, chronic obstructive
- vaildation study
INTRODUCTION
Chronic conditions impose an enormous impact on health care in general and especially on the people living with them.1,2 Effective disease management is essential in care for people with chronic conditions. A key element in disease management is self-management, which starts with a patient’s insight into their experience with the burden of disease.3,4 Burden of disease can be defined as “a reflection of the impact of disease, which is suffering due to symptom severity (intensity, frequency, duration); functioning (occupational, social, and leisure activities); and quality of life (patients’ satisfaction with health, occupational, social, and leisure activities).”5 To measure the burden of disease, patient-reported outcome measures (PROMs) can be used. Currently, most questionnaires in clinical practice fail to include the full scope of burden of disease, but rather focus only on quality of life (QoL). These QoL questionnaires are either fully generic or disease specific. To have actual impact on a person’s burden of disease, PROMs should also function as the starting point for conversation about a personalized care plan.
The Assessment of Burden of Chronic Conditions (ABCC) tool was developed to measure burden of disease and to facilitate shared decision making, self-management, patient-health care, communication about experienced burden, and burden-guided care plans.5 The ABCC tool is used during clinical consultations, and consists of the following steps: (1) assessing experienced burden with a short scale (ie, the ABCC scale); (2) visualizing with a representation of the outcomes in a comprehensible balloon chart (Figure 1); (3) having a shared decision-making conversation between patient and health care clinician supported by treatment advice that is presented by clicking on 1 or more balloons; and (4) formulating personalized care goals.
Visual representation of the ABCC scale outcomes.
ABCC = Assessment of Burden of Chronic Conditions; BMI = body mass index; DM = diabetes mellitus; Hypo = hypoglycemia.
The outcomes of the ABCC scale are visualized into a balloon chart. High green balloons indicate low burden. Low red balloons indicate high burden. Orange and yellow balloons indicate changes between red and green. The results from the previous visit are depicted in gray for easy monitoring.
The ABCC tool has several characteristics. First, it includes the full scope of the burden of disease.5 Second, it combines the advantages of both generic and disease-specific questionnaires (ie, it can be used in cases of multimorbidity and is able to detect and provide detail of specific symptom and disease-related changes over time). Third, it visualizes the outcomes in a comprehensible manner and provides direction for shared decision making. These attributes allow for incorporation of the tool into a clinical setting. The ABCC tool is currently developed for people with chronic obstructive pulmonary disease (COPD), asthma, and type 2 diabetes (T2D) and is designed for expansion to other chronic conditions (Supplemental Appendixes 1, 2). However, assessing the psychometric properties of the ABCC scale is a necessary element before bringing the ABCC tool to clinical practice. Therefore, in this paper, the validity and reliability of the ABCC scale (step 1) are assessed. The aim of this study is to evaluate the ABCC scale’s convergent construct validly, known group validity, internal consistency, and test-retest reliability in people with COPD, asthma, or T2D.
METHODS
A cross-sectional questionnaire study was conducted in the Netherlands from April 2019 through March 2020 and reported using the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) guidelines.6 The developmental process of the questionnaire is described elsewhere.5 The Medical Ethics Committee of Zuyderland Hospital, Heerlen, approved the study (METCZ20180131). All participants provided written informed consent before participation.
Outcomes
To assess the validity and reliability of the ABCC tool, this study focused on evaluating (1) convergent construct validity by means of hypothesis testing, (2) the scale’s ability to differentiate between known groups, (3) the internal consistency of the total score and its multi-item domains, and (4) the scale’s test-retest reliability.
Participant Selection and Recruitment
Participants were eligible if they self-reported a diagnosis of either COPD, asthma, or T2D. Additional inclusion criteria were aged over 18 years and being able to read and understand Dutch. Participants were excluded if they had a pulmonary episode within 6 weeks before study onset (COPD or asthma) or had been diagnosed with T2D within 3 months before study onset. To aid in recruitment, all participants were incentivized with the possibility of winning 1 of 10 gift cards with a value of 10 euro through random selection.
The patient organization Lung Foundation Netherlands was used to recruit participants with asthma or COPD. The Lung Foundation Netherlands uses a forum for patients to be updated about novel developments relevant to people with pulmonary disease. Researchers can collaborate with the Lung Foundation Netherlands to inform and recruit people with specific pulmonary conditions about current research projects.
Participants with T2D were recruited by the patient organization Dutch Diabetes Association, who use a forum similar to the Lung Foundation Netherlands. Because the required sample size was not met for people with T2D after a first call, additional recruitment strategies were deployed. A newsletter was sent by the Dutch Institute for Rational Use of Medicine (who serve a large population of people with T2D that are often prescribed medical treatment). Also, recruitment posters were placed in waiting rooms of 8 general practices and 3 internal medicine departments throughout the Netherlands. Response rates could not be calculated for these passive recruitment strategies.
Sample Size
Recommendations for validation studies state that the participant-to-item ratio should be between 2 and 20 participants per item.7 The ABCC tool consists of 3 scales with 15 items for COPD, 16 items for asthma, and 14 items for T2D. Considering the numbers of items and a participant-to-item ratio of about 4, the sample size needed was estimated at 60 participants per scale for each chronic condition. Based on this sample size we calculated the effect sizes that could be detected with 80% power and a 2-sided significance level α of 0.05 to reflect on the adequacy of the sample size. PASS version 19.0.9 (NCSS Statistical Software) was used for all calculations. In short, with this sample size we could detect Pearson correlation coefficient of 0.345, standardized effect sizes of 0.74, Cronbach’s α of 0.466, and intraclass coefficients of 0.31 (see Supplemental Appendix 3 for a detailed explanation).
Data Collection
All participants completed a self-administered paper questionnaire at home, which included baseline characteristics, the ABCC scale, and a disease-specific set of questionnaires for inclusion at baseline. All questionnaires were sent through postal services, which included free return of the completed questionnaires. The researchers were not present during completion of the questionnaires. Baseline characteristics included: sex, age, level of education, time since diagnosis, smoking, treating physician, exacerbations in the previous year, and prescribed medication. The ABCC scale measures experienced burden. Experienced burden of disease is the impact of a chronic condition on a person’s life in terms of symptom severity, functioning, and quality of life (QoL).8
Experienced burden is rarely fully evaluated, more commonly, QoL is assessed, which is only part of the experienced burden.8,9 In the absence of measures that evaluate experienced burden, for the analyses of convergent validity, the ABCC scale will be compared with commonly used QoL measures. For people with COPD this included the Saint George Respiratory Questionnaire (SGRQ).10 For people with asthma, this included the Standardized Asthma Quality of Life Questionnaire (AQLQ-S).11 For people with T2D, this included the Audit of Diabetes-Dependent Quality of Life (ADDQoL19).12 Additionally, the Hospital Anxiety and Depression Scale (HADS) was completed by people with COPD to assess known-group validity.13 Detailed information about these questionnaires is presented in Supplemental Appendix 3.
Two weeks after completing the first set of questionnaires, all participants completed the ABCC scale again, with an additional question about whether their health status had changed since baseline (ie, worse, the same, or better). Permission to use the questionnaires was obtained from the developers of each questionnaire.
Data Analysis
An overview of the outcome parameters of this study is presented in Table 1. Validity was evaluated based on the assessment of convergence with a questionnaire measuring a related construct, and the questionnaire’s ability to differentiate between known clinical groups. Either t-tests or Mann Whitney U tests were used to evaluate validity based on whether the data were approximately normally distributed (histogram and QQ-plot). Convergent validity was implied if at least 75% of the absolute value of the postulated Pearson correlation coefficient was higher than 0.7 for the total score or multi-item subscales or between 0.3 and 0.7 for single-item subscales.16 For single-item domains, the threshold for validity was between 0.3 and 0.7 as only moderate correlation coefficients can be expected from single-item correlations. As the ABCC scale implies a low burden at low scores and both the AQLQ and ADDQoL imply a low burden (or high QoL) at high scores, these correlation coefficients are expected to be negative. The SGRQ for COPD, AQLQ-S for asthma, and ADDQoL for T2D were used as comparator questionnaires to evaluate the convergent validity of the ABCC scale.
Outcomes for Each Subgroup
To assess the discriminative properties of the ABCC scale for known groups of people with COPD, 2 pairs, characterized by either exacerbation status (<2 vs ≥2 exacerbations in the past year)17-21 or the HADS depression subscale (depression score <8 vs ≥8),22,23 were compared. To check the discriminative properties of the ABCC scale for known groups of people with asthma, 2 pairs, characterized by either exacerbation status (0 vs ≥1 exacerbation in the past year)5 or asthma control status according to the Global Initiative for Asthma (GINA; controlled vs uncontrolled),24-27 were compared. To check the discriminative properties of the ABCC scale for known groups of people with T2D, 3 pairs, characterized by insulin use (insulin-independent vs insulin-dependent),28-30 presence of complications (no complications vs the presence of any the conditions: nephropathy, neuropathy, retinopathy, sexual dysfunction, amputation of any limb, diabetic foot, or cardiovascular disease),31-34 or obesity (body mass index <30 vs ≥30),28,29,31,32,34,35 were compared.
Domains that were hypothesized to differentiate between known groups are presented in Table 1. A Cronbach’s α of ≥0.90 for the total scale or α ≥0.70 for subscales with multiple items was maintained as cut-off point for determining adequate internal consistency.36,37 Test-retest reliability was evaluated for those subjects who had an unchanged self-reported health status at 2 weeks after baseline. An intraclass correlation coefficient (ICC) of 0.90 was considered acceptable for evaluating the ABCC scale as sufficiently reproducible.16,38 Statistical significance was credited to P ≤.05. All statistical analyses were performed using IBM SPSS version 25.0 (IBM Corp).
RESULTS
A total of 196 people gave informed consent: 67 with COPD, 64 with asthma, and 65 with T2D. Telephone contact was established to gather the self-reported diagnoses. During these contacts, 2 people with COPD, 2 with asthma, and 5 with T2D withdrew their consent for participation (due to illness or absence). A total of 65 people with COPD, 62 with asthma, and 60 with T2D were included in the study. One participant with asthma and 1 with T2D were lost to follow-up between baseline and 2 week follow up.
The baseline characteristics of the study population are presented in Table 2. Between baseline and 2 week follow up, 5 people with COPD, 11 people with asthma, and 6 people with T2D indicated a changed health status and were excluded from test-retest analyses. An additional list of participant characteristics is presented in Supplemental Appendix 4. Outcomes on the various questionnaires are presented in Supplemental Appendix 5.
Baseline Characteristics for Each Subgroup
Validity and Reliability of the ABCC scale
People With COPD
The correlation between the total score of the ABCC scale for people with COPD and the SGRQ total score exceeded the threshold for validity (0.7) as it was 0.866. This was also true for all subscales (Table 3). The domains of physical limitations and pulmonary complaints of the ABCC scale correlated with the SGRQ total score (r = 0.829 and r = 0.761, respectively). Out of the 12 postulated correlations, 9 were higher than 0.7, indicating that 75% of these hypotheses were met (Table 3). People with 2 or more exacerbations scored significantly higher on the ABCC scale total, as well as on the hypothesized domains. People with COPD with depression (indicated by HADS) scored significantly higher on the ABCC scale total, as well as on the fatigue, feelings and emotions, and relations and work domains. The Cronbach’s α of the ABCC scale total for people with COPD was 0.90. Domain scores had a Cronbach’s α of 0.92 for physical limitations, 0.77 for feelings and emotions, and 0.65 for pulmonary complaints. The ICC for the ABCC scale for people with COPD was 0.95.
Psychometric Properties of the ABCC Scale for People with COPD (n = 65)
People With Asthma
The correlation between the total score of the ABCC scale for people with asthma and the AQLQ-S total score exceeded the threshold for validity (0.7) as it was 0.851. This was also true for all subscales (Table 4). The physical limitations and asthma complaints domains of the ABCC scale correlated with the total scores (r = −0.777 and r = −0.835, respectively). All 10 correlations were lower than −0.7, indicating that 100% of the hypotheses were met (Table 4). People who had exacerbations and people with uncontrolled asthma scored significantly higher on the ABCC scale total, as well as on the hypothesized domains (Table 4). The Cronbach’s α of the ABCC scale total for people with asthma was 0.92. Domain scores had a Cronbach’s α of 0.88 for physical limitations, 0.74 for feelings and emotions, and 0.73 for asthma complaints. The ICC for the ABCC scale for people with asthma was 0.93.
Psychometric Properties of the ABCC Scale for People With Asthma (n = 62)
People With T2D
The total score of the ABCC scale for people with T2D correlated moderately (ie, −0.7 < r < −0.3) with the ADDQoL19 average weighted impact (r = −0.548) (Table 5). The ABCC domains correlated for each hypothesized comparison, except for the comparison between the ABCC domain eating and drinking and the ADDQoL19 item freedom to drink (Table 5). Out of the 12 postulated correlations, 9 were between −0.7 and −0.3, indicating that 75% of the hypotheses were met. People who were insulin dependent scored significantly higher on the ABCC scale total as well as on the hypothesized domains, except for the domain worry about future (Table 5). People with at least 1 complication scored significantly higher on the ABCC scale total, as well as on the hypothesized domains. People who were obese (body mass index ≥30) scored significantly higher on the ABCC scale total as well as for the hypothesized domains, except on the domain eating and drinking. The Cronbach’s α of the ABCC scale total for people with T2D was 0.91. Domain scores had a Cronbach’s α of 0.87 for physical limitations and 0.76 for feelings and emotions. The ICC for the ABCC scale for people with T2D was 0.95.
Psychometric Properties of the ABCC Scale for People With T2D (n = 60)
DISCUSSION
This study shows the ABCC scale to be a valid and reliable instrument for evaluating the experienced burden of disease for people with COPD, asthma, and T2D. First, the ABCC scale was shown to correlate in at least 75% of the postulated hypotheses, thereby confirming its construct validity. Second, in most cases, the ABCC scale was able to distinguish known groups of people with COPD, asthma, and T2D. Third, the ABCC scale has adequate internal consistency for the total score and multi-item domains (ie, physical limitations, feelings and emotions, and pulmonary or asthma complaints). Last, the ABCC scale was shown to have excellent test-retest reliability.
The results should be reviewed with several concepts and limitations in mind. First, recruitment efforts led to sample sizes ranging from 60 to 65 persons per condition which provided sufficient power to detect the outcomes in this study. Second, the ability to distinguish known groups from the literature adds to the relevance of the ABCC tool for clinical use. Third, the developmental process of the ABCC tool adhered closely to the clinical requirements of a brief tool that assesses relevant domains with the smallest number of items possible (often only 1 item per domain).5 This is different from the classical approach of creating a larger item bank and then reducing it. Our approach starts with a minimal number of items based on expert opinions (health care clinicians and patients) which are, if appropriate, clustered based on clinical usability. As this approach does not allow for items to be restructured, factor analyses would be inconsequential and was not performed. Fourth, this questionnaire and its validity and reliability are evaluated in Dutch language. For people to use the tool in different languages, thorough linguistic translation must be undertaken, including determining what constitutes burden of disease for people in countries and cultures different from the Netherlands. Fifth, recruitment bias may have occurred. Upon careful examination of the outcomes of all questionnaires, we concluded that the participants of this study experienced a low burden of disease. This may relate to the recruitment of people from patient advocate groups, who are generally well-educated and connected to patient organizations.40 The validity and reliability observed in our study may not hold true for populations that experience a high burden of disease. Additionally, a substantial proportion of participants with COPD or asthma received care from medical specialists, with a smaller portion receiving care from general practitioners. However, due to the relatively low average scores, it is expected that these results can be translated to a predominantly primary care population.
To our knowledge, this is the first study of a questionnaire that combines the experienced burden of disease for people with COPD, asthma, or T2D into a single questionnaire. The validity and reliability of the ABCC scale for these conditions separately justify investigation of its psychometric properties for people with multimorbidity. Additionally, in contrast to many other questionnaires, the ABCC scale largely consists of single-item domains. This means that it is suited for brief and efficient clinical application, where more robust questionnaires are too time consuming. The results of this study are in line with the results from the ABCC tool’s predecessor, the Assessment of Burden of COPD (ABC) tool.41 Although the content of the ABC scale was changed while developing the ABCC scale for multiple chronic conditions, the resulting domains are still valid. The results of this study justify the use of the ABCC scale within the ABCC tool for people with COPD, asthma, or T2D.
This study builds on the development of the ABCC tool and facilitates future research in several ways. The conversation is guided by the domain scores of the ABCC tool. In the current score calculation, all domains are assumed to be equally relevant to the total score. This may not be the case and should be studied, for example, by performing a discrete choice experiment.42,43 Furthermore, knowledge of the psychometric properties of the ABCC tool in the single conditions serves as a basis and a prerequisite to study its properties in people with multiple conditions. Lastly, to test the effectiveness of the ABCC tool and evaluate user experiences when employing the tool in clinical practice, further research should be performed.44
The ABCC scale is a brief self-administered questionnaire that measures the experienced burden of disease for people with COPD, asthma, or T2D. This study provides evidence for the validity and reliability of the ABCC scale in a Dutch population.
Acknowledgments
We would like to acknowledge the great help of the patient organizations Lung Foundation Netherlands and Diabetes Association Netherlands, and the Dutch Institute for Rational Use of Medicine, in recruiting participants. We also would like to thank all participants who completed the questionnaires. Additionally, we would like to express gratitude to Mascha Twellaar for her assistance in conducting this study.
Footnotes
Conflicts of interest: authors report none.
Read or post commentaries in response to this article.
Author contributions: D. C. and E. B. designed the questionnaire study. D. C., E. B., and L. K. conducted the data collection. D. C. and E. B. conducted the analyses. B. W. assisted with the analysis and interpretation of the data as a methodological and statistical expert. A. G. and O. S. supervised all phases of the study. D. C. wrote the first version of the manuscript. All authors have read and approved the final version of the manuscript.
Funding support: This study was funded by the Netherlands Organisation for Health Research and Development (104006001). The funding party had no role in study design, data collection, analysis, and interpretation, or manuscript writing.
- Received for publication March 29, 2022.
- Revision received September 19, 2022.
- Accepted for publication October 10, 2022.
- © 2023 Annals of Family Medicine, Inc.
