Abstract
PURPOSE Because dizziness in older people is often chronic and can substantially affect daily functioning, it is important to identify those at risk for an unfavorable course of dizziness to optimize their care. We aimed to develop and externally validate a prediction model for an unfavorable course of dizziness in older patients in primary care, and to construct an easy-to-use risk prediction tool.
METHODS We used data from 2 prospective cohorts: a development cohort with 203 patients aged 65 years or older who consulted their primary care physician for dizziness and had substantial dizziness-related impairment (Dizziness Handicap Inventory [DHI] ≥30), and a validation cohort with 415 patients aged 65 years or older who consulted their primary care physician for dizziness of any severity. An unfavorable course was defined as presence of substantial dizziness-related impairment (DHI ≥30) after 6 months.
RESULTS Prevalence of an unfavorable course of dizziness was 73.9% in the development cohort and 43.6% in the validation cohort. Predictors in the final model were the score on the screening version of the DHI, age, history of arrhythmia, and looking up as a provoking factor. The model showed good calibration and fair discrimination (area under the curve = 0.77). On external validation, discriminative ability remained stable (area under the curve = 0.78). The constructed risk score was strongly correlated with the prediction model. Performance measures for risk score cut-off values are presented to determine the optimal cut-off point for clinical practice.
CONCLUSIONS We developed an easy-to-use risk score for dizziness-related impairment in primary care. The risk score, consisting of only 4 predictors, will help primary care physicians identify patients at high risk for an unfavorable course of dizziness.
INTRODUCTION
One out of 10 older patients visits their primary care physician at least once a year because of dizziness.1,2 Dizziness strongly affects daily functioning in older adults3–6 and is associated with increased depressive symptoms, lower self-rated health, and reduced social activities.3,7 Furthermore, older dizzy people have an increased risk of falling.8
Dizziness can refer to several sensations including a giddy or rotational sensation, a loss of balance, a faint feeling, light-headedness, instability or unsteadiness, a tendency to fall, or a feeling of everything turning black.9 Dizziness is often divided into 4 subtypes: vertigo, presyncope, disequilibrium, and other dizziness.10 Defining the subtype might help the physician to get a grip on the cause. Yet, categorizing dizziness is difficult in older patients because many report more than 1 subtype.11–14
Most guidelines on dizziness promote a diagnosis-oriented approach, starting with a search for its cause followed by treatment once the underlying illness has been diagnosed.15,16 Dizziness mostly has a benign origin. Potential life-threatening illness, such as stroke or a brain tumor, is rare. If an accurate diagnosis of dizziness can be established, there is potential for effective treatment. Examples are the Epley maneuver for benign paroxysmal positional vertigo,17,18 and vestibular rehabilitation for vertigo persisting after the Epley maneuver.19 Dizziness in older people is, however, often a diagnostic challenge for physicians because it can refer to a variety of sensations, as there are many potential underlying causes, and there is often a multifactorial origin.9,13 As a result, primary care physicians often manage dizziness in this age-group with a wait-and-see strategy,2 although they could also focus on targeting potential contributing factors such as multiple drug use or depressive symptoms.9,13,20–27 Identification of patients with an unfavorable prognosis, even those with an unknown cause of dizziness, may contribute to better care, for example, by targeting potential contributing factors to dizziness in patients with an unfavorable dizziness prognosis.9,13,20–27
Only a single study has presented a prediction model to assess dizziness prognosis in older patients.24 Yet, the researchers choose to exclude the strongest predictor, dizziness-related impairment as measured with the Dizziness Handicap Inventory (DHI), from their final prediction model because the score does not indicate causal mechanisms and therefore does not provide clues for treatment.24 A prediction model to identify older patients at high risk for an unfavorable course of dizziness that includes the DHI score as predictor may have an impact on health care, as it helps to discriminate those in need of intervention. The aim of this study was to develop and externally validate a prediction model for an unfavorable course of dizziness in older people in primary care, with dizziness-related impairment as a starting point. Using this prediction model, we aimed to construct an easy-to-use risk score for identifying older adults at high risk for an unfavorable course of dizziness.
METHODS
Development Cohort
We collected data for model development in a prospective observational cohort study having a 6-month follow-up, as part of the Reduction Of Dizziness in older pEOple (RODEO) study. A detailed description of the study protocol has been published elsewhere.28
Participants were recruited from 45 primary care practices in the Netherlands between January 2015 and July 2016. Inclusion criteria were age 65 years or older, consultation of the primary care physician for dizziness in the preceding 3 months, and having substantial dizziness-related impairment (DHI score of 30 or higher).29 Patients with severe cognitive impairment, terminal illness, severe psychiatric problems, or inability to speak Dutch were excluded.
Sample size calculation was based on the commonly used minimum of 10 patients with the outcome per predictor30 and on our intention to develop a risk score with a maximum of 10 predictors. With the assumption that 60% of the patients probably would have an unfavorable outcome after 6 months of follow-up,14 we needed at least 170 participants.
Validation Cohort
To validate the prediction model in an external cohort, we used data from the Dizziness In Elderly Patients (DIEP) study.24 A total of 415 patients from 24 primary care practices in the Netherlands enrolled in this study between July 2006 and January 2008. Inclusion criteria were age 65 years or older and consulting the primary care physician because of dizziness. In contrast to the case for the development cohort, substantial dizziness-related impairment was not an inclusion criterion in this cohort. Exclusion criteria were inability to speak Dutch; severe cognitive, visual or hearing impairment; and wheelchair dependency.
Outcome Measure
The outcome, an unfavorable course of dizziness, was defined as substantial dizziness-related impairment at 6 months of follow-up. Dizziness-related impairment was assessed with the DHI.29 A DHI score of 30 or higher correlates with substantial dizziness-related impairment.24,29,31,32 The DHI is a widely used self-report questionnaire that is designed to quantify the impact of dizziness on everyday life. It consists of 25 items. The DHI score (range = 0 to 100) is obtained by summing ordinal scale responses—yes (4), sometimes (2), no (0)—with higher total score corresponding to more severe dizziness-related impairment. Previous studies have shown good construct validity, high internal consistency, and satisfactory test-retest reliability.29,32
Candidate Predictors
We aimed to develop an easy-to-use risk score with predictors that can readily be assessed during consultations in primary care. Selection of candidate predictors was therefore based on previous research and feasibility.9,13,14,20–27,33 Candidate predictors were self-reported and spanned 3 domains: demographic (6 variables), health status (28 variables), and dizziness (33 variables). Dizziness-related impairment at baseline was assessed with both the full 25-item DHI29 and the 10-item screening version of the DHI (DHI-S).34 Supplemental Appendix 1, http://www.annfammed.org/content/16/5/428/suppl/DC1, provides a complete overview of candidate predictors.
Model Development
We performed a univariate logistic regression analysis to assess the association between candidate predictors and outcome. Associated variables with P <.10 were included in a multivariate logistic regression model. Continuous variables were checked for linearity. Through backward selection, candidate predictors with P >.157 were excluded from the multivariate logistic regression model.35
We imputed missing data on variables having less than 5% missing values using single stochastic imputation.36 Multiple imputation using the chained equation method was performed when a variable was missing 5% of values or more, with 10 imputations in the case of 5% to 10% missing values and 15 imputations in the case of 10% to 15% missing values.36 Two deceased patients were excluded from the analysis.
The prediction model was developed in the combined multiple imputed data sets of the development cohort, applying the Rubin rules for pooling results.37 A prediction model with a dichotomous outcome is very useful in daily clinical practice because the outcome is easy to interpret for primary care physicians: a patient is either at high risk or at low risk for an unfavorable course of dizziness. Yet, converting a continuous outcome measure into a dichotomous outcome might result in information loss.38 We therefore developed an additional prediction model introducing dizziness-related impairment, also assessed with the DHI, as a continuous outcome. Other than performing linear regression analyses instead of logistic regression analyses, we applied the same methods as we did when constructing the original prediction model.
Model Performance
We assessed model performance by applying calibration and discrimination tests. Calibration refers to the agreement between observed endpoints and probabilities predicted by the model.39 Calibration can be assessed graphically by plotting the observed outcome frequencies against the predicted probabilities. Calibration was also assessed with the Hosmer-Lemeshow goodness-of-fit statistic. Discrimination is the ability of the model to differentiate between patients with and without an unfavorable dizziness course. Discrimination was assessed with the area under the receiver operating characteristic curve (AUC). To estimate the potential for overfitting and optimism in model performance we internally validated the model using bootstrapping techniques.40 Overoptimism (if present) was corrected for by adjusting the regression coefficients with a slope value retrieved on internal validation and by calculating a new intercept thereafter via an offset procedure.
For external validation, we used the fixed coefficients of the developed model and assessed calibration and discrimination of the model in the validation cohort. In case of poor performance we planned to update the model by adjustment of the intercept.41
We assessed model performance in the multiple imputed data sets of the development cohort and validation cohort. Calibration plots, Hosmer-Lemeshow statistics, receiver operating characteristic curves, and AUCs were generated in all separate data sets of the multiple imputed data sets. Subsequently, Hosmer-Lemeshow statistics were pooled, and the median and interquartile range (IQR) of the AUCs were calculated. Calibration plots and receiver operating characteristic curves from separate data sets of the multiple imputed data sets are presented.
Risk Score
We converted the prediction model to a risk score by dividing each regression coefficient by the lowest regression coefficient. Agreement between the prediction model and the risk score was assessed with the Pearson correlation coefficient. Sensitivity, specificity, and predictive values were calculated for different risk score cut-off values to determine the optimal cut-off point for daily clinical practice. A cut-off point was generated from the validation sample as this cohort comprises patients the primary care physician will encounter in daily practice: dizzy patients having mild to severe dizziness-related impairment. The cut-off point was set at a score with a high specificity because we wanted to avoid large numbers of false-positives.
Statistical analyses were performed with SPSS 22.0 (SPSS Inc) and R software (R Foundation for Statistical Computing). For reporting this prediction and validation study, we used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement.42
The study was approved by the Medical Ethics Review Committee of VU University Medical Center Amsterdam (approval number: NL49604.029.14).
RESULTS
Patients and Model Development
Prevalence of the outcome, that is, an unfavorable course of dizziness at 6 months of follow-up, was 73.9% among the 203 patients in the development cohort and 43.6% among the 415 patients in the validation cohort. Table 1 provides an overview of characteristics for both cohorts. For 6 potential candidate predictors in the development cohort, values were missing (≤4% missing for the DHI-S score, any psychiatric illness, depressive disorder, anxiety disorder, and impaired functional mobility; 9.4% missing for the DHI score [Supplemental Appendix 1]). Outcome information was missing for 11.4% of patients in the development cohort and for 9.6% of patients in the validation cohort.
Ten candidate predictors were associated with the outcome in univariate analysis (Supplemental Appendix 1). Two of these 10 candidate predictors were the DHI score and the DHI-S score, both of which quantify dizziness-related impairment. Because the DHI score and the DHI-S score both measure the same concept, we choose to construct 2 models: 1 with the DHI score and 1 with the DHI-S score. As both models performed almost equally, we chose the model including the DHI-S score for feasibility reasons.
Predictors in the final prediction model were the DHI-S score, age, history of arrhythmia, and looking up as provoking factor (Table 2). History of arrhythmia and looking up as provoking factor were assessed during an interview with the questions “Do you have a history of arrhythmia?” and “Does looking up provoke your dizziness?” Table 3 shows the prevalence of the predictors in the development and validation cohorts. The prediction model with the DHI as a continuous instead of a dichotomous outcome is presented in Supplemental Appendix 2 (http://www.annfammed.org/content/16/5/428/suppl/DC1).
Model Performance
Calibration of the prediction model was adequate, with a nonsignificant Hosmer-Lemeshow statistic (P = .63). The calibration plot showed good agreement between predicted and observed numbers of patients having an unfavorable course of dizziness (Supplemental Appendix 3A, http://www.annfammed.org/content/16/5/428/suppl/DC1). Discriminative ability of the model was fair, with an AUC of 0.77 (IQR, 0.76–0.78). On internal validation, the model also performed adequately (Hosmer-Lemeshow statistic P = .75, AUC = 0.75; IQR, 0.74–0.75). We corrected for overoptimism of the original model by multiplying all regression coefficients with a slope value of 0.90 and subtracting 0.698 from the intercept.
On external validation, performance of the adjusted model remained stable (AUC = 0.78; IQR, 0.78–0.79) but calibration deteriorated (Supplemental Appendix 3B, http://www.annfammed.org/content/16/5/428/suppl/DC1). Yet, recalibration with adjustment of the intercept (intercept of –1.08) resulted in good calibration (Supplemental Appendix 3C, http://www.annfammed.org/content/16/5/428/suppl/DC1, Hosmer-Lemeshow statistic P = .83). The discriminative ability of the recalibrated model did not change.
Risk Score
The constructed risk score is presented in Table 4. We found strong correlations between the model-predicted probabilities and the risk score in both the development cohort (r = 0.96, P ≤.001) and the validation cohort (r = 0.97, P ≤.001). Table 5 and Supplemental Appendix 4 (http://www.annfammed.org/content/16/5/428/suppl/DC1) provide overviews of risk score thresholds and their respective classification measures. A score of 134 or higher corresponds to a high risk of an unfavorable course of dizziness with a specificity of 91.9%. Applying the risk score with a threshold of 134 or higher to the validation cohort, the pretest risk of 43.6% for an unfavorable course of dizziness rises to a posttest risk of 77.1%, without many patients having false-positive results.
DISCUSSION
Summary
Our aim was to develop and externally validate a prediction model for a 6-month unfavorable course of dizziness among older patients in primary care. On the basis of this prediction model, we constructed an easy-to-use risk score for identifying patients at high risk for an unfavorable course of dizziness. The developed prediction model and accompanying risk score consist of 4 easily obtainable predictors: DHI-S score, age, history of arrhythmia, and looking up as a provoking factor. The prediction model performed well, with adequate calibration and fair discrimination (AUC = 0.77). In the external cohort, discriminative ability of the model remained stable (AUC = 0.78). The risk score was strongly correlated with the prediction model (r = 0.96, P ≤.001) and will enable primary care physicians to easily identify patients at high risk for an unfavorable course of dizziness.
Strengths and Limitations
An important strength of this study is that the developed prediction model and accompanying risk score were built with readily available variables. With only 4 predictors, the model discriminates fairly well between patients with and without high risk for an unfavorable course of dizziness. The primary care physician can derive 1 of the 4 predictors (age) from the patients’ medical chart and 2 predictors by asking short questions (history of arrhythmia and looking up as provoking factor). The fourth predictor, DHI-S score, is based on a short questionnaire with 10 multiple-choice questions.
Another strength of this study is the external validation of our prediction model. It is essential to evaluate a model’s predictive performance in an independent data set because model performance is generally lower in a new data set than in the initial population.43 Moreover, our study showed stable model performance on external validation in a different but related patient subgroup, also referred to as domain validation, which is considered the strongest evidence for the generalizability of a prediction model to new patients.44 We purposely selected this subgroup with various dizziness-related impairment for external validation because we consider it to be better representative of the population visiting primary care physicians, in contrast to the development cohort, which included only patients with severe dizziness-related impairment.
This study also has some limitations. First, the development cohort was a selected population of patients willing to participate in a cluster-randomized trial (the RODEO study).28 Dizzy patients who declined to participate in the trial might, for example, have been older, with more comorbidities or less dizziness. Second, as 73.9% of the patients in the development cohort had an unfavorable course of dizziness (the outcome), the number of patients without the outcome was relatively small. We therefore internally validated the prediction model and corrected for overoptimism. Third, there was a considerable loss to follow-up, that is, missing data on the outcome. Exclusion of patients with missing values may cause biased results and decreases statistical efficiency.36 We therefore applied a multiple imputation technique in cases where the outcome was missing.
Comparison With Existing Literature
Literature on the prognosis of dizziness in older patients is scarce: only 1 previous study, by Dros et al,24 has presented a prediction model to assess dizziness prognosis in older people. In the current study, we identified dizziness-related impairment (based on DHI-S score) as a strong predictor for an unfavorable course of dizziness. Dros et al24 chose to exclude the DHI score from their final model because this score does not provide guidance for clinical interventions. By ignoring the strongest predictor, there was room in their model for other predictors having a smaller predictive ability for identifying patients with an unfavorable course of dizziness. This difference explains why our prediction model including the DHI-S score contains a different set of predictors.
Three other prospective cohort studies have examined predictors of persistent dizziness in older people.21,23,25 Two of those studies focused on predictors of dizziness in the long term (3- to 10-year follow-up), however.23,25 The third study investigated only univariate associations between potential predictors and dizziness.21 Our study distinguishes itself from the others by including only patients with substantial dizziness-related impairment. This inclusion criterion, together with differences in design and statistical methods, might explain why our final prediction model did not include previously identified predictors such as impaired mobility,21,23,24 depression,23,24 anxiety,23,24 and multiple prescribed drugs.21,23–25
Implications for Research and Practice
With the presented risk score, primary care physicians will be able to easily detect older dizzy patients at high risk for an unfavorable course of dizziness. This identification could prompt the physician to consider targeting factors potentially contributing to dizziness in these high-risk patients, even when the cause is unclear.9,13,20–27 Two ongoing trials are investigating the effectiveness of a multifactorial intervention for factors contributing to dizziness in older adults.28,45 Future results of these studies may provide more specific guidance on multifactorial therapy that is effective in reducing dizziness symptoms.
Acknowledgments
We thank all patients and primary care physicians who participated in the RODEO study.
Footnotes
Conflicts of interest: authors report none.
Funding support: This research project is funded by the Netherlands Organisation for Health Research and Development (project no. 839110001).
Disclaimer: The funding body did not have any role in trial design, collection, analysis, or interpretation of data, or in writing the manuscript.
Supplementary materials: Available at http://www.AnnFamMed.org/content/16/5/428/suppl/DC1/.
- Received for publication December 30, 2017.
- Revision received June 1, 2018.
- Accepted for publication June 28, 2018.
- © 2018 Annals of Family Medicine, Inc.