Recalibration and update of a Hong Kong Chinese non-laboratory-based risk model to estimate prediabetes risk in primary care

Context We validated a new non-laboratory-based logistic regression risk prediction model for the early detection of prediabetes (pre-DM) in an external primary care (PC) study population in Hong Kong. While it showed good discriminatory power, it produced significantly lower risk estimates than the observed incidence. The model was poorly calibrated and could be updated to optimize its clinical utility in PC.

Objective To test whether recalibrating and/or updating methods could improve the model’s accuracy in estimating pre-DM/DM risks in PC in Hong Kong.

Study Design In a random 1:1 ratio, we split the data of the PC study population into a training set and a test set. We updated the model based on the training set and evaluated the performance of the updated models using the test set.

Dataset A study population of 919 Chinese adults recruited from public or private PC clinics in Hong Kong. It contained participants’ anthropometric and lifestyle data for risk estimation and blood tests results for diagnosis of pre-DM/DM.

Population Studied Hong Kong Chinese adults aged between 18-84 years who did not have a prior diagnosis of DM were considered as eligible.

Intervention/Instrument A seven-step methodology (two methods on recalibration, two methods that revised the regression coefficients of the model’s predictors and three methods that extended the model with additional predictors that were unavailable during its development, i.e. the presence of a family history of DM or the amount of vegetables consumed weekly) was applied to update the model.

Outcome Measures The updated model’s discrimination by area under the receiver operating characteristic curves (AUC-ROC) and its calibration by calibration plots that showed the concordance between estimated outcome probabilities against observed outcome.

Results We found that recalibrating the model by its recalibration intercept was sufficient to enhance the accuracy in estimating pre-DM/DM risks in the PC population. Extensive methods, i.e. model revision and extension, did not improve its estimation accuracy any more than recalibration. None of the methods improved the model’s discrimination in case finding of pre-DM/DM when applied to the test set.

Conclusions A simple update by recalibrating the model’s intercept to cater for difference in prevalence between the original and target populations was sufficient to improve its accuracy in estimating pre-DM/DM risk in the Hong Kong PC population.