Automatic Dementia Identification for eConsult Service Users by Explainable-AI (XAI) and Natural Language Processing

Context Proactive detection of dementia cases facilitates research for potential interventions, informs the caregivers and prognosis. Primary care providers (PCP) formulate the questions to the specialist through the eConsult online system.

Objective The XAI paradigm and Natural Language Processing (NLP) helps detect whether the eConsult instance is a dementia case, plus it provides with the associated topics that were otherwise latent in the text.

Study Design and Analysis We used data transformations such as TFIDF, and multiple NLP algorithms such as SGD to build a classifier (supervised) model. For explanations, we used language models (LM) such as BERT to cluster the dementia cases by their semantic similarities.

Setting or Dataset Our physicians and experts sub-selected and annotated 199 as gold-standard cases that regarded an authentic dementia patient, sourced from a searched sample of eConsult cases filtered by list of terms such as “dementia” and “Alz” in the PCP questions. Then we removed such terms to train the machine in the absence of such direct clues. We also took an equal size unbiased random sample as a control group of non-dementia to makeup a total balanced of 398 contrasting cases as training data with binary target.

Population Studied Population includes completed Champlain eConsult BASE™ cases in 2021 of patients aged 65 years or older.

Intervention/Instrument eConsult not only provides care more efficiently but is the intervention instrument to take away the need for dementia patients’ personal visits.

Outcome Measures We evaluated the results using accuracy, precision, recall, F1 scores that measure the frequency of true-positives and true-negatives, comparing predictions with the actual labels.

Results We proved NLP was feasible to automatically detect dementia cases using expert annotations of dementia cases vs. control sample in eConsult text communications. Supervised SGD classifier with TF-IDF features was our champion model that achieved 83% of micro-averaged F1 score against an unseen test set.

Language model (XAI) revealed important topics associated with dementia cases such as lesion, wound, pain, fracture, bleeding risk, apixaban.

Conclusion NLP model helped identify dementia cases automatically with high accuracies. LM helped extract latent information embedded in PCP’s reflections on dementia patients. We showed that aggregation of eConsult dementia cases might inform the prognosis of dementia, an epidemiological gap.