Automatic Detection of Osteoporosis in Electronic Consultation (eConsult) Service Using Natural Language Processing

Context The primary care practitioners’ (PCP) perceptions of disease given patients’ conditions are reflected in their narrated questions to specialists while using eConsult, an online primary care service.

Objective The aim is to analyze the feasibility of natural language processing (NLP) to train and automatically detect osteo from endocrinology cases in eConsult, to use primary care providers’ text data, provide explanations and help route endo cases to the more appropriate MD.

Study Design and Analysis We set up NLP experimentations and used multiple transformations of text to vector space, such as normalized bag-of-words (TFIDF) with machine learning algorithms. We then used pretrained Language Model (BERT) to extract important terms associated with contrasting segments of data.

Setting or Dataset Out of 844 size sample of endocrinology cases, our human expert annotated only 144 as true osteoporosis cases, flagged as a binary target. We took a random sample of 144 endo cases as non-osteo.

Population Studied Population is a sample of the 2019-20 of endocrinology cases off the Champlain eConsult BASE™ system.

Intervention/Instrument eConsult is an online system that facilitates more timely access to specialist advice and eliminates a significant portion of patients that otherwise required in-person visits with specialists.

Outcome Measures We used a combination of evaluation indices, such as accuracy, precision, recall, to decide the optimum feature engineering approach and model.

Results Training sample from eConsult, TFIDF transformation with the Stochastic Gradient Descent model could distinguish osteoporosis cases by 91% accuracy. Large language model extracted important semantic patterns associated with osteoporosis cases in contrast with endocrinology such as risk of fracture, bisphosphonate, calcium hypercalcemia, prolia denosumab against insulin, diabetes, thyroid, testosterone, cortisol, adrenal, urine adenoma, prolactin, ozempic, glucose respectively. Inherited gender bias observed in osteoporosis cases from the training samples remained constant in test of up to 15% more female. Model predicted up to 19.3% of the total endocrinology cases as being osteoporosis.

Conclusion Osteoporosis narrations had sufficient semantic properties for the machine to learn and accurately distinguish them from endocrinology cases. Latent extracted terms and conceptual symptoms helped users gain confidence in AI, better informed diagnostics, PCPs understandings.